QA/Execution/Web Testing/SUMO/Test Plan: Difference between revisions
< QA | Execution | Web Testing | SUMO
Jump to navigation
Jump to search
No edit summary |
|||
| (12 intermediate revisions by 2 users not shown) | |||
| Line 4: | Line 4: | ||
==Current release== | ==Current release== | ||
[https://wiki.mozilla.org/Support/SUMOdev_Meeting_Notepad 1. | [https://wiki.mozilla.org/Support/SUMOdev_Meeting_Notepad 1.5.3] | ||
https://wiki.mozilla.org/index.php?title=QA/Execution/Web_Testing/SUMO/Test_Plan/2.2] | |||
==Up-to-date Timeline== | ==Up-to-date Timeline== | ||
* Dev freeze: | * Dev freeze: March 19, 2010 | ||
* QA start: | * QA start: March 20, 2010 | ||
* Push to production: | * Push to production: April 19, 2010 | ||
==Project Page== | ==Project Page== | ||
| Line 20: | Line 22: | ||
** Nickname "vish_moz" on [http://widget.mibbit.com/?settings=0a187437b06ca9bcb0cff2a717e985c5&server=irc.mozilla.org&channel=%23sumodev&noServerTab=false&autoConnect=true IRC (via Mibbit.com)] | ** Nickname "vish_moz" on [http://widget.mibbit.com/?settings=0a187437b06ca9bcb0cff2a717e985c5&server=irc.mozilla.org&channel=%23sumodev&noServerTab=false&autoConnect=true IRC (via Mibbit.com)] | ||
== | ==Entry Criteria== | ||
# QA has access to a staging server | # QA has access to a staging server, which: | ||
# Contains the release-specific changes (has had SQL run, .htaccess updated) | |||
==Exit Criteria== | |||
# All P1s, P2s, and Blocker and Critical bugs have been verified by QA or have been agreed upon as not needing verification | # All P1s, P2s, and Blocker and Critical bugs have been verified by QA or have been agreed upon as not needing verification | ||
## (This gives us leeway in cases where, say, there are critical backend bugs that development files but we can't verify easily, or it's not needed.) | ## (This gives us leeway in cases where, say, there are critical backend bugs that development files but we can't verify easily, or it's not needed.) | ||
| Line 30: | Line 35: | ||
# QA assigns an engineer for the push | # QA assigns an engineer for the push | ||
==How you can help with this release ([https://wiki.mozilla.org/QA/Execution/Web_Testing/ | ==How you can help with this release ([https://wiki.mozilla.org/QA/Execution/Web_Testing/Contribute More general Contribute Page])== | ||
* Run [https://wiki.mozilla.org/QA/Execution/Web_Testing/SUMO/Test_Plan#Litmus_.28manual_testcases.29 Litmus testcases] | * Run [https://wiki.mozilla.org/QA/Execution/Web_Testing/SUMO/Test_Plan#Litmus_.28manual_testcases.29 Litmus testcases] | ||
** We need help with this from the QA start date until the day before the push (generally) | ** We need help with this from the QA start date until the day before the push (generally) | ||
| Line 52: | Line 57: | ||
==Bugs to verify== | ==Bugs to verify== | ||
[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&classification=Other&product=support.mozilla.com&target_milestone=1.4 | [https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&classification=Other&product=support.mozilla.com&target_milestone=1.5.4&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=&bug_status=RESOLVED&resolution=FIXED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailqa_contact2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&known_name=SUMO1.5.4&query_based_on=SUMO1.5.4&field0-0-0=noop&type0-0-0=noop&value0-0-0= SUMO 1.5.4 bugs] | ||
==Automation (Selenium IDE/PHP)== | ==Automation (Selenium IDE/PHP)== | ||
* Describe scope here | * Describe scope here | ||
* Which testcases will be run (some might be broken), and a sense of what they cover | * Which testcases will be run (some might be broken), where, and when (staging vs. prod), and a sense of what they cover | ||
* [http://viewvc.svn.mozilla.org/vc/projects/sumo/trunk/tests/frontend/ SUMO Selenium testcases] | * [http://viewvc.svn.mozilla.org/vc/projects/sumo/trunk/tests/frontend/ SUMO Selenium testcases] | ||
==Security== | ==Security== | ||
* [http://ha.ckers.org/xss.html XSS testing] on new inputs? | * [http://ha.ckers.org/xss.html XSS testing] on new inputs? | ||
* SQL Injection testing ( | * SQL Injection testing ("[https://addons.mozilla.org/en-US/firefox/addon/6727 SQL Injection]" add-on, which can be used to test this) | ||
==Accessibility== | ==Accessibility== | ||
| Line 68: | Line 73: | ||
==Localization== | ==Localization== | ||
Describe scope here | Describe scope here | ||
==Resources== | |||
[https://wiki.mozilla.org/QA/Execution/Web_Testing/SUMO/Tips_and_Pointers Tips and pointers for testing] | |||
Latest revision as of 18:17, 14 June 2010
SUMO (support.mozilla.com) Test Plan
Purpose: this document was written to engage community, better foster WebDev/WebQA communication, and (most importantly) to help ensure higher quality releases
Current release
https://wiki.mozilla.org/index.php?title=QA/Execution/Web_Testing/SUMO/Test_Plan/2.2]
Up-to-date Timeline
- Dev freeze: March 19, 2010
- QA start: March 20, 2010
- Push to production: April 19, 2010
Project Page
https://wiki.mozilla.org/Support/SUMOdev_Meeting_Notepad
Lead QA Engineer(s)
- Stephen Donner, stephend@mozilla.com
- Nickname "stephend" on IRC (via Mibbit.com) Mibbit]
- Vishal Kamdar, vkamdar@mozilla.com
- Nickname "vish_moz" on IRC (via Mibbit.com)
Entry Criteria
- QA has access to a staging server, which:
- Contains the release-specific changes (has had SQL run, .htaccess updated)
Exit Criteria
- All P1s, P2s, and Blocker and Critical bugs have been verified by QA or have been agreed upon as not needing verification
- (This gives us leeway in cases where, say, there are critical backend bugs that development files but we can't verify easily, or it's not needed.)
- QA has met with or emailed to Web Development (and in some cases, Marketing, Evangelism, etc.) the latest test plan for the current release
- QA has apprised WevDev either on IRC or through email of the testing performed, and the outstanding bug list
- QA and development have assessed and vetted Litmus-testrun failures
- QA signs off on the release
- QA assigns an engineer for the push
How you can help with this release (More general Contribute Page)
- Run Litmus testcases
- We need help with this from the QA start date until the day before the push (generally)
- Verify Resolved Fixed bugs
- This can (and in most cases should) begin as soon as development fixes bugs by marking them as Resolved FIXED, up until the push
- Ad-hoc test
- Write automated (Selenium) tests
- See the Test Coverage Plan document for existing coverage, and areas of need
IRC Channel (real-time chat)
irc://irc.mozilla.org/#sumodev (New to IRC? Check out http://irc.mozilla.org/ to get started quickly!)
Servers
- Staging server: https://support-stage.mozilla.org
- Production server: https://support.mozilla.com
Litmus (manual testcases)
Test cases to be run: https://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=12&branch_id=30&testgroup_id=153&subgroup_id=1338 BFT
Bugs to verify
Automation (Selenium IDE/PHP)
- Describe scope here
- Which testcases will be run (some might be broken), where, and when (staging vs. prod), and a sense of what they cover
- SUMO Selenium testcases
Security
- XSS testing on new inputs?
- SQL Injection testing ("SQL Injection" add-on, which can be used to test this)
Accessibility
Describe scope here
Localization
Describe scope here