Confirmed users
1,504
edits
QA/Execution/Web Testing/SUMO/Test Plan: Difference between revisions
QA/Execution/Web Testing/SUMO/Test Plan (view source)
Revision as of 18:17, 14 June 2010
, 14 June 2010→Current release
No edit summary |
|||
| (76 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
SUMO Test Plan | '''SUMO ([http://support.mozilla.com/en-US/kb/ support.mozilla.com]) Test Plan''' | ||
Purpose: this document was written to engage community, better foster [https://wiki.mozilla.org/Webdev WebDev]/[https://wiki.mozilla.org/QA/Execution/Web_Testing WebQA] communication, and (most importantly) to help ensure higher quality releases | |||
==Current release== | |||
[https://wiki.mozilla.org/Support/SUMOdev_Meeting_Notepad 1.5.3] | |||
https://wiki.mozilla.org/index.php?title=QA/Execution/Web_Testing/SUMO/Test_Plan/2.2] | |||
==Up-to-date Timeline== | |||
* Dev freeze: March 19, 2010 | |||
* QA start: March 20, 2010 | |||
* Push to production: April 19, 2010 | |||
==Project Page== | ==Project Page== | ||
https://wiki.mozilla.org/Support/SUMOdev_Meeting_Notepad | https://wiki.mozilla.org/Support/SUMOdev_Meeting_Notepad | ||
==How you can help== | ==Lead QA Engineer(s)== | ||
* Run Litmus testcases | * Stephen Donner, [mailto:stephend@mozilla.com stephend@mozilla.com] | ||
* Verify Resolved Fixed bugs | ** Nickname "stephend" on [http://widget.mibbit.com/?settings=0a187437b06ca9bcb0cff2a717e985c5&server=irc.mozilla.org&channel=%23sumodev&noServerTab=false&autoConnect=true IRC (via Mibbit.com)] Mibbit] | ||
* Vishal Kamdar, [mailto:vkamdar@mozilla.com vkamdar@mozilla.com] | |||
** Nickname "vish_moz" on [http://widget.mibbit.com/?settings=0a187437b06ca9bcb0cff2a717e985c5&server=irc.mozilla.org&channel=%23sumodev&noServerTab=false&autoConnect=true IRC (via Mibbit.com)] | |||
==Entry Criteria== | |||
# QA has access to a staging server, which: | |||
# Contains the release-specific changes (has had SQL run, .htaccess updated) | |||
==Exit Criteria== | |||
# All P1s, P2s, and Blocker and Critical bugs have been verified by QA or have been agreed upon as not needing verification | |||
## (This gives us leeway in cases where, say, there are critical backend bugs that development files but we can't verify easily, or it's not needed.) | |||
# QA has met with or emailed to Web Development (and in some cases, Marketing, Evangelism, etc.) the latest test plan for the current release | |||
# QA has apprised WevDev either on IRC or through email of the testing performed, and the outstanding bug list | |||
# QA and development have assessed and vetted Litmus-testrun failures | |||
# QA signs off on the release | |||
# QA assigns an engineer for the push | |||
==How you can help with this release ([https://wiki.mozilla.org/QA/Execution/Web_Testing/Contribute More general Contribute Page])== | |||
* Run [https://wiki.mozilla.org/QA/Execution/Web_Testing/SUMO/Test_Plan#Litmus_.28manual_testcases.29 Litmus testcases] | |||
** We need help with this from the QA start date until the day before the push (generally) | |||
* Verify [https://wiki.mozilla.org/QA/Execution/Web_Testing/SUMO/Test_Plan#Bugs_to_verify Resolved Fixed bugs] | |||
** This can (and in most cases should) begin as soon as development fixes bugs by marking them as Resolved FIXED, up until the push | |||
* Ad-hoc test | * Ad-hoc test | ||
* Write automated (Selenium) tests | * Write automated (Selenium) tests | ||
** See the [http://spreadsheets.google.com/ccc?key=0AkOIUMTNpBYKdGVrZFpmbjV0Q1Q3bmdYQXVFeUYzalE&hl=en Test Coverage Plan] document for existing coverage, and areas of need | |||
==IRC Channel== | ==IRC Channel (real-time chat)== | ||
irc://irc.mozilla.org/#sumodev | irc://irc.mozilla.org/#sumodev | ||
(New to IRC? Check out [http://irc.mozilla.org/ http://irc.mozilla.org/] to get started quickly!) | |||
==Servers== | ==Servers== | ||
| Line 19: | Line 54: | ||
==Litmus (manual testcases)== | ==Litmus (manual testcases)== | ||
Test cases: [https://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=12&branch_id=30&testgroup_id=153&subgroup_id=1338 https://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=12&branch_id=30&testgroup_id=153&subgroup_id=1338] | Test cases to be run: [https://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=12&branch_id=30&testgroup_id=153&subgroup_id=1338 https://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=12&branch_id=30&testgroup_id=153&subgroup_id=1338 BFT] | ||
==Bugs to verify== | |||
[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&classification=Other&product=support.mozilla.com&target_milestone=1.5.4&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=&bug_status=RESOLVED&resolution=FIXED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailqa_contact2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&known_name=SUMO1.5.4&query_based_on=SUMO1.5.4&field0-0-0=noop&type0-0-0=noop&value0-0-0= SUMO 1.5.4 bugs] | |||
==Automation (Selenium IDE/PHP)== | ==Automation (Selenium IDE/PHP)== | ||
* Describe scope here | |||
* Which testcases will be run (some might be broken), where, and when (staging vs. prod), and a sense of what they cover | |||
* [http://viewvc.svn.mozilla.org/vc/projects/sumo/trunk/tests/frontend/ SUMO Selenium testcases] | |||
==Security== | ==Security== | ||
* [http://ha.ckers.org/xss.html XSS testing] on new inputs? | |||
* SQL Injection testing ("[https://addons.mozilla.org/en-US/firefox/addon/6727 SQL Injection]" add-on, which can be used to test this) | |||
==Accessibility== | ==Accessibility== | ||
Describe scope here | |||
==Localization== | ==Localization== | ||
Describe scope here | |||
==Resources== | |||
[https://wiki.mozilla.org/QA/Execution/Web_Testing/SUMO/Tips_and_Pointers Tips and pointers for testing] | |||