QA/Execution/Web Testing/SUMO/Test Plan: Difference between revisions

From MozillaWiki
< QA‎ | Execution‎ | Web Testing‎ | SUMO
Jump to navigation Jump to search
(Created page with 'SUMO Test Plan')
 
 
(84 intermediate revisions by 2 users not shown)
Line 1: Line 1:
SUMO Test Plan
'''SUMO ([http://support.mozilla.com/en-US/kb/ support.mozilla.com]) Test Plan'''
 
Purpose: this document was written to engage community, better foster [https://wiki.mozilla.org/Webdev WebDev]/[https://wiki.mozilla.org/QA/Execution/Web_Testing WebQA] communication, and (most importantly) to help ensure higher quality releases
 
==Current release==
[https://wiki.mozilla.org/Support/SUMOdev_Meeting_Notepad 1.5.3]
 
https://wiki.mozilla.org/index.php?title=QA/Execution/Web_Testing/SUMO/Test_Plan/2.2]
 
==Up-to-date Timeline==
* Dev freeze: March 19, 2010
* QA start: March 20, 2010
* Push to production: April 19, 2010
 
==Project Page==
https://wiki.mozilla.org/Support/SUMOdev_Meeting_Notepad
 
==Lead QA Engineer(s)==
* Stephen Donner, [mailto:stephend@mozilla.com stephend@mozilla.com]
** Nickname "stephend" on [http://widget.mibbit.com/?settings=0a187437b06ca9bcb0cff2a717e985c5&server=irc.mozilla.org&channel=%23sumodev&noServerTab=false&autoConnect=true IRC (via Mibbit.com)] Mibbit]
* Vishal Kamdar, [mailto:vkamdar@mozilla.com vkamdar@mozilla.com]
** Nickname "vish_moz" on [http://widget.mibbit.com/?settings=0a187437b06ca9bcb0cff2a717e985c5&server=irc.mozilla.org&channel=%23sumodev&noServerTab=false&autoConnect=true IRC (via Mibbit.com)]
 
==Entry Criteria==
# QA has access to a staging server, which:
# Contains the release-specific changes (has had SQL run, .htaccess updated)
 
==Exit Criteria==
# All P1s, P2s, and Blocker and Critical bugs have been verified by QA or have been agreed upon as not needing verification
## (This gives us leeway in cases where, say, there are critical backend bugs that development files but we can't verify easily, or it's not needed.)
# QA has met with or emailed to Web Development (and in some cases, Marketing, Evangelism, etc.) the latest test plan for the current release
# QA has apprised WevDev either on IRC or through email of the testing performed, and the outstanding bug list
# QA and development have assessed and vetted Litmus-testrun failures
# QA signs off on the release
# QA assigns an engineer for the push
 
==How you can help with this release ([https://wiki.mozilla.org/QA/Execution/Web_Testing/Contribute More general Contribute Page])==
* Run [https://wiki.mozilla.org/QA/Execution/Web_Testing/SUMO/Test_Plan#Litmus_.28manual_testcases.29 Litmus testcases]
** We need help with this from the QA start date until the day before the push (generally)
* Verify [https://wiki.mozilla.org/QA/Execution/Web_Testing/SUMO/Test_Plan#Bugs_to_verify Resolved Fixed bugs]
** This can (and in most cases should) begin as soon as development fixes bugs by marking them as Resolved FIXED, up until the push
* Ad-hoc test
* Write automated (Selenium) tests
** See the [http://spreadsheets.google.com/ccc?key=0AkOIUMTNpBYKdGVrZFpmbjV0Q1Q3bmdYQXVFeUYzalE&hl=en Test Coverage Plan] document for existing coverage, and areas of need
 
==IRC Channel (real-time chat)==
irc://irc.mozilla.org/#sumodev
(New to IRC?  Check out [http://irc.mozilla.org/ http://irc.mozilla.org/] to get started quickly!)
 
==Servers==
*Staging server: https://support-stage.mozilla.org
*Production server: https://support.mozilla.com
 
==Litmus (manual testcases)==
 
Test cases to be run: [https://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=12&branch_id=30&testgroup_id=153&subgroup_id=1338 https://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=12&branch_id=30&testgroup_id=153&subgroup_id=1338 BFT]
 
==Bugs to verify==
[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&classification=Other&product=support.mozilla.com&target_milestone=1.5.4&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=&bug_status=RESOLVED&resolution=FIXED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailqa_contact2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&known_name=SUMO1.5.4&query_based_on=SUMO1.5.4&field0-0-0=noop&type0-0-0=noop&value0-0-0= SUMO 1.5.4 bugs]
 
==Automation (Selenium IDE/PHP)==
* Describe scope here
* Which testcases will be run (some might be broken), where, and when (staging vs. prod), and a sense of what they cover
* [http://viewvc.svn.mozilla.org/vc/projects/sumo/trunk/tests/frontend/ SUMO Selenium testcases]
 
==Security==
* [http://ha.ckers.org/xss.html XSS testing] on new inputs?
* SQL Injection testing ("[https://addons.mozilla.org/en-US/firefox/addon/6727 SQL Injection]" add-on, which can be used to test this)
 
==Accessibility==
Describe scope here
 
==Localization==
Describe scope here
 
==Resources==
[https://wiki.mozilla.org/QA/Execution/Web_Testing/SUMO/Tips_and_Pointers Tips and pointers for testing]

Latest revision as of 18:17, 14 June 2010

SUMO (support.mozilla.com) Test Plan

Purpose: this document was written to engage community, better foster WebDev/WebQA communication, and (most importantly) to help ensure higher quality releases

Current release

1.5.3

https://wiki.mozilla.org/index.php?title=QA/Execution/Web_Testing/SUMO/Test_Plan/2.2]

Up-to-date Timeline

  • Dev freeze: March 19, 2010
  • QA start: March 20, 2010
  • Push to production: April 19, 2010

Project Page

https://wiki.mozilla.org/Support/SUMOdev_Meeting_Notepad

Lead QA Engineer(s)

Entry Criteria

  1. QA has access to a staging server, which:
  2. Contains the release-specific changes (has had SQL run, .htaccess updated)

Exit Criteria

  1. All P1s, P2s, and Blocker and Critical bugs have been verified by QA or have been agreed upon as not needing verification
    1. (This gives us leeway in cases where, say, there are critical backend bugs that development files but we can't verify easily, or it's not needed.)
  2. QA has met with or emailed to Web Development (and in some cases, Marketing, Evangelism, etc.) the latest test plan for the current release
  3. QA has apprised WevDev either on IRC or through email of the testing performed, and the outstanding bug list
  4. QA and development have assessed and vetted Litmus-testrun failures
  5. QA signs off on the release
  6. QA assigns an engineer for the push

How you can help with this release (More general Contribute Page)

  • Run Litmus testcases
    • We need help with this from the QA start date until the day before the push (generally)
  • Verify Resolved Fixed bugs
    • This can (and in most cases should) begin as soon as development fixes bugs by marking them as Resolved FIXED, up until the push
  • Ad-hoc test
  • Write automated (Selenium) tests

IRC Channel (real-time chat)

irc://irc.mozilla.org/#sumodev (New to IRC? Check out http://irc.mozilla.org/ to get started quickly!)

Servers

Litmus (manual testcases)

Test cases to be run: https://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=12&branch_id=30&testgroup_id=153&subgroup_id=1338 BFT

Bugs to verify

SUMO 1.5.4 bugs

Automation (Selenium IDE/PHP)

  • Describe scope here
  • Which testcases will be run (some might be broken), where, and when (staging vs. prod), and a sense of what they cover
  • SUMO Selenium testcases

Security

Accessibility

Describe scope here

Localization

Describe scope here

Resources

Tips and pointers for testing

Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Execution/Web_Testing/SUMO/Test_Plan&oldid=230540"