QA/Execution/Web Testing/SUMO/Test Plan
From MozillaWiki
< QA | Execution | Web Testing | SUMO
SUMO (support.mozilla.com) Test Plan
Purpose: this document was written to engage community, better foster WebDev/WebQA communication, and (most importantly) to help ensure higher quality releases
Contents
- 1 Current release
- 2 Up-to-date Timeline
- 3 Project Page
- 4 Lead QA Engineer(s)
- 5 Entry Criteria
- 6 Exit Criteria
- 7 How you can help with this release (More general Contribute Page)
- 8 IRC Channel (real-time chat)
- 9 Servers
- 10 Litmus (manual testcases)
- 11 Bugs to verify
- 12 Automation (Selenium IDE/PHP)
- 13 Security
- 14 Accessibility
- 15 Localization
- 16 Resources
Current release
https://wiki.mozilla.org/index.php?title=QA/Execution/Web_Testing/SUMO/Test_Plan/2.2]
Up-to-date Timeline
- Dev freeze: March 19, 2010
- QA start: March 20, 2010
- Push to production: April 19, 2010
Project Page
https://wiki.mozilla.org/Support/SUMOdev_Meeting_Notepad
Lead QA Engineer(s)
- Stephen Donner, stephend@mozilla.com
- Nickname "stephend" on IRC (via Mibbit.com) Mibbit]
- Vishal Kamdar, vkamdar@mozilla.com
- Nickname "vish_moz" on IRC (via Mibbit.com)
Entry Criteria
- QA has access to a staging server, which:
- Contains the release-specific changes (has had SQL run, .htaccess updated)
Exit Criteria
- All P1s, P2s, and Blocker and Critical bugs have been verified by QA or have been agreed upon as not needing verification
- (This gives us leeway in cases where, say, there are critical backend bugs that development files but we can't verify easily, or it's not needed.)
- QA has met with or emailed to Web Development (and in some cases, Marketing, Evangelism, etc.) the latest test plan for the current release
- QA has apprised WevDev either on IRC or through email of the testing performed, and the outstanding bug list
- QA and development have assessed and vetted Litmus-testrun failures
- QA signs off on the release
- QA assigns an engineer for the push
How you can help with this release (More general Contribute Page)
- Run Litmus testcases
- We need help with this from the QA start date until the day before the push (generally)
- Verify Resolved Fixed bugs
- This can (and in most cases should) begin as soon as development fixes bugs by marking them as Resolved FIXED, up until the push
- Ad-hoc test
- Write automated (Selenium) tests
- See the Test Coverage Plan document for existing coverage, and areas of need
IRC Channel (real-time chat)
irc://irc.mozilla.org/#sumodev (New to IRC? Check out http://irc.mozilla.org/ to get started quickly!)
Servers
- Staging server: https://support-stage.mozilla.org
- Production server: https://support.mozilla.com
Litmus (manual testcases)
Test cases to be run: https://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=12&branch_id=30&testgroup_id=153&subgroup_id=1338 BFT
Bugs to verify
Automation (Selenium IDE/PHP)
- Describe scope here
- Which testcases will be run (some might be broken), where, and when (staging vs. prod), and a sense of what they cover
- SUMO Selenium testcases
Security
- XSS testing on new inputs?
- SQL Injection testing ("SQL Injection" add-on, which can be used to test this)
Accessibility
Describe scope here
Localization
Describe scope here