QA/Execution/Web Testing/SUMO/Test Plan

SUMO (support.mozilla.com) Test Plan

Purpose: this document was written to engage community, better foster WebDev/WebQA communication, and (most importantly) to help ensure higher quality releases

Current release

1.4.1

Up-to-date Timeline

• Dev freeze: October 15, 2009
• QA start: October 15, 2009
• Push to production: October 20, 2009

Project Page

https://wiki.mozilla.org/Support/SUMOdev_Meeting_Notepad

Lead QA Engineer(s)

• Stephen Donner
  • • stephend@mozilla.com
    • Nickname "stephend" on IRC ([1])
  • Vishal Kamdar
    • vkamdar@mozilla.com
    • Nickname "vish_moz" on IRC

Release Criteria

1. QA has access to a staging server
2. All P1s, P2s, and Blocker and Critical bugs have been verified by QA or have been agreed upon as not needing verification
3. QA has met with or emailed to Web Development (and in some cases, Marketing, Evangelism, etc.) the latest test plan for the current release
   1. (This gives us leeway in cases where, say, there are critical backend bugs that development files but we can't verify easily, or it's not needed.)
4. QA has apprised WevDev either on IRC or through email of the testing performed, and the outstanding bug list
5. QA and development have assessed and vetted Litmus-testrun failures
6. QA signs off on the release
7. QA assigns an engineer for the push

How you can help with this release (More general Contributor Page)

• Run Litmus testcases
  • We need help with this from the QA start date until the day before the push (generally)
• Verify Resolved Fixed bugs
  • This can (and in most cases should) begin as soon as development fixes bugs by marking them as Resolved FIXED, up until the push
• Ad-hoc test
• Write automated (Selenium) tests
  • See the Test Coverage Plan document for existing coverage, and areas of need

IRC Channel (real-time chat)

irc://irc.mozilla.org/#sumodev (New to IRC? Check out http://irc.mozilla.org/ to get started quickly!)

Servers

• Staging server: https://support-stage.mozilla.org
• Production server: https://support.mozilla.com

Litmus (manual testcases)

Test cases to be run: https://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=12&branch_id=30&testgroup_id=153&subgroup_id=1338 BFT

Bugs to verify

SUMO 1.4.1 bugs

Automation (Selenium IDE/PHP)

• Describe scope here
• Which testcases will be run (some might be broken), and a sense of what they cover
• SUMO Selenium testcases

Security

• XSS testing on new inputs?

Accessibility

Describe scope here

Localization

Describe scope here