QA/Execution/Web Testing/SUMO/Test Plan

From MozillaWiki
< QA‎ | Execution‎ | Web Testing‎ | SUMO
Jump to: navigation, search

SUMO ( Test Plan

Purpose: this document was written to engage community, better foster WebDev/WebQA communication, and (most importantly) to help ensure higher quality releases

Current release


Up-to-date Timeline

  • Dev freeze: March 19, 2010
  • QA start: March 20, 2010
  • Push to production: April 19, 2010

Project Page

Lead QA Engineer(s)

Entry Criteria

  1. QA has access to a staging server, which:
  2. Contains the release-specific changes (has had SQL run, .htaccess updated)

Exit Criteria

  1. All P1s, P2s, and Blocker and Critical bugs have been verified by QA or have been agreed upon as not needing verification
    1. (This gives us leeway in cases where, say, there are critical backend bugs that development files but we can't verify easily, or it's not needed.)
  2. QA has met with or emailed to Web Development (and in some cases, Marketing, Evangelism, etc.) the latest test plan for the current release
  3. QA has apprised WevDev either on IRC or through email of the testing performed, and the outstanding bug list
  4. QA and development have assessed and vetted Litmus-testrun failures
  5. QA signs off on the release
  6. QA assigns an engineer for the push

How you can help with this release (More general Contribute Page)

  • Run Litmus testcases
    • We need help with this from the QA start date until the day before the push (generally)
  • Verify Resolved Fixed bugs
    • This can (and in most cases should) begin as soon as development fixes bugs by marking them as Resolved FIXED, up until the push
  • Ad-hoc test
  • Write automated (Selenium) tests

IRC Channel (real-time chat)

irc:// (New to IRC? Check out to get started quickly!)


Litmus (manual testcases)

Test cases to be run: BFT

Bugs to verify

SUMO 1.5.4 bugs

Automation (Selenium IDE/PHP)

  • Describe scope here
  • Which testcases will be run (some might be broken), where, and when (staging vs. prod), and a sense of what they cover
  • SUMO Selenium testcases



Describe scope here


Describe scope here


Tips and pointers for testing