QA/Execution/Web Testing/SUMO/Release Checklist

From MozillaWiki
< QA‎ | Execution‎ | Web Testing‎ | SUMO
Jump to: navigation, search

This is the general release checklist we should use to plan, certify, and push SUMO releases.

It is organized by major functional activity in roughly chronological order. At the end of each bullet is the owner of the checklist item.

Team

  • Project lead: David Tenser
  • Dev lead: James Socol
  • QA lead: Rebecca Billings
  • IT lead: Jeremy Orem

Checklist

First things first

  • Meet and schedule release - Entire SUMO dev team
  • Decide on release date - Entire team
    • Update SUMO Meeting Notepad page with correct milestones and target freeze/push dates - Dev lead
    • Email sumo-dev@mozilla.com with proposed schedule - Dev lead
  • Triage of features/bug fixes/nominations, as needed (on-going) - Dev lead / Project lead
    • Schedule meetings - Project lead

Development Phase

  • Development lands all planned patches - Dev lead
  • Development code freeze - Dev lead
    • Hand off to QA for testing/verifications - QA Lead
    • Alert developers to blockers - QA lead

Testing phase -- happens in tandem with the development phase

  • QA tests on staging - QA lead (usually and should happen(s) before tree is tagged)
  • QA runs ad-hoc and Litmus tests, and individual bug verifications for that milestone
  • Triage of blocking/approval requests as needed - Dev lead
    • When signed off, email sumo-dev@mozilla.com with notification - QA lead

Feature development/testing finished, code frozen

  • Dev lead tags the tree - Dev lead
  • Dev lead files the IT push bug - Dev lead, which includes:
    • Tag information
    • Changes to libraries/added dependencies (if any)
    • Needed SQL (if any)
    • Cache-clearing instructions (if any)
  • IT/Dev updates https://wiki.mozilla.org/Releases with the release date
  • Determine release manager, typically the project manager or lead developer. This person will make calls as to whether to rollback.
  • Prior to the release team meets to discuss:
    • Steps for deployment will be talked through and docs checked
    • Roll back plan will be discussed
    • Rollback decision points will be reviewed
    • Downtime notice plans

Push day

  • Team is notified via IT push bug of the push time - IT lead
  • All team members who are responsible for code or infrastructure affected by the release will be present, or the release will not go ahead. - Entire SUMO dev team + IT
  • IT pushes at the agreed-upon time and updates the bug and #sumodev channel of the changes
  • Dev/QA/IT agree it's stable to test - Entire SUMO dev team + IT
  • QA begins testing fixed bugs and running a combination of Litmus tests + automation (eventually -- not there yet w/Kitsune) - QA lead
  • Once QA (and everyone) feels comfortable with the build, they verify the push bug - QA lead