B2G/QA/Smoke Tests

High Severity Bugs

• Any action that bricks the device.
• Any action that causes the device not to come up again after reboot.
• Any action that causes the device to crash and reboot.
• The following Sanity Test Cases fail without a workaround:
  • [RIL] Correct Radio Interface Layer is installed
  • [OTA] Over The Air update is successful
  • [Homescreen] The Homescreen is viewable
  • [Dialer] Can make an Outbound Call twice
  • [Dialer] Can make an Inbound Call twice
  • [SMS] Can send an Outbound Message twice
  • [SMS] Can receive an Inbound Message twice
  • [Camera] Can take a photo
  • [Camera] Can view the photo taken
  • [Browser] Can browse for 1-2 minutes over WiFi

Note: The telephony tests are done twice to make sure that the process cleans up after itself when the first instance is over so that the second instance is also successful.

Escalation Strategy

• When a High Severity Bug is found, it is reported in Bugzilla with the Whiteboard tag "smoketest".
• A notification email is sent by QAnalysts to the following mailing lists: mozilla-dev-b2g@lists.mozilla.org, dev-b2g@lists.mozilla.org, dev-gaia@lists.mozilla.org and b2g-release-drivers@mozilla.org
• The IRC channel owners of #b2g and #gaia are notified in their respective channels.
• A bug is filed requesting OTA to be blocked.
• A regression window for the root cause of the bug is identified.
• The developer who authored the commit is needinfo?ed with a request to back out the commit as soon as possible.
• The Whiteboard tag "[backout-needed]" is added to the bug.
• The module owner is also needinfo?ed.
• QA tracks the bug until the commit has been backed out. Bugzilla Query for the Whiteboard tag "[backout-needed]": http://mzl.la/1v2bcJJ
• QA requests for a new nightly build to be created.
• When a new build becomes available, QA verifies that the bug no longer reproduces.
• If the bug no longer reproduces, the bug requesting OTA to be blocked is updated.

Smoke Test Blockers

[procedure to follow when Smoke Test Blockers are found]