B2G/QA/Automation/UI/Minimized Acceptance Execution

This document is currently in work in progress

Solid suites

Objective: Keep the build green, even if there's a current bug in the product.

• Solid suites = Super sanity, sanity, smoketests
• No more report investigation every day, the team (or more) is alerted by email sent by Jenkins.
• Workflow: http://mzl.la/1vFYdTD *TODO: replace this by an actual SVG when finalized*

Flaky suite

Objectives: Make sure the features are working even when they depend from external services..

• A report is generated everyday and somebody is in charge of it.
• Workflow: http://mzl.la/1zlNFEE *TODO: replace this by an actual SVG when finalized*

Flaky trend analysis

Every triage, the team is in charge to take a look at the trends of failures. If a test got worse from the last triages, a bug to investigate the potential reasons of the failures will be filed. If a test went above 50% *TODO: does this value look correct?* a failures, a bug will be filed to remove it.

Triages

On bugzilla, QA Whiteboard: [fxosqa-auto-suite-triage?]

Event: Every week, either during the automation roundtable or another event of half an hour