B2G/QA/Automation/UI/Minimized Acceptance Execution: Difference between revisions
< B2G | QA | Automation | UI
Jump to navigation
Jump to search
(Minor fixes) |
(Small fixes on Geo) |
||
| Line 5: | Line 5: | ||
* Solid suites = Super sanity, sanity, smoketests | * Solid suites = Super sanity, sanity, smoketests | ||
* No more report investigation every day, the team (or more) is alerted by email sent by Jenkins. | * No more report investigation every day, the team (or more) is alerted by email sent by Jenkins. | ||
* Workflow: http://mzl.la/1vFYdTD *TODO: replace this by an actual SVG when finalized* | * Workflow: http://mzl.la/1vFYdTD *TODO: replace this by an actual SVG when finalized* | ||
| Line 14: | Line 13: | ||
== Flaky trend analysis == | == Flaky trend analysis == | ||
Every | Every triage, the team is in charge to take a look at the trends of failures. If a test got worse from the last triages, a bug to investigate the potential reasons of the failures will be filed. If a test went above 50% *TODO: does this value look correct?* a failures, a bug will be filed to remove it. | ||
= Triages = | = Triages = | ||
Revision as of 17:35, 4 March 2015
This document is currently in work in progress
Solid suites
Objective: Keep the build green, even if there's a current bug in the product.
- Solid suites = Super sanity, sanity, smoketests
- No more report investigation every day, the team (or more) is alerted by email sent by Jenkins.
- Workflow: http://mzl.la/1vFYdTD *TODO: replace this by an actual SVG when finalized*
Flaky suite
Objectives: Make sure the features are working even when they depend from external services..
- A report is generated everyday and somebody is in charge of it.
- Workflow: http://mzl.la/1zlNFEE *TODO: replace this by an actual SVG when finalized*
Flaky trend analysis
Every triage, the team is in charge to take a look at the trends of failures. If a test got worse from the last triages, a bug to investigate the potential reasons of the failures will be filed. If a test went above 50% *TODO: does this value look correct?* a failures, a bug will be filed to remove it.
Triages
On bugzilla, QA Whiteboard: [fxosqa-auto-suite-triage?]
Event: Every week, either during the automation roundtable or another event of half an hour