B2G/QA/Automation/UI/Minimized Acceptance Execution: Difference between revisions
< B2G | QA | Automation | UI
Jump to navigation
Jump to search
(Initial revision) |
(Minor fixes) |
||
| Line 5: | Line 5: | ||
* Solid suites = Super sanity, sanity, smoketests | * Solid suites = Super sanity, sanity, smoketests | ||
* No more report investigation every day, the team (or more) is alerted by email sent by Jenkins. | * No more report investigation every day, the team (or more) is alerted by email sent by Jenkins. | ||
* Corollary: No more messages from Jenkins on IRC. | |||
* Workflow: http://mzl.la/1vFYdTD *TODO: replace this by an actual SVG when finalized* | * Workflow: http://mzl.la/1vFYdTD *TODO: replace this by an actual SVG when finalized* | ||
| Line 17: | Line 18: | ||
= Triages = | = Triages = | ||
On bugzilla, QA Whiteboard: [fxosqa-auto-suite-triage?] | On bugzilla, QA Whiteboard: [fxosqa-auto-suite-triage?] | ||
Event: Every week, either during the automation roundtable or another event of half an hour | Event: Every week, either during the automation roundtable or another event of half an hour | ||
Revision as of 16:22, 27 February 2015
This document is currently in work in progress
Solid suites
Objective: Keep the build green, even if there's a current bug in the product.
- Solid suites = Super sanity, sanity, smoketests
- No more report investigation every day, the team (or more) is alerted by email sent by Jenkins.
- Corollary: No more messages from Jenkins on IRC.
- Workflow: http://mzl.la/1vFYdTD *TODO: replace this by an actual SVG when finalized*
Flaky suite
Objectives: Make sure the features are working even when they depend from external services..
- A report is generated everyday and somebody is in charge of it.
- Workflow: http://mzl.la/1zlNFEE *TODO: replace this by an actual SVG when finalized*
Flaky trend analysis
Every month, a member of the team is in charge to take a look at the trends of failures. If a test got worse from , he will file a bug to investigate the potential reasons of the failures. If a test went above 50% *TODO: does this value look correct?* a failures, he will file a bug and nominate it for removal.
Triages
On bugzilla, QA Whiteboard: [fxosqa-auto-suite-triage?]
Event: Every week, either during the automation roundtable or another event of half an hour