B2G/QA/Automation/UI/Minimized Acceptance Execution

From MozillaWiki
< B2G‎ | QA‎ | Automation‎ | UI
Revision as of 16:16, 27 February 2015 by Jlorenzo (talk | contribs) (Initial revision)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

This document is currently in work in progress

Solid suites

Objective: Keep the build green, even if there's a current bug in the product.

  • Solid suites = Super sanity, sanity, smoketests
  • No more report investigation every day, the team (or more) is alerted by email sent by Jenkins.
  • Workflow: http://mzl.la/1vFYdTD *TODO: replace this by an actual SVG when finalized*

Flaky suite

Objectives: Make sure the features are working even when they depend from external services..

  • A report is generated everyday and somebody is in charge of it.
  • Workflow: http://mzl.la/1zlNFEE *TODO: replace this by an actual SVG when finalized*

Flaky trend analysis

Every month, a member of the team is in charge to take a look at the trends of failures. If a test got worse from , he will file a bug to investigate the potential reasons of the failures. If a test went above 50% *TODO: does this value look correct?* a failures, he will file a bug and nominate it for removal.

Triages

On bugzilla, QA Whiteboard: [fxosqa-auto-suite-triage?] Event: Every week, either during the automation roundtable or another event of half an hour

Retrieved from "https://wiki.mozilla.org/index.php?title=B2G/QA/Automation/UI/Minimized_Acceptance_Execution&oldid=1058725"