B2G/QA/Automation/UI/Strategy/Increase End to end coverage: Difference between revisions
m (Jlorenzo moved page B2G/QA/Automation/UI/Strategy/Increase Acceptance coverage to B2G/QA/Automation/UI/Strategy/Increase End to end coverage: Renaming "acceptance" to end-to-end) |
|||
| (5 intermediate revisions by one other user not shown) | |||
| Line 5: | Line 5: | ||
=Challenges Addressed= | =Challenges Addressed= | ||
* Development and QA have different approaches and needs from UI testing | * Development and QA have different approaches and needs from UI testing | ||
* Acceptance coverage is insufficient to guarantee a quality build | |||
=The Problem= | =The Problem= | ||
| Line 18: | Line 19: | ||
Q2: | Q2: | ||
* Establish JS suites for Gaia Acceptance | |||
Ongoing: | |||
* Triage of new smoketests for Gaia Acceptance backlog | * Triage of new smoketests for Gaia Acceptance backlog | ||
* Triage of dogfood tests for Gaia Acceptance backlog | * Triage of dogfood tests for Gaia Acceptance backlog | ||
* Triage of additional areas for Gaia Acceptance backlog | * Triage of additional areas for Gaia Acceptance backlog | ||
* Additions to JS suites for Gaia Acceptance tests | * Additions to JS suites for Gaia Acceptance tests | ||
| Line 27: | Line 30: | ||
==Risks== | ==Risks== | ||
Given the impending move to JS and the need to stabilize the existing tests per the [[../Streamline | Given the impending move to JS and the need to stabilize the existing tests per the [[../Streamline Acceptance Execution|Streamlining plan]], it doesn't make sense to continue to expand the Python suite. | ||
That means we're gated on [[../Develop Gaia | That means we're gated on [[../Develop Gaia Acceptance|developing Gaia Acceptance]]. Its risks also apply here. | ||
Latest revision as of 17:52, 17 November 2015
Objective
Increase the coverage of our Acceptance tests so we can more quickly and confidently accept builds and raise errors.
Challenges Addressed
- Development and QA have different approaches and needs from UI testing
- Acceptance coverage is insufficient to guarantee a quality build
The Problem
As QA, we have to do a lot of manual regression testing to know that builds remain acceptable. This takes disproportionate amounts of time compared to the benefit of verifying the absence of bugs. By implementing more Acceptance automation, we can increase our impact and free up people for more fruitful exploratory testing.
The Solution
Add any tests that we plan to accept builds on. Don't consider appropriateness for Gaia Integration.
Timeline
Q2:
- Establish JS suites for Gaia Acceptance
Ongoing:
- Triage of new smoketests for Gaia Acceptance backlog
- Triage of dogfood tests for Gaia Acceptance backlog
- Triage of additional areas for Gaia Acceptance backlog
- Additions to JS suites for Gaia Acceptance tests
Risks
Given the impending move to JS and the need to stabilize the existing tests per the Streamlining plan, it doesn't make sense to continue to expand the Python suite.
That means we're gated on developing Gaia Acceptance. Its risks also apply here.