Add-ons/QA/Testplan/Add-ons Post Reviews Process: Difference between revisions
< Add-ons
Jump to navigation
Jump to search
ValentinaP (talk | contribs) |
ValentinaP (talk | contribs) |
||
| Line 10: | Line 10: | ||
= Overview = | = Overview = | ||
* | * Add-ons listed on AMO are all pre-reviewed by a team of employees and volunteers. This process creates problems with waiting times, especially for add-ons escalated for special review, leading to developers to choose self-hosting or not develop for Firefox at all. | ||
We aim to a post-review model (for WebExtensions), where they are approved and published - if they pass a series of automatic tests - and manually post-reviewed. | |||
== | |||
* | == Goal == | ||
* Automatically approve WebExtensions without human intervention; human reviews are done after the add-on has been pushed to public, prioritized depending on various factors. | |||
=== Entry Criteria === | === Entry Criteria === | ||
Revision as of 12:19, 20 June 2017
Revision History
| Date | Version | Author | Description |
|---|---|---|---|
| 20/06/2017 | 1.0 | Valentina Virlics | Created first draft |
Overview
- Add-ons listed on AMO are all pre-reviewed by a team of employees and volunteers. This process creates problems with waiting times, especially for add-ons escalated for special review, leading to developers to choose self-hosting or not develop for Firefox at all.
We aim to a post-review model (for WebExtensions), where they are approved and published - if they pass a series of automatic tests - and manually post-reviewed.
Goal
- Automatically approve WebExtensions without human intervention; human reviews are done after the add-on has been pushed to public, prioritized depending on various factors.
Entry Criteria
Current Status
- The feature is under continuous development
Exit Criteria
- All related bugs triaged
- All blockers fixed
- All resolved bugs verified by QA
- Found-fixed bugs rate going down in time
Scope
what's in scope?
In discussions:
what's out of scope?
- Add-ons/Webextensions functionality
Ownership
Product Manager: Jorge Villalobos; irc nick :jorgev
QA Manager: Krupa Raj; irc nick :krupa
QA Lead: Victor Carciu; irc nick :victorc
Add-ons QA: Valentina Virlics; irc nick :ValentinaV
Requirements for testing
Environments
- Windows
- Mac OS
Servers
- Stage: https://addons.allizom.org/en-US/
- Dev: https://addons-dev.allizom.org/en-US/
- Production: https://addons.mozilla.org/en-US/
Channels
- Release
Test Strategy
Test Execution Schedule
The following table identifies the anticipated testing period available for test execution.
| Project phase | Start Date | End Date |
|---|---|---|
| Start project | 14.03.2017 | |
| Study PRD/mocks received | 20.04.2017 | |
| QA - Test plan creation | 20.06.2017 | |
| QA - Test cases preparation | ||
| QA - Test cases execution | ||
| Release Date | Firefox Release 57 |
Testing Tools
| Process | Tool |
|---|---|
| Test plan creation | Mozilla wiki |
| Test case creation | [TestRail] / [Google docs] |
| Test case execution | [TestRail] |
| Bugs management | Github |
References
- PRD - https://docs.google.com/document/d/1BdiJSYD4aOMClasGxx5wVShHEk8VbxefE3Gf_6jpgkQ/edit#
- Tracker: https://github.com/mozilla/addons-server/issues/5211
- Add-ons Auto Approvals test plan: https://wiki.mozilla.org/Add-ons/QA/Testplan/Auto-Approvals
- Implementation plan for Q2: https://docs.google.com/document/d/1rZvM2QGk8WtkNUDRvLEvrbqZ190hvHfOle4zVRllQmw/edit
- Post-review weights for prioritization: https://docs.google.com/spreadsheets/d/1CZEECHHqEmK87fNiWNslAsFQBL7zxYF-c63-zZVZuqw/edit#gid=0
Testcases
Test Areas
- Review of add-ons
Test suite
Full Test Suite: [here]
Bug Work
- Feature implementation bug - 5210
Bug fix verification
Sign off
Criteria
Check list
- All test cases should be executed
- All blockers must be fixed and verified or have an agreed-upon timeline for being fixed