Add-ons/QA/Testplan/Listed-Unlisted Submissions
Revision History
Date | Version | Author | Description |
---|---|---|---|
01/27/2017 | 1.0 | Valentina Virlics | Created first draft |
01/30/2017 | 1.1 | Valentina Virlics | Updated |
Contents
Overview
The new add-on submission flow for listed and unlisted add-ons, uploading a new version or a new file, for the same version, together with add-ons with mixed versions, came in a moment when add-on developers needed clarification towards an easy way to finalize the add-on submission and distribution process, and also have multiple distribution channels for their add-ons without any weird workarounds.
Purpose
Simplify the submission flow by reducing the number of steps required to submit an add-on. Have mixed versions (multiple distribution) for the same add-on.
Entry Criteria
- QA has access to PRD and some mocks (found in bugs)
- The feature has landed in -dev
Current Status
- The feature is in production
Exit Criteria
- All related bugs triaged
- All blockers fixed
- All resolved bugs verified by QA
- Found-fixed bugs rate going down in time
Scope
what's in scope?
- Add-on distribution
- Listed add-on submission flow
- Unlisted add-on submission flow
- Add-on with mixed versions
- Add-on update submission flow
- New file (inside of a version) submission flow
- Add-ons submission flow in different locales
- Versions availability in AMO public pages
- Listing visibility - Visible/Invisible
what's out of scope?
- Add-ons functionality
Ownership
Product Manager: Jorge Villalobos; irc nick :jorgev
QA Manager: Krupa Raj; irc nick :krupa
QA Lead: Victor Carciu; irc nick :victorc
Add-ons QA: Valentina Virlics; irc nick :ValentinaV
Requirements for testing
Environments
- Windows
- Mac OS
Servers
- Stage: https://addons.allizom.org/en-US/
- Dev: https://addons-dev.allizom.org/en-US/
- Production: https://addons.mozilla.org/en-US/
Channels
- Release
Test Strategy
Test Execution Schedule
The following table identifies the anticipated testing period available for test execution.
Project phase | Start Date | End Date |
---|---|---|
Start project | 31.08.2016 | |
Study PRD/mocks received | 30.09.2016 | |
QA - Test plan creation | 03.01.2017 | 06.01.2017 |
QA - Test cases preparation | 10.01.2017 | 11.01.2017 |
QA - Test cases execution | 11.01.2017 | 18.01.2017 |
Release Date | 26.01.2017 |
Testing Tools
Process | Tool |
---|---|
Test plan creation | Mozilla wiki |
Test case creation | TestRail / Google docs |
Test case execution | TestRail |
Bugs management | Github |
References
* PRD - https://docs.google.com/document/d/1v32JqoLcn_6MfviFXOY0UTq-tigifTllL3yOUyn0-1M/edit * Tracker: 3465
Testcases
Test Areas
- Submission of add-ons and webextensions
Test suite
Full Test suite - here
Bug Work
Bug fix verification
- 4081 - Last step of new file submission mentions "version" submission
- 4134 - "Lastest version" or "Latest version" ?
- 3704 - Hide unreviewed versions (if there is at least one reviewed version) on the 'all versions' page
- 4302 - Misleading info while adding a second beta-listed version for an unlisted add-on
- 4269 - "Incomplete" status for unlisted add-ons in Dev Hub homepage
- 4395 - Direct links to beta versions are broken
- 4282 - Click on incomplete status redirects to final step of submission
- 4245 - Add-on admin page should point to review-unlisted pages when relevant
- 4024 - Setting an add-on waiting for approval to invisible leaves it with incorrect add-on status
- 318 - Old version addon links broken, only static link to latest available
- 4394 - "Completed" add-on is displayed as incomplete
- 4282 - Click on incomplete status redirects to final step of submission
- 4420 - Blocking submission for second listed version in 6 steps
- 4245 - Add-on admin page should point to review-unlisted pages when relevant
- 4398 - the (AMO) and (Self) tags aren't localised
- 4022 - Error displayed while adding a developer reply for second unlisted version of an add-on
- 4479 - Add-on unlisted review page is returning 500 error for add-ons with mixed unlisted/listed versions
- 4484 - Search in "All Unlisted Add-ons" is not working for add-on with mixed versions if first version is unlisted
Logged bugs
- 4217 - Beta-channel tooltip from beta-version submission step is not aligned
- 4220 - Different format for beta versions used in Add-ons and WebExtensions
- 4226 - "Review History" links for beta versions are useless
- 4269 - "Incomplete" status for unlisted add-ons in Dev Hub homepage
- 4282 - Click on incomplete status redirects to final step of submission
- 4488 - Handle duplicate version better
- 4491 - No source-code file available anywhere if added during new file submission
- 4394 - "Completed" add-on is displayed as incomplete
- 4302 - Misleading info while adding a second beta-listed version for an unlisted add-on
- 4420 - Blocking submission for second listed version in 6 steps
- 4427 - Beta add-on asking for resume after deleting first listed version
- 4479 - 500 error page is loaded while unlisted version number is clicked from Admin Manage page
- 4484 - Search in "All Unlisted Add-ons" is not working for add-on with mixed versions if first version is unlisted
Sign off
Criteria
Check list
- All test cases should be executed
- All blockers must be fixed and verified or have an agreed-upon timeline for being fixed