Add-ons/QA/Testplan/Listed-Unlisted Submissions

From MozillaWiki
Jump to: navigation, search

Revision History

Date Version Author Description
01/27/2017 1.0 Valentina Virlics Created first draft
01/30/2017 1.1 Valentina Virlics Updated

Overview

The new add-on submission flow for listed and unlisted add-ons, uploading a new version or a new file, for the same version, together with add-ons with mixed versions, came in a moment when add-on developers needed clarification towards an easy way to finalize the add-on submission and distribution process, and also have multiple distribution channels for their add-ons without any weird workarounds.

Purpose

Simplify the submission flow by reducing the number of steps required to submit an add-on. Have mixed versions (multiple distribution) for the same add-on.

Entry Criteria

  • QA has access to PRD and some mocks (found in bugs)
  • The feature has landed in -dev

Current Status

  • The feature is in production

Exit Criteria

  • All related bugs triaged
  • All blockers fixed
  • All resolved bugs verified by QA
  • Found-fixed bugs rate going down in time

Scope

what's in scope?

  • Add-on distribution
  • Listed add-on submission flow
  • Unlisted add-on submission flow
  • Add-on with mixed versions
  • Add-on update submission flow
  • New file (inside of a version) submission flow
  • Add-ons submission flow in different locales
  • Versions availability in AMO public pages
  • Listing visibility - Visible/Invisible

what's out of scope?

  • Add-ons functionality

Ownership

Product Manager: Jorge Villalobos; irc nick :jorgev
QA Manager: Krupa Raj; irc nick :krupa
QA Lead: Victor Carciu; irc nick :victorc
Add-ons QA: Valentina Virlics; irc nick :ValentinaV

Requirements for testing

Environments

  • Windows
  • Mac OS

Servers

Channels

  • Release

Test Strategy

Test Execution Schedule

The following table identifies the anticipated testing period available for test execution.

Project phase Start Date End Date
Start project 31.08.2016
Study PRD/mocks received 30.09.2016
QA - Test plan creation 03.01.2017 06.01.2017
QA - Test cases preparation 10.01.2017 11.01.2017
QA - Test cases execution 11.01.2017 18.01.2017
Release Date 26.01.2017

Testing Tools

Process Tool
Test plan creation Mozilla wiki
Test case creation TestRail / Google docs
Test case execution TestRail
Bugs management Github

References

* PRD - https://docs.google.com/document/d/1v32JqoLcn_6MfviFXOY0UTq-tigifTllL3yOUyn0-1M/edit
* Tracker: 3465

Testcases

Test Areas

  • Submission of add-ons and webextensions

Test suite

Full Test suite - here

Bug Work

Bug fix verification
  • 4081 - Last step of new file submission mentions "version" submission
  • 4134 - "Lastest version" or "Latest version" ?
  • 3704 - Hide unreviewed versions (if there is at least one reviewed version) on the 'all versions' page
  • 4302 - Misleading info while adding a second beta-listed version for an unlisted add-on
  • 4269 - "Incomplete" status for unlisted add-ons in Dev Hub homepage
  • 4395 - Direct links to beta versions are broken
  • 4282 - Click on incomplete status redirects to final step of submission
  • 4245 - Add-on admin page should point to review-unlisted pages when relevant
  • 4024 - Setting an add-on waiting for approval to invisible leaves it with incorrect add-on status
  • 318 - Old version addon links broken, only static link to latest available
  • 4394 - "Completed" add-on is displayed as incomplete
  • 4282 - Click on incomplete status redirects to final step of submission
  • 4420 - Blocking submission for second listed version in 6 steps
  • 4245 - Add-on admin page should point to review-unlisted pages when relevant
  • 4398 - the (AMO) and (Self) tags aren't localised
  • 4022 - Error displayed while adding a developer reply for second unlisted version of an add-on
  • 4479 - Add-on unlisted review page is returning 500 error for add-ons with mixed unlisted/listed versions
  • 4484 - Search in "All Unlisted Add-ons" is not working for add-on with mixed versions if first version is unlisted
Logged bugs
  • 4217 - Beta-channel tooltip from beta-version submission step is not aligned
  • 4220 - Different format for beta versions used in Add-ons and WebExtensions
  • 4226 - "Review History" links for beta versions are useless
  • 4269 - "Incomplete" status for unlisted add-ons in Dev Hub homepage
  • 4282 - Click on incomplete status redirects to final step of submission
  • 4488 - Handle duplicate version better
  • 4491 - No source-code file available anywhere if added during new file submission
  • 4394 - "Completed" add-on is displayed as incomplete
  • 4302 - Misleading info while adding a second beta-listed version for an unlisted add-on
  • 4420 - Blocking submission for second listed version in 6 steps
  • 4427 - Beta add-on asking for resume after deleting first listed version
  • 4479 - 500 error page is loaded while unlisted version number is clicked from Admin Manage page
  • 4484 - Search in "All Unlisted Add-ons" is not working for add-on with mixed versions if first version is unlisted

Sign off

Criteria

Check list

  • All test cases should be executed
  • All blockers must be fixed and verified or have an agreed-upon timeline for being fixed