Add-ons/QA/Testplan/Add-ons Post Reviews Process

Revision History

Date	Version	Author	Description
20/06/2017	1.0	Valentina Virlics	Created first draft

Overview

Purpose

Entry Criteria

• QA has access to PRD and some mocks (found in bugs)
• The feature has landed in -dev

Current Status

• The feature is under continuous development

Exit Criteria

• All related bugs triaged
• All blockers fixed
• All resolved bugs verified by QA
• Found-fixed bugs rate going down in time

Scope

what's in scope?

• Inform add-on developers of the submission restrictions in advance
• Support submission of new versions for all existing add-ons
• Block new submissions of (after the release of Firefox 53):
  • non-WebExtensions extensions
  • add-ons targeting only Firefox, Firefox for Android, or both
    • listed and unlisted distribution are both affected
• Only accept legacy extensions with strict compatibility and a maxVersion lower than 53.0

In discussions:

• Support submission of add-ons targeting Thunderbird and SeaMonkey
• Block multipackage .xpis

what's out of scope?

• Add-ons/Webextensions functionality

Ownership

Product Manager: Jorge Villalobos; irc nick :jorgev
QA Manager: Krupa Raj; irc nick :krupa
QA Lead: Victor Carciu; irc nick :victorc
Add-ons QA: Valentina Virlics; irc nick :ValentinaV

Requirements for testing

Environments

• Windows
• Mac OS

Servers

• Stage: https://addons.allizom.org/en-US/
• Dev: https://addons-dev.allizom.org/en-US/
• Production: https://addons.mozilla.org/en-US/

Channels

• Release

Test Strategy

Test Execution Schedule

The following table identifies the anticipated testing period available for test execution.

Project phase	Start Date	End Date
Start project	15.12.2016
Study PRD/mocks received	06.01.2017
QA - Test plan creation	03.02.2017
QA - Test cases preparation	06.03.2017
QA - Test cases execution
Release Date	17.04.2017 (Firefox Release 53)

Created a doc with more detailed scenarios - Restrict Legacy Add-ons Submission

Testing Tools

Process	Tool
Test plan creation	Mozilla wiki
Test case creation	TestRail / Google docs
Test case execution	TestRail
Bugs management	Github

References

* PRD - https://docs.google.com/document/d/1BdiJSYD4aOMClasGxx5wVShHEk8VbxefE3Gf_6jpgkQ/edit#
* Tracker: not available yet

Testcases

Test Areas

• Submission of add-ons vs. submission of webextensions

Test suite

Full Test Suite: here

Bug Work

• Feature implementation bug - 4234
• All Feature bugs:
  • Open
  • Closed

Bug fix verification

Logged bugs

Sign off

Criteria

Check list

• All test cases should be executed
• All blockers must be fixed and verified or have an agreed-upon timeline for being fixed