Add-ons/QA/Testplan/Add-ons Post Reviews Process: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
Line 9: Line 9:
|}
|}


= Overview =
== Overview ==
* Add-ons listed on AMO are all pre-reviewed by a team of employees and volunteers. This process creates problems with waiting times, especially for add-ons escalated for special review, leading to developers to choose self-hosting or not develop for Firefox at all.
* Add-ons listed on AMO are all pre-reviewed by a team of employees and volunteers. This process creates problems with waiting times, especially for add-ons escalated for special review, leading to developers to choose self-hosting or not develop for Firefox at all.
* We aim to a post-review model (for WebExtensions), where they are approved and published - if they pass a series of automatic tests - and manually post-reviewed.
* We aim to a post-review model (for WebExtensions), where they are approved and published - if they pass a series of automatic tests - and manually post-reviewed.

Revision as of 12:32, 20 June 2017

Revision History

Date Version Author Description
20/06/2017 1.0 Valentina Virlics Created first draft

Overview

  • Add-ons listed on AMO are all pre-reviewed by a team of employees and volunteers. This process creates problems with waiting times, especially for add-ons escalated for special review, leading to developers to choose self-hosting or not develop for Firefox at all.
  • We aim to a post-review model (for WebExtensions), where they are approved and published - if they pass a series of automatic tests - and manually post-reviewed.


Goals

  • Automatically approve WebExtensions without human intervention
    • Decrease the waiting time for publishing a WebExtension or a WebExtension update to less than one day. Currently, waiting times vary between a few hours to several weeks.
    • Improve the value of user feedback channels (abuse reports, ratings on AMO) so they can be used to prioritize post-reviews.
    • Better expose developers to documentation on add-on policies and rules during the submission process.


Entry Criteria

  • QA has access to PRD and some mocks (found in bugs)
  • The feature has landed in -dev

Current Status

  • The feature is under continuous development

Exit Criteria

  • All related bugs triaged
  • All blockers fixed
  • All resolved bugs verified by QA
  • Found-fixed bugs rate going down in time

Scope

what's in scope?

In discussions:

what's out of scope?

  • Add-ons/Webextensions functionality

Ownership

Product Manager: Jorge Villalobos; irc nick :jorgev
QA Manager: Krupa Raj; irc nick :krupa
QA Lead: Victor Carciu; irc nick :victorc
Add-ons QA: Valentina Virlics; irc nick :ValentinaV

Requirements for testing

Environments

  • Windows
  • Mac OS

Servers

Channels

  • Release

Test Strategy

Test Execution Schedule

The following table identifies the anticipated testing period available for test execution.

Project phase Start Date End Date
Start project 14.03.2017
Study PRD/mocks received 20.04.2017
QA - Test plan creation 20.06.2017
QA - Test cases preparation
QA - Test cases execution
Release Date Firefox Release 57


Testing Tools

Process Tool
Test plan creation Mozilla wiki
Test case creation [TestRail] / [Google docs]
Test case execution [TestRail]
Bugs management Github

References

Testcases

Test Areas

  • Review of add-ons

Test suite

Full Test Suite: [here]

Bug Work

  • Feature implementation bug - 5210

Bug fix verification

Sign off

Criteria

Check list

  • All test cases should be executed
  • All blockers must be fixed and verified or have an agreed-upon timeline for being fixed
Retrieved from "https://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Add-ons_Post_Reviews_Process&oldid=1174184"