Add-ons/QA/Testplan/Recommended Extensions in AMO

From MozillaWiki
Jump to: navigation, search

Revision History

Date Version Author Description
04/04/2019 1.0 Alexandra Moga Created first draft
TBD 2.0 Alexandra Moga Updated

Overview

  • In order to increase Firefox extension user security, Mozilla will make a concerted effort to expose users more to a recommended (and more strictly reviewed) set of extensions, and decrease exposure to the rest.
  • AMO will need a mechanism to manage this set of extensions, and highlight them on their product pages and in about:addons.


Goals

  • Provide a more secure extension environment in order to boost user confidence in Mozilla's dedication to user security and privacy. This will be achieved in several steps:
    • Implement new signature types (autograph) for recommended extensions;
    • Create a new review process for recommended addons - new review queue, new statuses and approval methods;
    • Add a visual cue that makes it easy for users to distinguish recommended and non-recommended extensions on AMO.


Entry Criteria

  • Product provides the final feature set-up based on which acceptance criteria can be established
  • QA has access to all the necessary documentation and some visual specifications (found in bugs)
  • The feature has landed in -dev

Current Status

  • The feature is under development

Exit Criteria

  • All related bugs triaged
  • All blockers fixed
  • All resolved bugs verified by QA
  • Found-fixed bugs rate going down in time

Scope

what's in scope?

1. Setting up new signature rules in autograph

  • Regular extensions will continue to be signed by autograph through the current COSE signature, there are no exceptions to their visibility on AMO and their ability to be discovered and installed by users. Recommended extensions need to be a special case - similar to mozilla-signed extensions, namely, they will have a unique signature type that will qualify them for the recommended list.

2. Reviewer Tools - Implement a new review queue

  • Recommended extensions will automatically appear in a dedicated review queue. They will have an identifiable status - "Pending Recommendation → pre-review → approval → Recommended". They will be thoroughly investigated through a manual review process foe each of the new uploaded version. A new addon version can be downgraded from its recommended status if the reviewer finds any digressions from the original scope.


3. Admin Tools - curation mechanism for recommended extensions

  • Similar to what we currently have for TAAR, curators will be have the ability to monitor (add/remove) addons through an admin tool


4. Frontend changes

  • The search API should return recommended addons higher in search results
  • AMO homepage will only feature recommended addons in the guides, categories and collection shelves
  • Recommended addons will have a special badge exposed in search results and on their product page


what's out of scope?

  • Add-ons/Webextensions functionality

Risks

  • This is major change in the way we expose extensions on AMO, since Mozilla is vouching for all the items that reach the recommended list, so this leaves no room for error in extension code reviews;
  • The recommended list needs to be focused on almost every user need, specifically we need to be able to identify suitable extensions for all the major current user interests: privacy, security, password management, user interface, planners etc;
  • Recommended extensions need to be reviewed more carefully and this can extend the time until an addon becomes available to users. Developers need to be informed about this process.

Ownership

Product Manager: Jorge Villalobos; irc nick :jorgev
QA Manager: Krupa Raj; irc nick :krupa
QA Lead: Victor Carciu; irc nick :victorc
Add-ons QA: Alexandra Moga; irc nick :LexaSV

Requirements for testing

Environments

  • Windows
  • Mac OS
  • Android

Servers

Channels

  • Release, ESR

Test Strategy

Test Execution Schedule

The following table identifies the anticipated testing period available for test execution.

Project phase Date
Start project 01.04.2019
Study PRD/mocks received 01.04.2019
QA - Test plan creation 04.04.2019
QA - Test cases preparation 05.04.2019
QA - Test cases execution as features become available
Release Date TBD

Testing Tools

Process Tool
Test plan creation Mozilla wiki
Test case creation TestRail / Google doc
Test case execution TestRail
Bugs management Github

References

Testcases

Test Areas

  • Addon submission and signing
  • Reviewer Tools
  • Admin Tools
  • Installation/Updates
  • Search
  • Addon listings
  • AMO homepage

Test suite

TBD Test Suite.

Sign off

Criteria

Check list

  • All test cases should be executed
  • All blockers must be fixed and verified or have an agreed-upon timeline for being fixed
Retrieved from "https://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Recommended_Extensions_in_AMO&oldid=1210033"