Add-ons/QA/Testplan/Language Pack Signing

From MozillaWiki
Jump to: navigation, search

Revision History

Date Version Author Description
03/07/2018 1.0 Santa Marius Created first draft

Overview

  • Currently lang packs are signed in Beta and Release, but Firefox doesn't block an unsigned language pack from running. Making Firefox require signed language packs would avoid what has become an exploitable area (unsigned language packs).
  • Unsigned language packs would now trigger "This add-on could not be installed because it has not been verified."

Purpose

  • This document's purpose is to detail the test approach to the Language Packs Signin packaging, including Entry/Exit criteria, Scope for testing, links to testcases etc

Entry Criteria

  • QA has access to PRDs and mocks (from bugs)
  • The feature has landed in -dev

Current Status

  • The feature is in -dev

Exit Criteria

  • All related bugs triaged
  • All blockers fixed
  • All resolved bugs verified by QA
  • Found-fixed bugs rate going down in time

Scope

This section describes what parts of the feature will be tested and what parts won't be.

what's in scope?

  • Lang Packs Signing process
  • Installation of Signed Lang Packs
  • Blocking of Unsigned Lang Pacs

what's out of scope?

  • Extensions functionality

Ownership

Product Manager: Jorge Villalobos; irc nick :jorgev
QA Manager: Krupa Raj; irc nick :krupa
QA Lead: Victor Carciu; irc nick :victorc
Webextensions QA: Marius Santa; irc nick :Santa

Requirements for testing

Environments

  • Windows
  • Mac OS
  • Linux

Servers

Channel dependent settings (configs) and environment setups

  • Nightly
  • Beta
  • Release

Test Strategy

Builds

This section should contain links for builds with the feature -

Test Execution Schedule

The following table identifies the anticipated testing period available for test execution.

Project phase Start Date End Date
Start project 24.08.2015
Study PRD/mocks received
QA - Test plan creation 07.03.2018
QA - Test cases preparation
QA - Test cases execution
Release Date

Testing Tools

Process Tool
Test plan creation Mozilla wiki
Test case creation [TestRail] / [docs]
Test case execution [TestRail]
Bugs management Bugzilla

References

* Webextenstions Language Packs accepted structure  
* Tracker: 1197876

Testcases

Test Areas

  • Installation of Signed/Unsigned Webextensions Lang Packs

Test Areas

Test Areas Covered Details
Installation from AMO
Installation from Add-ons Manager
Installation from Third-party
Language Packs updates
Sideloading
Other

Bug Work

  • Feature bugs
* bug 1395363 - Switch to webextension-langpacks
Full Query
ID Priority Component Assigned to Summary Status Target milestone
922577 -- HTML Bindings Staś Małolepszy :stas Attributes may leak localizations on language change RESOLVED Next
1280901 -- Automation Support ftl/l20n in l10n automation RESOLVED ---
1347798 -- Internationalization Zibi Braniecki [:zbraniecki][:gandalf] Create a small per-document bindings for Localization API RESOLVED mozilla58
1382005 P1 General Axel Hecht [:Pike] Update in-tree version of compare-locales to 2.1 RESOLVED mozilla58
1382538 P2 Pontoon Jarek Śmiejczak [:jotes] [FTL] Upgrade python-fluent to incorporate serializer fixes RESOLVED ---
1394891 P3 Internationalization Zibi Braniecki [:zbraniecki][:gandalf] Add a temporary whitelist of files localized via L20n to control the exposure RESOLVED ---
1395363 -- Internationalization Zibi Braniecki [:zbraniecki][:gandalf] Switch to webextension-langpacks RESOLVED mozilla58
1398723 -- Python Library [tracking] Make python-fluent ready for L20n's Soft Launch milestone RESOLVED ---
1398726 -- JS Library [tracking] Make fluent.js ready for L20n's Soft Launch milestone RESOLVED ---
1399055 -- General Staś Małolepszy :stas Make cross-channel localization ready for Fluent RESOLVED ---
1402069 P3 Internationalization Zibi Braniecki [:zbraniecki][:gandalf] Add a test string using new localization API RESOLVED mozilla58
1410451 -- General Zibi Braniecki [:zbraniecki][:gandalf] Do not take en-US .ftl file if localized file is missing RESOLVED mozilla58
1410731 P3 Internationalization Zibi Braniecki [:zbraniecki][:gandalf] [l20n] Fix selection of packaged locales at runtime RESOLVED mozilla58
1410733 P3 Internationalization Zibi Braniecki [:zbraniecki][:gandalf] Open up LocaleService::GetRequestedLocales to more than one entry RESOLVED mozilla58
1410734 P3 Internationalization Zibi Braniecki [:zbraniecki][:gandalf] Ensure correct runtime fallbacking in L20n API RESOLVED ---
1412751 -- Internationalization Zibi Braniecki [:zbraniecki][:gandalf] Missing multilocale.json in non-packaged builds RESOLVED mozilla58
1413985 P3 Internationalization Zibi Braniecki [:zbraniecki][:gandalf] .ftl files not packaged on repackage RESOLVED mozilla58
1415432 -- Internationalization Zibi Braniecki [:zbraniecki][:gandalf] .ftl files repackaged multiple times RESOLVED ---
1416058 -- Android Studio and Gradle Integration Error when full build - ould not find LLVM/Clang installation RESOLVED ---
1416449 -- Elmo main.ftl is ignored by compare-locales and elmo RESOLVED ---
1425689 -- Android Studio and Gradle Integration Zibi Braniecki [:zbraniecki][:gandalf] Move search-plugin resource entry to fennec's chrome jar.mn RESOLVED mozilla59

21 Total; 0 Open (0%); 21 Resolved (100%); 0 Verified (0%);


Sign off

Criteria

Check list

  • All test cases should be executed
  • All blockers must be fixed and verified or have an agreed-upon timeline for being fixed

Checklist

Exit Criteria Status Notes/Details
Testing Prerequisites (specs, use cases)
Testing Infrastructure setup
Test Plan Creation
Test Cases Creation
Full Functional Tests Execution
Automation Coverage
Performance Testing
All Defects Logged
Critical/Blockers Fixed and Verified
Metrics/Telemetry
QA Signoff - Nightly Release
QA Beta - Full Testing
QA Signoff - Beta Release