Add-ons/QA/Testplan/Language Pack Signing
From MozillaWiki
< Add-ons
Revision History
Date | Version | Author | Description |
---|---|---|---|
03/07/2018 | 1.0 | Santa Marius | Created first draft |
Contents
Overview
- Currently lang packs are signed in Beta and Release, but Firefox doesn't block an unsigned language pack from running. Making Firefox require signed language packs would avoid what has become an exploitable area (unsigned language packs).
- Unsigned language packs would now trigger "This add-on could not be installed because it has not been verified."
Purpose
- This document's purpose is to detail the test approach to the Language Packs Signin packaging, including Entry/Exit criteria, Scope for testing, links to testcases etc
Entry Criteria
- QA has access to PRDs and mocks (from bugs)
- The feature has landed in -dev
Current Status
- The feature is in -dev
Exit Criteria
- All related bugs triaged
- All blockers fixed
- All resolved bugs verified by QA
- Found-fixed bugs rate going down in time
Scope
This section describes what parts of the feature will be tested and what parts won't be.
what's in scope?
- Lang Packs Signing process
- Installation of Signed Lang Packs
- Blocking of Unsigned Lang Pacs
what's out of scope?
- Extensions functionality
Ownership
Product Manager: Jorge Villalobos; irc nick :jorgev
QA Manager: Krupa Raj; irc nick :krupa
QA Lead: Victor Carciu; irc nick :victorc
Webextensions QA: Marius Santa; irc nick :Santa
Requirements for testing
Environments
- Windows
- Mac OS
- Linux
Servers
- Stage: https://addons.allizom.org/en-US/
- Dev: https://addons-dev.allizom.org/en-US/
- Production: https://addons.mozilla.org/en-US/
Channel dependent settings (configs) and environment setups
- Nightly
- Beta
- Release
Test Strategy
Builds
This section should contain links for builds with the feature -
- Nightly builds
- Beta builds
- Beta Unbranded builds:
- Release builds
- Release Unbranded builds:
Test Execution Schedule
The following table identifies the anticipated testing period available for test execution.
Project phase | Start Date | End Date |
---|---|---|
Start project | 24.08.2015 | |
Study PRD/mocks received | ||
QA - Test plan creation | 07.03.2018 | |
QA - Test cases preparation | ||
QA - Test cases execution | ||
Release Date |
Testing Tools
Process | Tool |
---|---|
Test plan creation | Mozilla wiki |
Test case creation | [TestRail] / [docs] |
Test case execution | [TestRail] |
Bugs management | Bugzilla |
References
* Webextenstions Language Packs accepted structure * Tracker: 1197876
Testcases
Test Areas
- Installation of Signed/Unsigned Webextensions Lang Packs
Test Areas
Test Areas | Covered | Details |
---|---|---|
Installation from AMO | ||
Installation from Add-ons Manager | ||
Installation from Third-party | ||
Language Packs updates | ||
Sideloading | ||
Other |
Bug Work
- Feature bugs
* bug 1395363 - Switch to webextension-langpacks
ID | Priority | Component | Assigned to | Summary | Status | Target milestone |
---|---|---|---|---|---|---|
922577 | -- | HTML Bindings | Staś Małolepszy :stas | Attributes may leak localizations on language change | RESOLVED | Next |
1280901 | -- | Automation | Support ftl/l20n in l10n automation | RESOLVED | --- | |
1347798 | -- | Internationalization | Zibi Braniecki [:zbraniecki][:gandalf] | Create a small per-document bindings for Localization API | RESOLVED | mozilla58 |
1382005 | P1 | General | Axel Hecht [:Pike] | Update in-tree version of compare-locales to 2.1 | RESOLVED | mozilla58 |
1382538 | P2 | Pontoon | Jarek Śmiejczak [:jotes] | [FTL] Upgrade python-fluent to incorporate serializer fixes | RESOLVED | --- |
1394891 | P3 | Internationalization | Zibi Braniecki [:zbraniecki][:gandalf] | Add a temporary whitelist of files localized via L20n to control the exposure | RESOLVED | --- |
1395363 | -- | Internationalization | Zibi Braniecki [:zbraniecki][:gandalf] | Switch to webextension-langpacks | RESOLVED | mozilla58 |
1398723 | -- | Python Library | [tracking] Make python-fluent ready for L20n's Soft Launch milestone | RESOLVED | --- | |
1398726 | -- | JS Library | [tracking] Make fluent.js ready for L20n's Soft Launch milestone | RESOLVED | --- | |
1399055 | -- | General | Staś Małolepszy :stas | Make cross-channel localization ready for Fluent | RESOLVED | --- |
1402069 | P3 | Internationalization | Zibi Braniecki [:zbraniecki][:gandalf] | Add a test string using new localization API | RESOLVED | mozilla58 |
1410451 | -- | General | Zibi Braniecki [:zbraniecki][:gandalf] | Do not take en-US .ftl file if localized file is missing | RESOLVED | mozilla58 |
1410731 | P3 | Internationalization | Zibi Braniecki [:zbraniecki][:gandalf] | [l20n] Fix selection of packaged locales at runtime | RESOLVED | mozilla58 |
1410733 | P3 | Internationalization | Zibi Braniecki [:zbraniecki][:gandalf] | Open up LocaleService::GetRequestedLocales to more than one entry | RESOLVED | mozilla58 |
1410734 | P3 | Internationalization | Zibi Braniecki [:zbraniecki][:gandalf] | Ensure correct runtime fallbacking in L20n API | RESOLVED | --- |
1412751 | -- | Internationalization | Zibi Braniecki [:zbraniecki][:gandalf] | Missing multilocale.json in non-packaged builds | RESOLVED | mozilla58 |
1413985 | P3 | Internationalization | Zibi Braniecki [:zbraniecki][:gandalf] | .ftl files not packaged on repackage | RESOLVED | mozilla58 |
1415432 | -- | Internationalization | Zibi Braniecki [:zbraniecki][:gandalf] | .ftl files repackaged multiple times | RESOLVED | --- |
1416058 | -- | Android Studio and Gradle Integration | Error when full build - ould not find LLVM/Clang installation | RESOLVED | --- | |
1416449 | -- | Elmo | main.ftl is ignored by compare-locales and elmo | RESOLVED | --- | |
1425689 | -- | Android Studio and Gradle Integration | Zibi Braniecki [:zbraniecki][:gandalf] | Move search-plugin resource entry to fennec's chrome jar.mn | RESOLVED | mozilla59 |
21 Total; 0 Open (0%); 21 Resolved (100%); 0 Verified (0%);
Sign off
Criteria
Check list
- All test cases should be executed
- All blockers must be fixed and verified or have an agreed-upon timeline for being fixed
Checklist
Exit Criteria | Status | Notes/Details |
---|---|---|
Testing Prerequisites (specs, use cases) | ||
Testing Infrastructure setup | ||
Test Plan Creation | ||
Test Cases Creation | ||
Full Functional Tests Execution | ||
Automation Coverage | ||
Performance Testing | ||
All Defects Logged | ||
Critical/Blockers Fixed and Verified | ||
Metrics/Telemetry | ||
QA Signoff - Nightly Release | ||
QA Beta - Full Testing | ||
QA Signoff - Beta Release |