Latest revision as of 16:20, 6 November 2017

Revision History

Date	Version	Author	Description
09/14/2017	1.0	Valentina Virlics	Created first draft
10/02/2017	1.1	Marius Santa	Updateds: Overview, Purpose, Scope, Bug Work, Builds, Checklist, Channel

Overview

A language pack is an add-on that changes the language of the user interface in a Mozilla application (Firefox, Thunderbird, SeaMonkey, etc.).
Switching to the new language pack formats will allow us to remove language packs from requiring the old add-ons ecosystem, they will be safer(no direct access to chrome registry) and will enable L10nRegistry (bug 1333980) which is required for transitioning to the new l10n API (bug 1365426).

Purpose

This document's purpose is to detail the test approach to the Language Packs Webextensions packaging, including Entry/Exit criteria, Scope for testing, links to testcases etc

Entry Criteria

QA has access to PRDs and mocks (from bugs)
The feature has landed in -dev

Current Status

The feature is in -dev

Exit Criteria

All related bugs triaged
All blockers fixed
All resolved bugs verified by QA
Found-fixed bugs rate going down in time

Scope

This section describes what parts of the feature will be tested and what parts won't be.

what's in scope?

Lang Packs validation - recognize the special kind of extension based on some entry in manifest.json
Lang Packs submission
Lang Packs auto-approval
Lang Packs structure and validation
Lang Packs installation

what's out of scope?

Extensions functionality

Ownership

Product Manager: Jorge Villalobos; irc nick :jorgev
QA Manager: Krupa Raj; irc nick :krupa
QA Lead: Victor Carciu; irc nick :victorc
Add-ons QA: Valentina Virlics; irc nick :ValentinaV
Webextensions QA: Cosmin Badescu; irc nick :CosminB
Webextensions QA: Marius Santa; irc nick :Santa

Requirements for testing

Environments

Windows
Mac OS
Linux

Servers

Stage: https://addons.allizom.org/en-US/
Dev: https://addons-dev.allizom.org/en-US/
Production: https://addons.mozilla.org/en-US/

Channel dependent settings (configs) and environment setups

Nightly
Beta
Release

Test Strategy

Builds

This section should contain links for builds with the feature -

Nightly builds
Beta builds
Beta Unbranded builds:
- All Builds
Release builds
Release Unbranded builds:

Test Execution Schedule

The following table identifies the anticipated testing period available for test execution.

Project phase	Start Date	End Date
Start project	17.05.2017
Study PRD/mocks received
QA - Test plan creation	14.09.2017
QA - Test cases preparation
QA - Test cases execution
Release Date

Testing Tools

Process	Tool
Test plan creation	Mozilla wiki
Test case creation	TestRail / Google docs
Test case execution	TestRail
Bugs management	Github

References

* Webextenstions Language Packs accepted structure  
* Tracker: 6297

Testcases

Test Areas

Submission/Approvals/Installation of Webextensions Lang Packs

Test Areas

Test Areas	Covered	Details
Installation from AMO
Installation from Add-ons Manager
Installation from Third-party
Language Packs updates
Sideloading
Other

Bug Work

Feature bugs

- 6297 (traker)  
- 1526  
- 6350  
- 6360

* bug 1395363 - Switch to webextension-langpacks

Full Query

ID	Priority	Component	Assigned to	Summary	Status	Target milestone
1365709	--	Add-ons Manager	Zibi Braniecki [:zbraniecki][:gandalf]	Consume new webextension-based langpacks	RESOLVED	mozilla57
1395456	P3	Internationalization	Zibi Braniecki [:zbraniecki][:gandalf]	Ensure we generate correct min/max versions for the new langpacks	RESOLVED	mozilla57
1395457	P3	Internationalization	Zibi Braniecki [:zbraniecki][:gandalf]	De-hardcode product name when producing webext-langpack	RESOLVED	mozilla57
1395459	P3	Internationalization	Zibi Braniecki [:zbraniecki][:gandalf]	When building webext-langpack pick up language version from the VCS if possible	RESOLVED	mozilla58
1396467	P3	Add-ons Manager	Zibi Braniecki [:zbraniecki][:gandalf]	Add tests for the new langpacks consumption code	RESOLVED	mozilla58
1405264	P2	General	Andrew Swan [:aswan]	webext-langpacks register chrome entries later than old-style langpacks	VERIFIED	mozilla58
1407395	--	Internationalization	Zibi Braniecki [:zbraniecki][:gandalf]	Guard against empty defines entry when generating langpack manifest	RESOLVED	mozilla58
1408380	--	Internationalization	Axel Hecht [:Pike]	localized nightlies broken after switch to webextensions if locale is missing toolkit/defines.inc	RESOLVED	mozilla58
1411951	P1	Internationalization	Zibi Braniecki [:zbraniecki][:gandalf]	Windows langpacks are generated with invalid paths	VERIFIED	mozilla58
1415117	P5	Internationalization		SessionRestore does not update tab title if language changes on restart	NEW	---

10 Total; 1 Open (10%); 7 Resolved (70%); 2 Verified (20%);

* bug 1365031 - Turn langpacks into webextension-like packages

Full Query

ID	Priority	Component	Assigned to	Summary	Status	Target milestone
1333980	--	Internationalization	Zibi Braniecki [:zbraniecki][:gandalf]	Land L10nRegistry	RESOLVED	mozilla57
1365440	--	General	Zibi Braniecki [:zbraniecki][:gandalf]	Generate webextension language packs	VERIFIED	mozilla57
1365709	--	Add-ons Manager	Zibi Braniecki [:zbraniecki][:gandalf]	Consume new webextension-based langpacks	RESOLVED	mozilla57
1377911	--	General	Zibi Braniecki [:zbraniecki][:gandalf]	Remove the override chrome entries from language manifests	RESOLVED	mozilla57
1385544	P3	Internationalization	Zibi Braniecki [:zbraniecki][:gandalf]	Add mozl10n: protocol	RESOLVED	---
1389397	--	Android Studio and Gradle Integration	Zibi Braniecki [:zbraniecki][:gandalf]	Remove the override chrome entries from language manifests (Fennec)	RESOLVED	mozilla57
1393848	--	General	Zibi Braniecki [:zbraniecki][:gandalf]	Move search-plugin resource entry to browser's content jar.mn	RESOLVED	mozilla57
1411951	P1	Internationalization	Zibi Braniecki [:zbraniecki][:gandalf]	Windows langpacks are generated with invalid paths	VERIFIED	mozilla58
1413322	P3	Add-ons Manager	Andrew Swan [:aswan]	Stop loading old-style langpacks	VERIFIED	mozilla58
1425689	--	Android Studio and Gradle Integration	Zibi Braniecki [:zbraniecki][:gandalf]	Move search-plugin resource entry to fennec's chrome jar.mn	RESOLVED	mozilla59

10 Total; 0 Open (0%); 7 Resolved (70%); 3 Verified (30%);

Sign off

Criteria

Check list

All test cases should be executed
All blockers must be fixed and verified or have an agreed-upon timeline for being fixed

Checklist

Exit Criteria	Status	Notes/Details
Testing Prerequisites (specs, use cases)
Testing Infrastructure setup
Test Plan Creation
Test Cases Creation
Full Functional Tests Execution
Automation Coverage
Performance Testing
All Defects Logged
Critical/Blockers Fixed and Verified
Metrics/Telemetry
QA Signoff - Nightly Release
QA Beta - Full Testing
QA Signoff - Beta Release

@@ Line 6: / Line 6: @@
 |-
 | 09/14/2017 || 1.0 || Valentina Virlics || Created first draft
+|-
+| 10/02/2017 || 1.1 || Marius Santa || Updateds: Overview, Purpose, Scope, Bug Work, Builds, Checklist, Channel
 |}
 = Overview =
-* A language pack is an extension (add-on) that changes the language of the user interface in a Mozilla application (Firefox, Thunderbird, SeaMonkey, etc.).
+* A language pack is an add-on that changes the language of the user interface in a Mozilla application (Firefox, Thunderbird, SeaMonkey, etc.).
-* For example, if you have an English version of Thunderbird, then the first button on Thunderbird's toolbar has the label "Get Mail" and the tooltip "Get new messages". But if you install the French language pack and switch the user interface to French, then the first button on Thunderbird's toolbar has the new label "Relever" and the tooltip "Relever les nouveaux messages".
+* Switching to the new language pack formats will allow us to remove language packs from requiring the old add-ons ecosystem, they will be safer(no direct access to chrome registry) and will enable L10nRegistry ([https://bugzilla.mozilla.org/show_bug.cgi?id=1333980 bug 1333980]) which is required for transitioning to the new l10n API ([https://bugzilla.mozilla.org/show_bug.cgi?id=1365426 bug 1365426]).
-*
 == Purpose ==
-* Together with Firefox 57 we must find ourselves using entirely webextensions.
+* This document's purpose is to detail the test approach to the Language Packs Webextensions packaging, including Entry/Exit criteria, Scope for testing, links to testcases etc
-* Being able to use Language Packs after changes land.
 === Entry Criteria ===
-* QA has access to PRD and some mocks (found in bugs)
+* QA has access to PRDs and mocks (from bugs)
 * The feature has landed in [https://addons-dev.allizom.org/en-US/firefox/ -dev]
@@ Line 31: / Line 31: @@
 == Scope ==
+This section describes what parts of the feature will be tested and what parts won't be.
 ===what's in scope?===
-*
+* Lang Packs validation - recognize the special kind of extension based on some entry in manifest.json
+* Lang Packs submission
+* Lang Packs auto-approval
+* Lang Packs structure and validation
+* Lang Packs installation
 ===what's out of scope?===
@@ Line 42: / Line 48: @@
 QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br />
 Add-ons QA: [mailto:valentina.peleskei@softvision.ro Valentina Virlics]; irc nick :ValentinaV<br />
+Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br />
+Webextensions QA: [mailto:marius.santa@softvision.ro Marius Santa]; irc nick :Santa<br />
 = Requirements for testing =
@@ Line 47: / Line 55: @@
 * Windows
 * Mac OS
+* Linux
 == Servers ==
@@ Line 53: / Line 62: @@
 * Production: https://addons.mozilla.org/en-US/
-== Channels ==
+== Channel dependent settings (configs) and environment setups ==
+* Nightly
+* Beta
 * Release
 = Test Strategy =
+== Builds ==
+This section should contain links for builds with the feature -
+* [http://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central/ Nightly builds]
+* [https://archive.mozilla.org/pub/firefox/candidates/ Beta builds]
+* Beta Unbranded builds:
+** [https://archive.mozilla.org/pub/devedition/candidates/ All Builds]
+* [https://archive.mozilla.org/pub/firefox/releases/ Release builds]
+* Release Unbranded builds:
+** [https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-release-win64-add-on-devel/ Windows Link]
+** [https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-release-linux64-add-on-devel/ Linux Link]
+** [https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-release-macosx64-add-on-devel/ Mac Link]
 == Test Execution Schedule ==
 The following table identifies the anticipated testing period available for test execution.
@@ Line 89: / Line 112: @@
 | Test plan creation || Mozilla wiki
 |-
-| Test case creation || TestRail / Google docs
+| Test case creation || [https://testrail.stage.mozaws.net/index.php?/suites/view/1703&group_by=cases:section_id&group_order=asc TestRail] / [https://docs.google.com/spreadsheets/d/1r64jOLIjDXi9Tuv30j7MTWS6ujr0MhQ8TCQjTRgAWgc/edit#gid=1 Google docs]
 |-
-| Test case execution || TestRail
+| Test case execution || [https://testrail.stage.mozaws.net/index.php?/runs/view/7005&group_by=cases:section_id&group_order=asc TestRail]
 |-
 | Bugs management || Github
@@ Line 97: / Line 120: @@
 = References =
-  * PRD -
+  * Webextenstions Language Packs accepted [https://hg.mozilla.org/mozilla-central/rev/960c3df4ced4#l2.12 structure]
   * Tracker: [https://github.com/mozilla/addons-server/issues/6297 6297]
 = Testcases =
@@ Line 104: / Line 127: @@
 * Submission/Approvals/Installation of Webextensions Lang Packs
-== Test suite ==
+== Test Areas ==
-  Full Test suite
+{| class="wikitable" style="width:80%"
+|-
+! Test Areas !! Covered !! Details
+|-
+| '''Installation from AMO'''
+|style="text-align:center;" | ||
+|-
+| '''Installation from Add-ons Manager'''
+|style="text-align:center;" |  ||
+|-
+| '''Installation from Third-party'''
+|style="text-align:center;" |  ||
+|-
+| '''Language Packs updates'''
+||  ||
+|-
+| '''Sideloading'''
+|style="text-align:center;" |  ||
+|-
+| '''Other'''
+|style="text-align:center;" |  ||
+|-
+|}
 = Bug Work =
-* Tracking bug - [https://github.com/mozilla/addons-server/issues/6297 6297]
+* Feature bugs
+ - [https://github.com/mozilla/addons-server/issues/6297 6297] (traker)
-<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
+ - [https://github.com/mozilla/addons-linter/issues/1526 1526]
+ - [https://github.com/mozilla/addons-server/issues/6350 6350]
-====== Bug fix verification ======
+  - [https://github.com/mozilla/addons-server/issues/6360 6360]
-<div class="mw-collapsible-content">
-*
-</div></div>
-<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
+ * {{Bug|1395363}} - Switch to webextension-langpacks
+<bugzilla>
+    {
+        "blocks":[1395363],
+        "include_fields": "id, priority, component, assigned_to, summary, status, target_milestone"
+    }
+</bugzilla>
-====== Logged bugs ======
+ * {{Bug|1365031}} - Turn langpacks into webextension-like packages
-<div class="mw-collapsible-content">
+<bugzilla>
-*
+    {
+        "blocks":[1365031],
-</div></div>
+        "include_fields": "id, priority, component, assigned_to, summary, status, target_milestone"
+    }
+</bugzilla>
 = Sign off =
@@ Line 131: / Line 180: @@
 * All test cases should be executed
 * All blockers must be fixed and verified or have an agreed-upon timeline for being fixed
+== Checklist ==
+{| class="wikitable" style="width:60%"
+|-
+! Exit Criteria !! Status !! Notes/Details
+|-
+|  Testing Prerequisites (specs, use cases)
+| style="text-align:center;" |
+| style="text-align:center;" |
+|-
+|  Testing Infrastructure setup
+|style="text-align:center;" |   ||
+|-
+|  Test Plan Creation
+| style="text-align:center;" |   ||
+|-
+|  Test Cases Creation
+|style="text-align:center;" |   ||
+|-
+|  Full Functional Tests Execution
+|style="text-align:center;" |
+|style="text-align:center;" |
+|-
+|  Automation Coverage ||
+|style="text-align:center;" |
+|-
+|  Performance Testing
+|style="text-align:center;" |  ||
+|-
+|  All Defects Logged || ||
+|-
+|  Critical/Blockers Fixed and Verified || ||
+|-
+|  Metrics/Telemetry||
+|style="text-align:center;" |
+|-
+|  QA Signoff  - Nightly Release||
+|style="text-align:center;" |
+|-
+|  QA Beta - Full Testing
+|style="text-align:center;" |  ||
+|-
+|  QA Signoff  - Beta Release ||
+|style="text-align:center;" |
+|}

Add-ons/QA/Testplan/Webextensions Language Packs: Difference between revisions

Latest revision as of 16:20, 6 November 2017

Contents

Overview

Purpose

Entry Criteria

Current Status

Exit Criteria

Scope

what's in scope?

what's out of scope?

Ownership

Requirements for testing

Environments

Servers

Channel dependent settings (configs) and environment setups

Test Strategy

Builds

Test Execution Schedule

Testing Tools

References

Testcases

Test Areas

Test Areas

Bug Work

Sign off

Criteria

Checklist