Add-ons/QA/Testplan/Webextensions Language Packs: Difference between revisions

← Older edit

Add-ons/QA/Testplan/Webextensions Language Packs (view source)

Revision as of 16:20, 6 November 2017

3,308 bytes added , 6 November 2017

→‎Testing Tools

Santa

31

edits

@@ Line 6: / Line 6: @@
 |-
 | 09/14/2017 || 1.0 || Valentina Virlics || Created first draft
+|-
+| 10/02/2017 || 1.1 || Marius Santa || Updateds: Overview, Purpose, Scope, Bug Work, Builds, Checklist, Channel
 |}
 = Overview =
-* A language pack is an extension (add-on) that changes the language of the user interface in a Mozilla application (Firefox, Thunderbird, SeaMonkey, etc.).
+* A language pack is an add-on that changes the language of the user interface in a Mozilla application (Firefox, Thunderbird, SeaMonkey, etc.).
-*
+* Switching to the new language pack formats will allow us to remove language packs from requiring the old add-ons ecosystem, they will be safer(no direct access to chrome registry) and will enable L10nRegistry ([https://bugzilla.mozilla.org/show_bug.cgi?id=1333980 bug 1333980]) which is required for transitioning to the new l10n API ([https://bugzilla.mozilla.org/show_bug.cgi?id=1365426 bug 1365426]).
 == Purpose ==
-* Introducing Language Packs that use WebExtensions packaging.
+* This document's purpose is to detail the test approach to the Language Packs Webextensions packaging, including Entry/Exit criteria, Scope for testing, links to testcases etc
-*
 === Entry Criteria ===
@@ Line 30: / Line 31: @@
 == Scope ==
+This section describes what parts of the feature will be tested and what parts won't be.
 ===what's in scope?===
+* Lang Packs validation - recognize the special kind of extension based on some entry in manifest.json
 * Lang Packs submission
 * Lang Packs auto-approval
 * Lang Packs structure and validation
 * Lang Packs installation
-*
 ===what's out of scope?===
@@ Line 45: / Line 48: @@
 QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br />
 Add-ons QA: [mailto:valentina.peleskei@softvision.ro Valentina Virlics]; irc nick :ValentinaV<br />
+Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br />
 Webextensions QA: [mailto:marius.santa@softvision.ro Marius Santa]; irc nick :Santa<br />
@@ Line 58: / Line 62: @@
 * Production: https://addons.mozilla.org/en-US/
-== Channels ==
+== Channel dependent settings (configs) and environment setups ==
-* Release
 * Nightly
 * Beta
+* Release
 = Test Strategy =
 == Builds ==
-This section should contain links for builds with the feature
+This section should contain links for builds with the feature -
-* Link for Release build
+* [http://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central/ Nightly builds]
-* [https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central/ Link] for Nightly builds
+* [https://archive.mozilla.org/pub/firefox/candidates/ Beta builds]
-* [https://archive.mozilla.org/pub/firefox/candidates/ Link] for Beta builds
+* Beta Unbranded builds:
+** [https://archive.mozilla.org/pub/devedition/candidates/ All Builds]
+* [https://archive.mozilla.org/pub/firefox/releases/ Release builds]
+* Release Unbranded builds:
+** [https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-release-win64-add-on-devel/ Windows Link]
+** [https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-release-linux64-add-on-devel/ Linux Link]
+** [https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-release-macosx64-add-on-devel/ Mac Link]
 == Test Execution Schedule ==
@@ Line 102: / Line 112: @@
 | Test plan creation || Mozilla wiki
 |-
-| Test case creation || TestRail / Google docs
+| Test case creation || [https://testrail.stage.mozaws.net/index.php?/suites/view/1703&group_by=cases:section_id&group_order=asc TestRail] / [https://docs.google.com/spreadsheets/d/1r64jOLIjDXi9Tuv30j7MTWS6ujr0MhQ8TCQjTRgAWgc/edit#gid=1 Google docs]
 |-
-| Test case execution || TestRail
+| Test case execution || [https://testrail.stage.mozaws.net/index.php?/runs/view/7005&group_by=cases:section_id&group_order=asc TestRail]
 |-
 | Bugs management || Github
@@ Line 117: / Line 127: @@
 * Submission/Approvals/Installation of Webextensions Lang Packs
-== Test suite ==
+== Test Areas ==
-  Full Test suite
+{| class="wikitable" style="width:80%"
+|-
+! Test Areas !! Covered !! Details
+|-
+| '''Installation from AMO'''
+|style="text-align:center;" | ||
+|-
+| '''Installation from Add-ons Manager'''
+|style="text-align:center;" |  ||
+|-
+| '''Installation from Third-party'''
+|style="text-align:center;" |  ||
+|-
+| '''Language Packs updates'''
+||  ||
+|-
+| '''Sideloading'''
+|style="text-align:center;" |  ||
+|-
+| '''Other'''
+|style="text-align:center;" |  ||
+|-
+|}
 = Bug Work =
@@ Line 126: / Line 158: @@
   - [https://github.com/mozilla/addons-server/issues/6350 6350]
   - [https://github.com/mozilla/addons-server/issues/6360 6360]
-<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
+ * {{Bug|1395363}} - Switch to webextension-langpacks
+<bugzilla>
+    {
+        "blocks":[1395363],
+        "include_fields": "id, priority, component, assigned_to, summary, status, target_milestone"
+    }
+</bugzilla>
-====== Bug fix verification ======
+ * {{Bug|1365031}} - Turn langpacks into webextension-like packages
-<div class="mw-collapsible-content">
+<bugzilla>
-*
+    {
+        "blocks":[1365031],
-</div></div>
+        "include_fields": "id, priority, component, assigned_to, summary, status, target_milestone"
+    }
-<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
+</bugzilla>
-====== Logged bugs ======
-<div class="mw-collapsible-content">
-*
-</div></div>
 = Sign off =
@@ Line 149: / Line 180: @@
 * All test cases should be executed
 * All blockers must be fixed and verified or have an agreed-upon timeline for being fixed
+== Checklist ==
+{| class="wikitable" style="width:60%"
+|-
+! Exit Criteria !! Status !! Notes/Details
+|-
+|  Testing Prerequisites (specs, use cases)
+| style="text-align:center;" |
+| style="text-align:center;" |
+|-
+|  Testing Infrastructure setup
+|style="text-align:center;" |   ||
+|-
+|  Test Plan Creation
+| style="text-align:center;" |   ||
+|-
+|  Test Cases Creation
+|style="text-align:center;" |   ||
+|-
+|  Full Functional Tests Execution
+|style="text-align:center;" |
+|style="text-align:center;" |
+|-
+|  Automation Coverage ||
+|style="text-align:center;" |
+|-
+|  Performance Testing
+|style="text-align:center;" |  ||
+|-
+|  All Defects Logged || ||
+|-
+|  Critical/Blockers Fixed and Verified || ||
+|-
+|  Metrics/Telemetry||
+|style="text-align:center;" |
+|-
+|  QA Signoff  - Nightly Release||
+|style="text-align:center;" |
+|-
+|  QA Beta - Full Testing
+|style="text-align:center;" |  ||
+|-
+|  QA Signoff  - Beta Release ||
+|style="text-align:center;" |
+|}