Add-ons/QA/Testplan/Webextensions Language Packs: Difference between revisions

← Older edit

Add-ons/QA/Testplan/Webextensions Language Packs (view source)

Revision as of 16:20, 6 November 2017

4,303 bytes added , 6 November 2017

→‎Testing Tools

Santa

31

edits

@@ Line 6: / Line 6: @@
 |-
 | 09/14/2017 || 1.0 || Valentina Virlics || Created first draft
+|-
+| 10/02/2017 || 1.1 || Marius Santa || Updateds: Overview, Purpose, Scope, Bug Work, Builds, Checklist, Channel
 |}
 = Overview =
-*
+* A language pack is an add-on that changes the language of the user interface in a Mozilla application (Firefox, Thunderbird, SeaMonkey, etc.).
+* Switching to the new language pack formats will allow us to remove language packs from requiring the old add-ons ecosystem, they will be safer(no direct access to chrome registry) and will enable L10nRegistry ([https://bugzilla.mozilla.org/show_bug.cgi?id=1333980 bug 1333980]) which is required for transitioning to the new l10n API ([https://bugzilla.mozilla.org/show_bug.cgi?id=1365426 bug 1365426]).
 == Purpose ==
-*
+* This document's purpose is to detail the test approach to the Language Packs Webextensions packaging, including Entry/Exit criteria, Scope for testing, links to testcases etc
 === Entry Criteria ===
-* QA has access to PRD and some mocks (found in bugs)
+* QA has access to PRDs and mocks (from bugs)
 * The feature has landed in [https://addons-dev.allizom.org/en-US/firefox/ -dev]
@@ Line 28: / Line 31: @@
 == Scope ==
+This section describes what parts of the feature will be tested and what parts won't be.
 ===what's in scope?===
-*
+* Lang Packs validation - recognize the special kind of extension based on some entry in manifest.json
+* Lang Packs submission
+* Lang Packs auto-approval
+* Lang Packs structure and validation
+* Lang Packs installation
 ===what's out of scope?===
@@ Line 39: / Line 48: @@
 QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br />
 Add-ons QA: [mailto:valentina.peleskei@softvision.ro Valentina Virlics]; irc nick :ValentinaV<br />
+Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br />
+Webextensions QA: [mailto:marius.santa@softvision.ro Marius Santa]; irc nick :Santa<br />
 = Requirements for testing =
@@ Line 44: / Line 55: @@
 * Windows
 * Mac OS
+* Linux
 == Servers ==
@@ Line 50: / Line 62: @@
 * Production: https://addons.mozilla.org/en-US/
-== Channels ==
+== Channel dependent settings (configs) and environment setups ==
+* Nightly
+* Beta
 * Release
 = Test Strategy =
+== Builds ==
+This section should contain links for builds with the feature -
+* [http://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central/ Nightly builds]
+* [https://archive.mozilla.org/pub/firefox/candidates/ Beta builds]
+* Beta Unbranded builds:
+** [https://archive.mozilla.org/pub/devedition/candidates/ All Builds]
+* [https://archive.mozilla.org/pub/firefox/releases/ Release builds]
+* Release Unbranded builds:
+** [https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-release-win64-add-on-devel/ Windows Link]
+** [https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-release-linux64-add-on-devel/ Linux Link]
+** [https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-release-macosx64-add-on-devel/ Mac Link]
 == Test Execution Schedule ==
 The following table identifies the anticipated testing period available for test execution.
@@ Line 86: / Line 112: @@
 | Test plan creation || Mozilla wiki
 |-
-| Test case creation || TestRail / Google docs
+| Test case creation || [https://testrail.stage.mozaws.net/index.php?/suites/view/1703&group_by=cases:section_id&group_order=asc TestRail] / [https://docs.google.com/spreadsheets/d/1r64jOLIjDXi9Tuv30j7MTWS6ujr0MhQ8TCQjTRgAWgc/edit#gid=1 Google docs]
 |-
-| Test case execution || TestRail
+| Test case execution || [https://testrail.stage.mozaws.net/index.php?/runs/view/7005&group_by=cases:section_id&group_order=asc TestRail]
 |-
 | Bugs management || Github
@@ Line 94: / Line 120: @@
 = References =
-  * PRD -
+  * Webextenstions Language Packs accepted [https://hg.mozilla.org/mozilla-central/rev/960c3df4ced4#l2.12 structure]
   * Tracker: [https://github.com/mozilla/addons-server/issues/6297 6297]
 = Testcases =
@@ Line 101: / Line 127: @@
 * Submission/Approvals/Installation of Webextensions Lang Packs
-== Test suite ==
+== Test Areas ==
-  Full Test suite
+{| class="wikitable" style="width:80%"
+|-
+! Test Areas !! Covered !! Details
+|-
+| '''Installation from AMO'''
+|style="text-align:center;" | ||
+|-
+| '''Installation from Add-ons Manager'''
+|style="text-align:center;" |  ||
+|-
+| '''Installation from Third-party'''
+|style="text-align:center;" |  ||
+|-
+| '''Language Packs updates'''
+||  ||
+|-
+| '''Sideloading'''
+|style="text-align:center;" |  ||
+|-
+| '''Other'''
+|style="text-align:center;" |  ||
+|-
+|}
 = Bug Work =
-* Tracking bug - [https://github.com/mozilla/addons-server/issues/6297 6297]
+* Feature bugs
-* All Feature bugs:
+ - [https://github.com/mozilla/addons-server/issues/6297 6297] (traker)
-** Open
+ - [https://github.com/mozilla/addons-linter/issues/1526 1526]
-** Closed
+ - [https://github.com/mozilla/addons-server/issues/6350 6350]
+ - [https://github.com/mozilla/addons-server/issues/6360 6360]
-<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
-====== Bug fix verification ======
-<div class="mw-collapsible-content">
-*
-</div></div>
-<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
+ * {{Bug|1395363}} - Switch to webextension-langpacks
+<bugzilla>
+    {
+        "blocks":[1395363],
+        "include_fields": "id, priority, component, assigned_to, summary, status, target_milestone"
+    }
+</bugzilla>
-====== Logged bugs ======
+ * {{Bug|1365031}} - Turn langpacks into webextension-like packages
-<div class="mw-collapsible-content">
+<bugzilla>
-*
+    {
+        "blocks":[1365031],
-</div></div>
+        "include_fields": "id, priority, component, assigned_to, summary, status, target_milestone"
+    }
+</bugzilla>
 = Sign off =
@@ Line 131: / Line 180: @@
 * All test cases should be executed
 * All blockers must be fixed and verified or have an agreed-upon timeline for being fixed
+== Checklist ==
+{| class="wikitable" style="width:60%"
+|-
+! Exit Criteria !! Status !! Notes/Details
+|-
+|  Testing Prerequisites (specs, use cases)
+| style="text-align:center;" |
+| style="text-align:center;" |
+|-
+|  Testing Infrastructure setup
+|style="text-align:center;" |   ||
+|-
+|  Test Plan Creation
+| style="text-align:center;" |   ||
+|-
+|  Test Cases Creation
+|style="text-align:center;" |   ||
+|-
+|  Full Functional Tests Execution
+|style="text-align:center;" |
+|style="text-align:center;" |
+|-
+|  Automation Coverage ||
+|style="text-align:center;" |
+|-
+|  Performance Testing
+|style="text-align:center;" |  ||
+|-
+|  All Defects Logged || ||
+|-
+|  Critical/Blockers Fixed and Verified || ||
+|-
+|  Metrics/Telemetry||
+|style="text-align:center;" |
+|-
+|  QA Signoff  - Nightly Release||
+|style="text-align:center;" |
+|-
+|  QA Beta - Full Testing
+|style="text-align:center;" |  ||
+|-
+|  QA Signoff  - Beta Release ||
+|style="text-align:center;" |
+|}