https://wiki.mozilla.org/api.php?action=feedcontributions&user=CosminB&feedformat=atomMozillaWiki - User contributions [en]2024-03-29T01:45:17ZUser contributionsMediaWiki 1.27.4https://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplans&diff=1213433Add-ons/QA/Testplans2019-06-07T12:26:10Z<p>CosminB: /* WebExtensions TestPlans */</p>
<hr />
<div>== QA Contacts == <br />
<br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Add-ons QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
== WebExtensions TestPlans==<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/about:addons_non-xul_rebuild about:addons non-xul rebuild]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_ContentScript Add-ons ContentScript]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-on_Compatibility_Check Add-on Compatibility Check]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/AMO_Process_Isolation AMO Process Isolation]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Android_Webextension_API Android Webextension API]<br />
* [https://wiki.mozilla.org/WebExtensions/chrome.storage.sync chrome.storage.sync]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/CSP CSP]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Context_Menu_Updates Context Menu Updates]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Content_handler_API Content handler API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_Storage_Local_API Improve Storage Local API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_keyboard_shortcut_support Improve keyboard shortcut support]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_support_for_private_browsing_mode Improve support for private browsing mode]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Identifying_recommended_extensions_in_about:addons Identifying recommended extensions in about:addons]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Limited_Tab_Hiding Limited Tab Hiding]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Language_Pack_Signing Language Pack Signing]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/OOP-WebExtensions OOP-Webextensions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Pre-installed_extension_in_Firefox Pre-installed extension in Firefox]<br />
* [https://wiki.mozilla.org/Add-ons/Permissions/Testplan Permissions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Roll_disco_pane_into_about:addons Roll disco pane into about:addons]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/SHA SHA]<br />
* [https://wiki.mozilla.org/QA/Theming/Testplan Theming]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Telemetry_for_add-ons_manager Telemetry for add-ons manager]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Permissions_Android Webextensions Permissions Android]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Language_Packs Webextensions Language Packs]<br />
* [https://wiki.mozilla.org/QA/WebExtensions WebExtensions-Plaform]<br />
* [https://wiki.mozilla.org/Add-ons/Webextensions_Cutover/Testplan WebExtensions Cutover]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Cutover_Android Webextensions Cutover Android]<br />
<br />
== AMO TestPlans==<br />
* [https://wiki.mozilla.org/QA/AMO_WebExtensions Webextension-AMO]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Activity Activity Mail]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Listed-Unlisted_Submissions New Submissions Flow]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Restrict_Legacy_Add-ons_Submission Restrict Legacy Add-ons Submission]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/NewAMOMobile New Mobile Frontend]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Auto-Approvals Auto-Approvals]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_Post_Reviews_Process Post Reviews Process]</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplans&diff=1213432Add-ons/QA/Testplans2019-06-07T12:25:42Z<p>CosminB: /* WebExtensions TestPlans */</p>
<hr />
<div>== QA Contacts == <br />
<br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Add-ons QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
== WebExtensions TestPlans==<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/about:addons_non-xul_rebuild about:addons_non-xul_rebuild]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_ContentScript Add-ons ContentScript]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-on_Compatibility_Check Add-on Compatibility Check]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/AMO_Process_Isolation AMO Process Isolation]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Android_Webextension_API Android Webextension API]<br />
* [https://wiki.mozilla.org/WebExtensions/chrome.storage.sync chrome.storage.sync]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/CSP CSP]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Context_Menu_Updates Context Menu Updates]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Content_handler_API Content handler API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_Storage_Local_API Improve Storage Local API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_keyboard_shortcut_support Improve keyboard shortcut support]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_support_for_private_browsing_mode Improve support for private browsing mode]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Identifying_recommended_extensions_in_about:addons Identifying recommended extensions in about:addons]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Limited_Tab_Hiding Limited Tab Hiding]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Language_Pack_Signing Language Pack Signing]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/OOP-WebExtensions OOP-Webextensions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Pre-installed_extension_in_Firefox Pre-installed extension in Firefox]<br />
* [https://wiki.mozilla.org/Add-ons/Permissions/Testplan Permissions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Roll_disco_pane_into_about:addons Roll disco pane into about:addons]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/SHA SHA]<br />
* [https://wiki.mozilla.org/QA/Theming/Testplan Theming]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Telemetry_for_add-ons_manager Telemetry for add-ons manager]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Permissions_Android Webextensions Permissions Android]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Language_Packs Webextensions Language Packs]<br />
* [https://wiki.mozilla.org/QA/WebExtensions WebExtensions-Plaform]<br />
* [https://wiki.mozilla.org/Add-ons/Webextensions_Cutover/Testplan WebExtensions Cutover]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Cutover_Android Webextensions Cutover Android]<br />
<br />
== AMO TestPlans==<br />
* [https://wiki.mozilla.org/QA/AMO_WebExtensions Webextension-AMO]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Activity Activity Mail]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Listed-Unlisted_Submissions New Submissions Flow]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Restrict_Legacy_Add-ons_Submission Restrict Legacy Add-ons Submission]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/NewAMOMobile New Mobile Frontend]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Auto-Approvals Auto-Approvals]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_Post_Reviews_Process Post Reviews Process]</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/AMO_Process_Isolation&diff=1213430Add-ons/QA/Testplan/AMO Process Isolation2019-06-07T12:16:23Z<p>CosminB: Created blank page</p>
<hr />
<div></div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/about:addons_non-xul_rebuild&diff=1211970Add-ons/QA/Testplan/about:addons non-xul rebuild2019-05-08T06:58:03Z<p>CosminB: /* Testing Tools */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 02/14/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
<br />
*Building new design and functionality in HTML will allow us to iterate faster, without the burden of dealing with XUL quirks and limitations. <br />
*It would help to avoid investing deeper in a technology that is deprecated.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* The XUL interface from about:addons converted to HTML which will iterate faster.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mstriemer@mozilla.com Mark Striemer]; irc nick :mstriemer <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.htmlaboutaddons.enabled from about:config<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.htmlaboutaddons.enabled from about:config<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 02-14-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | 02-13-2018 || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [https://testrail.stage.mozaws.net/index.php?/suites/view/3377&group_by=cases:section_id&group_order=asc TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - []<br />
Install flow - N/A<br />
<br />
<br />
<br />
* {{Bug|1505924}} - [meta] New about:addons interface<br />
<bugzilla><br />
{<br />
"blocks":[1505924],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [https://testrail.stage.mozaws.net/index.php?/suites/view/3377&group_by=cases:section_id&group_order=asc about:addons_non-xul_rebuild]<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 02-14-2019 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | 02-13-2019 || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/about:addons_non-xul_rebuild&diff=1211968Add-ons/QA/Testplan/about:addons non-xul rebuild2019-05-08T06:54:29Z<p>CosminB: /* Test suite */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 02/14/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
<br />
*Building new design and functionality in HTML will allow us to iterate faster, without the burden of dealing with XUL quirks and limitations. <br />
*It would help to avoid investing deeper in a technology that is deprecated.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* The XUL interface from about:addons converted to HTML which will iterate faster.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mstriemer@mozilla.com Mark Striemer]; irc nick :mstriemer <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.htmlaboutaddons.enabled from about:config<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.htmlaboutaddons.enabled from about:config<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 02-14-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | 02-13-2018 || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - []<br />
Install flow - N/A<br />
<br />
<br />
<br />
* {{Bug|1505924}} - [meta] New about:addons interface<br />
<bugzilla><br />
{<br />
"blocks":[1505924],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [https://testrail.stage.mozaws.net/index.php?/suites/view/3377&group_by=cases:section_id&group_order=asc about:addons_non-xul_rebuild]<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 02-14-2019 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | 02-13-2019 || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/about:addons_non-xul_rebuild&diff=1211967Add-ons/QA/Testplan/about:addons non-xul rebuild2019-05-08T06:51:17Z<p>CosminB: /* Test suite */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 02/14/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
<br />
*Building new design and functionality in HTML will allow us to iterate faster, without the burden of dealing with XUL quirks and limitations. <br />
*It would help to avoid investing deeper in a technology that is deprecated.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* The XUL interface from about:addons converted to HTML which will iterate faster.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mstriemer@mozilla.com Mark Striemer]; irc nick :mstriemer <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.htmlaboutaddons.enabled from about:config<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.htmlaboutaddons.enabled from about:config<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 02-14-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | 02-13-2018 || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - []<br />
Install flow - N/A<br />
<br />
<br />
<br />
* {{Bug|1505924}} - [meta] New about:addons interface<br />
<bugzilla><br />
{<br />
"blocks":[1505924],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [about:addons_non-xul_rebuild]<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 02-14-2019 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | 02-13-2019 || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Roll_disco_pane_into_about:addons&diff=1210843Add-ons/QA/Testplan/Roll disco pane into about:addons2019-04-18T09:16:23Z<p>CosminB: /* Checklist */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 04/15/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
<br />
*The discovery pane of about:addons is actually an AMO web page that is styled to look like it is part of about:addons. Since disco pane is part of the Add-on Manager experience, it should be integrated for efficiency and consistency.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* With the re-write of about:addons into HTML and changes to telemetry in disco pane, now is the time to integrate them so we can finally have them under one roof and avoid a disruption to disco pane’s telemetry. Furthermore, AMO and Firefox have different release cycles, making it difficult to keep disco pane and about:addons consistent.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mstriemer@mozilla.com Mark Striemer]; irc nick :mstriemer <br /><br />
Dev Lead: [mailto:rwu@mozilla.com Rob Wu]; irc nick :robwu_nl <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Card changed to the new format || To verify that the extensions card are displayed correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Recommended section || To verify that the transition from "Get Add-ons" section to the "Recommended" section is working properly || Manual || Add-ons QA Team<br />
|-<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 04-15-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - [https://docs.google.com/document/d/1_0FXaXkDSb2FSD_ZYE_SlOlZNR4cnIrmqHJuqz9ud-c/edit# Gdoc]<br />
Feature flow - [N/A]<br />
<br />
<br />
<br />
* {{Bug|1540173}} - [meta] change "Get Add-ons" (discopane) to be a proper part of about:addons rather than a framed page from AMO<br />
<bugzilla><br />
{<br />
"blocks":[1540173],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [ Test rail Roll disco pane into about:addons]<br />
<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, critical must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 04-15-2019 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | || <br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Roll_disco_pane_into_about:addons&diff=1210842Add-ons/QA/Testplan/Roll disco pane into about:addons2019-04-18T09:15:14Z<p>CosminB: /* Test Execution Schedule */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 04/15/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
<br />
*The discovery pane of about:addons is actually an AMO web page that is styled to look like it is part of about:addons. Since disco pane is part of the Add-on Manager experience, it should be integrated for efficiency and consistency.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* With the re-write of about:addons into HTML and changes to telemetry in disco pane, now is the time to integrate them so we can finally have them under one roof and avoid a disruption to disco pane’s telemetry. Furthermore, AMO and Firefox have different release cycles, making it difficult to keep disco pane and about:addons consistent.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mstriemer@mozilla.com Mark Striemer]; irc nick :mstriemer <br /><br />
Dev Lead: [mailto:rwu@mozilla.com Rob Wu]; irc nick :robwu_nl <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Card changed to the new format || To verify that the extensions card are displayed correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Recommended section || To verify that the transition from "Get Add-ons" section to the "Recommended" section is working properly || Manual || Add-ons QA Team<br />
|-<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 04-15-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - [https://docs.google.com/document/d/1_0FXaXkDSb2FSD_ZYE_SlOlZNR4cnIrmqHJuqz9ud-c/edit# Gdoc]<br />
Feature flow - [N/A]<br />
<br />
<br />
<br />
* {{Bug|1540173}} - [meta] change "Get Add-ons" (discopane) to be a proper part of about:addons rather than a framed page from AMO<br />
<bugzilla><br />
{<br />
"blocks":[1540173],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [ Test rail Roll disco pane into about:addons]<br />
<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, critical must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 04-15-2019 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | N/A || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Identifying_recommended_extensions_in_about:addons&diff=1210841Add-ons/QA/Testplan/Identifying recommended extensions in about:addons2019-04-18T09:14:45Z<p>CosminB: /* Checklist */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 04/15/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
<br />
*Extensions will be signed with additional metadata so Firefox can recognize if they are marked as Recommended or Partner.<br />
*The [https://docs.google.com/document/d/1fRxLy7h02tn1HBIlsTU_6h9DHmN8c3N6bg4JoxHF7J8/edit Recommended Extensions PRD] explains why this new distinction is important. The signature changes are necessary so Firefox can independently assess the extension regardless of how it was installed. Storing this data in the signature also protects it from being tampered with.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* Recommended extensions will be identified in Firefox in about:addons by an additional graphic/text on the universal card. As such, this is a slight feature to QA. We only need to verify that recommended extensions, once installed, are correctly displayed in about:addons.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mstriemer@mozilla.com Mark Striemer]; irc nick :mstriemer <br /><br />
Dev Lead: [mailto:aswan@mozilla.com Andrew Swan]; irc nick :aswan <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the recommendation badges are displayed correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the recommendation badges are displayed correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the recommendation badges are displayed correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the recommendation badges are displayed correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 04-15-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | ||<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - [https://docs.google.com/document/d/1CroMgHStmI5lsxStjhlIwLiTs1SlWf8DQ0IpkTieZMs/edit# Gdoc]<br />
Feature flow - [https://mozilla.invisionapp.com/share/BXQTIIC86ZN#/screens/351565696 Mocks]<br />
<br />
<br />
<br />
* {{Bug|1539699}} - [about:addons] identify recommended extensions in about:addons<br />
<bugzilla><br />
{<br />
"blocks":[1539699],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [ Test rail Identifying recommended extensions in about:addons]<br />
<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, critical must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 04-15-2019 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | || <br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Identifying_recommended_extensions_in_about:addons&diff=1210840Add-ons/QA/Testplan/Identifying recommended extensions in about:addons2019-04-18T09:13:05Z<p>CosminB: /* Test Execution Schedule */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 04/15/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
<br />
*Extensions will be signed with additional metadata so Firefox can recognize if they are marked as Recommended or Partner.<br />
*The [https://docs.google.com/document/d/1fRxLy7h02tn1HBIlsTU_6h9DHmN8c3N6bg4JoxHF7J8/edit Recommended Extensions PRD] explains why this new distinction is important. The signature changes are necessary so Firefox can independently assess the extension regardless of how it was installed. Storing this data in the signature also protects it from being tampered with.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* Recommended extensions will be identified in Firefox in about:addons by an additional graphic/text on the universal card. As such, this is a slight feature to QA. We only need to verify that recommended extensions, once installed, are correctly displayed in about:addons.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mstriemer@mozilla.com Mark Striemer]; irc nick :mstriemer <br /><br />
Dev Lead: [mailto:aswan@mozilla.com Andrew Swan]; irc nick :aswan <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the recommendation badges are displayed correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the recommendation badges are displayed correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the recommendation badges are displayed correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the recommendation badges are displayed correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 04-15-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | ||<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - [https://docs.google.com/document/d/1CroMgHStmI5lsxStjhlIwLiTs1SlWf8DQ0IpkTieZMs/edit# Gdoc]<br />
Feature flow - [https://mozilla.invisionapp.com/share/BXQTIIC86ZN#/screens/351565696 Mocks]<br />
<br />
<br />
<br />
* {{Bug|1539699}} - [about:addons] identify recommended extensions in about:addons<br />
<bugzilla><br />
{<br />
"blocks":[1539699],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [ Test rail Identifying recommended extensions in about:addons]<br />
<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, critical must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 04-15-2019 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | N/A || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Roll_disco_pane_into_about:addons&diff=1210613Add-ons/QA/Testplan/Roll disco pane into about:addons2019-04-15T11:42:53Z<p>CosminB: </p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 04/15/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
<br />
*The discovery pane of about:addons is actually an AMO web page that is styled to look like it is part of about:addons. Since disco pane is part of the Add-on Manager experience, it should be integrated for efficiency and consistency.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* With the re-write of about:addons into HTML and changes to telemetry in disco pane, now is the time to integrate them so we can finally have them under one roof and avoid a disruption to disco pane’s telemetry. Furthermore, AMO and Firefox have different release cycles, making it difficult to keep disco pane and about:addons consistent.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mstriemer@mozilla.com Mark Striemer]; irc nick :mstriemer <br /><br />
Dev Lead: [mailto:rwu@mozilla.com Rob Wu]; irc nick :robwu_nl <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Card changed to the new format || To verify that the extensions card are displayed correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Recommended section || To verify that the transition from "Get Add-ons" section to the "Recommended" section is working properly || Manual || Add-ons QA Team<br />
|-<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 04-15-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - [https://docs.google.com/document/d/1_0FXaXkDSb2FSD_ZYE_SlOlZNR4cnIrmqHJuqz9ud-c/edit# Gdoc]<br />
Feature flow - [N/A]<br />
<br />
<br />
<br />
* {{Bug|1540173}} - [meta] change "Get Add-ons" (discopane) to be a proper part of about:addons rather than a framed page from AMO<br />
<bugzilla><br />
{<br />
"blocks":[1540173],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [ Test rail Roll disco pane into about:addons]<br />
<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, critical must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 04-15-2019 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | N/A || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Identifying_recommended_extensions_in_about:addons&diff=1210612Add-ons/QA/Testplan/Identifying recommended extensions in about:addons2019-04-15T11:19:16Z<p>CosminB: /* Checklist */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 04/15/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
<br />
*Extensions will be signed with additional metadata so Firefox can recognize if they are marked as Recommended or Partner.<br />
*The [https://docs.google.com/document/d/1fRxLy7h02tn1HBIlsTU_6h9DHmN8c3N6bg4JoxHF7J8/edit Recommended Extensions PRD] explains why this new distinction is important. The signature changes are necessary so Firefox can independently assess the extension regardless of how it was installed. Storing this data in the signature also protects it from being tampered with.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* Recommended extensions will be identified in Firefox in about:addons by an additional graphic/text on the universal card. As such, this is a slight feature to QA. We only need to verify that recommended extensions, once installed, are correctly displayed in about:addons.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mstriemer@mozilla.com Mark Striemer]; irc nick :mstriemer <br /><br />
Dev Lead: [mailto:aswan@mozilla.com Andrew Swan]; irc nick :aswan <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the recommendation badges are displayed correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the recommendation badges are displayed correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the recommendation badges are displayed correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the recommendation badges are displayed correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 04-15-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - [https://docs.google.com/document/d/1CroMgHStmI5lsxStjhlIwLiTs1SlWf8DQ0IpkTieZMs/edit# Gdoc]<br />
Feature flow - [https://mozilla.invisionapp.com/share/BXQTIIC86ZN#/screens/351565696 Mocks]<br />
<br />
<br />
<br />
* {{Bug|1539699}} - [about:addons] identify recommended extensions in about:addons<br />
<bugzilla><br />
{<br />
"blocks":[1539699],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [ Test rail Identifying recommended extensions in about:addons]<br />
<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, critical must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 04-15-2019 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | N/A || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Identifying_recommended_extensions_in_about:addons&diff=1210601Add-ons/QA/Testplan/Identifying recommended extensions in about:addons2019-04-15T08:59:18Z<p>CosminB: /* Test Execution Schedule */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 04/15/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
<br />
*Extensions will be signed with additional metadata so Firefox can recognize if they are marked as Recommended or Partner.<br />
*The [https://docs.google.com/document/d/1fRxLy7h02tn1HBIlsTU_6h9DHmN8c3N6bg4JoxHF7J8/edit Recommended Extensions PRD] explains why this new distinction is important. The signature changes are necessary so Firefox can independently assess the extension regardless of how it was installed. Storing this data in the signature also protects it from being tampered with.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* Recommended extensions will be identified in Firefox in about:addons by an additional graphic/text on the universal card. As such, this is a slight feature to QA. We only need to verify that recommended extensions, once installed, are correctly displayed in about:addons.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mstriemer@mozilla.com Mark Striemer]; irc nick :mstriemer <br /><br />
Dev Lead: [mailto:aswan@mozilla.com Andrew Swan]; irc nick :aswan <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the recommendation badges are displayed correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the recommendation badges are displayed correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the recommendation badges are displayed correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the recommendation badges are displayed correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 04-15-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - [https://docs.google.com/document/d/1CroMgHStmI5lsxStjhlIwLiTs1SlWf8DQ0IpkTieZMs/edit# Gdoc]<br />
Feature flow - [https://mozilla.invisionapp.com/share/BXQTIIC86ZN#/screens/351565696 Mocks]<br />
<br />
<br />
<br />
* {{Bug|1539699}} - [about:addons] identify recommended extensions in about:addons<br />
<bugzilla><br />
{<br />
"blocks":[1539699],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [ Test rail Identifying recommended extensions in about:addons]<br />
<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, critical must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 04-15-2019 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | N/A || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Identifying_recommended_extensions_in_about:addons&diff=1210600Add-ons/QA/Testplan/Identifying recommended extensions in about:addons2019-04-15T08:58:21Z<p>CosminB: </p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 04/15/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
<br />
*Extensions will be signed with additional metadata so Firefox can recognize if they are marked as Recommended or Partner.<br />
*The [https://docs.google.com/document/d/1fRxLy7h02tn1HBIlsTU_6h9DHmN8c3N6bg4JoxHF7J8/edit Recommended Extensions PRD] explains why this new distinction is important. The signature changes are necessary so Firefox can independently assess the extension regardless of how it was installed. Storing this data in the signature also protects it from being tampered with.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* Recommended extensions will be identified in Firefox in about:addons by an additional graphic/text on the universal card. As such, this is a slight feature to QA. We only need to verify that recommended extensions, once installed, are correctly displayed in about:addons.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mstriemer@mozilla.com Mark Striemer]; irc nick :mstriemer <br /><br />
Dev Lead: [mailto:aswan@mozilla.com Andrew Swan]; irc nick :aswan <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the recommendation badges are displayed correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the recommendation badges are displayed correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the recommendation badges are displayed correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the recommendation badges are displayed correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 02-14-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | 02-13-2018 || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - [https://docs.google.com/document/d/1CroMgHStmI5lsxStjhlIwLiTs1SlWf8DQ0IpkTieZMs/edit# Gdoc]<br />
Feature flow - [https://mozilla.invisionapp.com/share/BXQTIIC86ZN#/screens/351565696 Mocks]<br />
<br />
<br />
<br />
* {{Bug|1539699}} - [about:addons] identify recommended extensions in about:addons<br />
<bugzilla><br />
{<br />
"blocks":[1539699],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [ Test rail Identifying recommended extensions in about:addons]<br />
<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, critical must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 04-15-2019 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | N/A || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplans&diff=1210598Add-ons/QA/Testplans2019-04-15T08:38:02Z<p>CosminB: /* WebExtensions TestPlans */</p>
<hr />
<div>== QA Contacts == <br />
<br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Add-ons QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
== WebExtensions TestPlans==<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/about:addons_non-xul_rebuild about:addons_non-xul_rebuild]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_ContentScript Add-ons ContentScript]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-on_Compatibility_Check Add-on Compatibility Check]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Android_Webextension_API Android Webextension API]<br />
* [https://wiki.mozilla.org/WebExtensions/chrome.storage.sync chrome.storage.sync]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/CSP CSP]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Context_Menu_Updates Context Menu Updates]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Content_handler_API Content handler API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_Storage_Local_API Improve Storage Local API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_keyboard_shortcut_support Improve_keyboard_shortcut_support]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_support_for_private_browsing_mode Improve_support_for_private_browsing_mode]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Identifying_recommended_extensions_in_about:addons Identifying_recommended_extensions_in_about:addons]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Limited_Tab_Hiding Limited_Tab_Hiding]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Language_Pack_Signing Language Pack Signing]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/OOP-WebExtensions OOP-Webextensions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Pre-installed_extension_in_Firefox Pre-installed extension in Firefox]<br />
* [https://wiki.mozilla.org/Add-ons/Permissions/Testplan Permissions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Roll_disco_pane_into_about:addons Roll_disco_pane_into_about:addons]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/SHA SHA]<br />
* [https://wiki.mozilla.org/QA/Theming/Testplan Theming]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Telemetry_for_add-ons_manager Telemetry_for_add-ons_manager]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Permissions_Android Webextensions Permissions Android]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Language_Packs Webextensions Language Packs]<br />
* [https://wiki.mozilla.org/QA/WebExtensions WebExtensions-Plaform]<br />
* [https://wiki.mozilla.org/Add-ons/Webextensions_Cutover/Testplan WebExtensions Cutover]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Cutover_Android Webextensions Cutover Android]<br />
<br />
== AMO TestPlans==<br />
* [https://wiki.mozilla.org/QA/AMO_WebExtensions Webextension-AMO]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Activity Activity Mail]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Listed-Unlisted_Submissions New Submissions Flow]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Restrict_Legacy_Add-ons_Submission Restrict Legacy Add-ons Submission]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/NewAMOMobile New Mobile Frontend]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Auto-Approvals Auto-Approvals]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_Post_Reviews_Process Post Reviews Process]</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplans&diff=1210597Add-ons/QA/Testplans2019-04-15T08:36:51Z<p>CosminB: /* WebExtensions TestPlans */</p>
<hr />
<div>== QA Contacts == <br />
<br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Add-ons QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
== WebExtensions TestPlans==<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/about:addons_non-xul_rebuild about:addons_non-xul_rebuild]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_ContentScript Add-ons ContentScript]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-on_Compatibility_Check Add-on Compatibility Check]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Android_Webextension_API Android Webextension API]<br />
* [https://wiki.mozilla.org/WebExtensions/chrome.storage.sync chrome.storage.sync]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/CSP CSP]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Context_Menu_Updates Context Menu Updates]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Content_handler_API Content handler API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_Storage_Local_API Improve Storage Local API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_keyboard_shortcut_support Improve_keyboard_shortcut_support]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_support_for_private_browsing_mode Improve_support_for_private_browsing_mode]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Identifying_recommended_extensions_in_about:addons]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Limited_Tab_Hiding Limited_Tab_Hiding]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Language_Pack_Signing Language Pack Signing]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/OOP-WebExtensions OOP-Webextensions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Pre-installed_extension_in_Firefox Pre-installed extension in Firefox]<br />
* [https://wiki.mozilla.org/Add-ons/Permissions/Testplan Permissions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Roll_disco_pane_into_about:addons]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/SHA SHA]<br />
* [https://wiki.mozilla.org/QA/Theming/Testplan Theming]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Telemetry_for_add-ons_manager Telemetry_for_add-ons_manager]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Permissions_Android Webextensions Permissions Android]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Language_Packs Webextensions Language Packs]<br />
* [https://wiki.mozilla.org/QA/WebExtensions WebExtensions-Plaform]<br />
* [https://wiki.mozilla.org/Add-ons/Webextensions_Cutover/Testplan WebExtensions Cutover]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Cutover_Android Webextensions Cutover Android]<br />
<br />
== AMO TestPlans==<br />
* [https://wiki.mozilla.org/QA/AMO_WebExtensions Webextension-AMO]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Activity Activity Mail]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Listed-Unlisted_Submissions New Submissions Flow]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Restrict_Legacy_Add-ons_Submission Restrict Legacy Add-ons Submission]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/NewAMOMobile New Mobile Frontend]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Auto-Approvals Auto-Approvals]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_Post_Reviews_Process Post Reviews Process]</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Roll_disco_pane_into_about:addons&diff=1210596Add-ons/QA/Testplan/Roll disco pane into about:addons2019-04-15T08:35:56Z<p>CosminB: Created page with "TBD"</p>
<hr />
<div>TBD</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplans&diff=1210595Add-ons/QA/Testplans2019-04-15T08:33:47Z<p>CosminB: /* WebExtensions TestPlans */</p>
<hr />
<div>== QA Contacts == <br />
<br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Add-ons QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
== WebExtensions TestPlans==<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/about:addons_non-xul_rebuild about:addons_non-xul_rebuild]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_ContentScript Add-ons ContentScript]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-on_Compatibility_Check Add-on Compatibility Check]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Android_Webextension_API Android Webextension API]<br />
* [https://wiki.mozilla.org/WebExtensions/chrome.storage.sync chrome.storage.sync]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/CSP CSP]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Context_Menu_Updates Context Menu Updates]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Content_handler_API Content handler API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_Storage_Local_API Improve Storage Local API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_keyboard_shortcut_support Improve_keyboard_shortcut_support]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_support_for_private_browsing_mode Improve_support_for_private_browsing_mode]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Identifying_recommended_extensions_in_about:addons]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Limited_Tab_Hiding Limited_Tab_Hiding]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Language_Pack_Signing Language Pack Signing]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/OOP-WebExtensions OOP-Webextensions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Pre-installed_extension_in_Firefox Pre-installed extension in Firefox]<br />
* [https://wiki.mozilla.org/Add-ons/Permissions/Testplan Permissions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/SHA SHA]<br />
* [https://wiki.mozilla.org/QA/Theming/Testplan Theming]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Telemetry_for_add-ons_manager Telemetry_for_add-ons_manager]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Permissions_Android Webextensions Permissions Android]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Language_Packs Webextensions Language Packs]<br />
* [https://wiki.mozilla.org/QA/WebExtensions WebExtensions-Plaform]<br />
* [https://wiki.mozilla.org/Add-ons/Webextensions_Cutover/Testplan WebExtensions Cutover]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Cutover_Android Webextensions Cutover Android]<br />
<br />
== AMO TestPlans==<br />
* [https://wiki.mozilla.org/QA/AMO_WebExtensions Webextension-AMO]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Activity Activity Mail]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Listed-Unlisted_Submissions New Submissions Flow]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Restrict_Legacy_Add-ons_Submission Restrict Legacy Add-ons Submission]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/NewAMOMobile New Mobile Frontend]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Auto-Approvals Auto-Approvals]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_Post_Reviews_Process Post Reviews Process]</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Identifying_recommended_extensions_in_about:addons&diff=1210594Add-ons/QA/Testplan/Identifying recommended extensions in about:addons2019-04-15T08:33:28Z<p>CosminB: Created page with "TBD"</p>
<hr />
<div>TBD</div>CosminBhttps://wiki.mozilla.org/index.php?title=Talk:Add-ons/QA/Testplan/Identifying_recommended_extensions_in_about:addons&diff=1210593Talk:Add-ons/QA/Testplan/Identifying recommended extensions in about:addons2019-04-15T08:31:40Z<p>CosminB: Created blank page</p>
<hr />
<div></div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/about:addons_non-xul_rebuild&diff=1207745Add-ons/QA/Testplan/about:addons non-xul rebuild2019-02-14T09:50:39Z<p>CosminB: /* Checklist */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 02/14/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
<br />
*Building new design and functionality in HTML will allow us to iterate faster, without the burden of dealing with XUL quirks and limitations. <br />
*It would help to avoid investing deeper in a technology that is deprecated.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* The XUL interface from about:addons converted to HTML which will iterate faster.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mstriemer@mozilla.com Mark Striemer]; irc nick :mstriemer <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.htmlaboutaddons.enabled from about:config<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.htmlaboutaddons.enabled from about:config<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 02-14-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | 02-13-2018 || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - []<br />
Install flow - N/A<br />
<br />
<br />
<br />
* {{Bug|1505924}} - [meta] New about:addons interface<br />
<bugzilla><br />
{<br />
"blocks":[1505924],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [https://testrail.stage.mozaws.net/index.php?/suites/view/3166&group_by=cases:section_id&group_order=asc Improve support for private browsing mode]<br />
<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 02-14-2019 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | 02-13-2019 || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/about:addons_non-xul_rebuild&diff=1207744Add-ons/QA/Testplan/about:addons non-xul rebuild2019-02-14T09:50:06Z<p>CosminB: /* References */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 02/14/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
<br />
*Building new design and functionality in HTML will allow us to iterate faster, without the burden of dealing with XUL quirks and limitations. <br />
*It would help to avoid investing deeper in a technology that is deprecated.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* The XUL interface from about:addons converted to HTML which will iterate faster.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mstriemer@mozilla.com Mark Striemer]; irc nick :mstriemer <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.htmlaboutaddons.enabled from about:config<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.htmlaboutaddons.enabled from about:config<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 02-14-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | 02-13-2018 || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - []<br />
Install flow - N/A<br />
<br />
<br />
<br />
* {{Bug|1505924}} - [meta] New about:addons interface<br />
<bugzilla><br />
{<br />
"blocks":[1505924],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [https://testrail.stage.mozaws.net/index.php?/suites/view/3166&group_by=cases:section_id&group_order=asc Improve support for private browsing mode]<br />
<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 05-22-2018 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | 05-21-2018 || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/about:addons_non-xul_rebuild&diff=1207743Add-ons/QA/Testplan/about:addons non-xul rebuild2019-02-14T09:48:37Z<p>CosminB: /* Test Execution Schedule */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 02/14/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
<br />
*Building new design and functionality in HTML will allow us to iterate faster, without the burden of dealing with XUL quirks and limitations. <br />
*It would help to avoid investing deeper in a technology that is deprecated.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* The XUL interface from about:addons converted to HTML which will iterate faster.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mstriemer@mozilla.com Mark Striemer]; irc nick :mstriemer <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.htmlaboutaddons.enabled from about:config<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.htmlaboutaddons.enabled from about:config<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 02-14-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | 02-13-2018 || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - [https://docs.google.com/document/d/1TJ9ncwORJ3nkQ4te78NU6lFmR-3kKTcKCqX7Gq1wQyo/edit]<br />
Install flow - N/A<br />
<br />
<br />
<br />
* {{Bug|1460738}} - [META] improve support for Private Browsing<br />
<bugzilla><br />
{<br />
"blocks":[1460738],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
* {{Bug|1380809}} - [META] manifest incognito feature support<br />
<bugzilla><br />
{<br />
"blocks":[1380809],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [https://testrail.stage.mozaws.net/index.php?/suites/view/3166&group_by=cases:section_id&group_order=asc Improve support for private browsing mode]<br />
<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 05-22-2018 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | 05-21-2018 || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/about:addons_non-xul_rebuild&diff=1207742Add-ons/QA/Testplan/about:addons non-xul rebuild2019-02-14T09:48:08Z<p>CosminB: /* Channel dependent settings (configs) and environment setups */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 02/14/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
<br />
*Building new design and functionality in HTML will allow us to iterate faster, without the burden of dealing with XUL quirks and limitations. <br />
*It would help to avoid investing deeper in a technology that is deprecated.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* The XUL interface from about:addons converted to HTML which will iterate faster.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mstriemer@mozilla.com Mark Striemer]; irc nick :mstriemer <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.htmlaboutaddons.enabled from about:config<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.htmlaboutaddons.enabled from about:config<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 01-29-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | 12-15-2018 || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - [https://docs.google.com/document/d/1TJ9ncwORJ3nkQ4te78NU6lFmR-3kKTcKCqX7Gq1wQyo/edit]<br />
Install flow - N/A<br />
<br />
<br />
<br />
* {{Bug|1460738}} - [META] improve support for Private Browsing<br />
<bugzilla><br />
{<br />
"blocks":[1460738],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
* {{Bug|1380809}} - [META] manifest incognito feature support<br />
<bugzilla><br />
{<br />
"blocks":[1380809],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [https://testrail.stage.mozaws.net/index.php?/suites/view/3166&group_by=cases:section_id&group_order=asc Improve support for private browsing mode]<br />
<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 05-22-2018 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | 05-21-2018 || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Improve_keyboard_shortcut_support&diff=1207741Add-ons/QA/Testplan/Improve keyboard shortcut support2019-02-14T09:46:16Z<p>CosminB: /* Ownership */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 01/29/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
Allow the user to understand what the current keyboard mapping is, highlighting keyboard mappings that are overriden by add-ons, and providing a UI for users to adjust keyboard shortcuts used by web extensions.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All the required Telemetry is in place<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* The assigned shortcut keys work as expected when they are assigned from the extension.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mstriemer@mozilla.com Mark Striemer]; irc nick :mstriemer <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 01-29-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | 12-15-2018 || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - [https://docs.google.com/document/d/1D0XRkNaO5YgwwSPg6JZn5lsdV-EyrlPbvZMqr4jeOVo/edit]<br />
Install flow - [https://mozilla.invisionapp.com/share/Y2MK3ZS8ZUP#/screens/299111761]<br />
<br />
<br />
<br />
* {{Bug|1215061}} - [META] Better keyboard shortcut support<br />
<bugzilla><br />
{<br />
"blocks":[1215061],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
* {{Bug|1303384}} - UI for re-assigning an extension's command shortcut<br />
<bugzilla><br />
{<br />
"blocks":[1303384],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [https://testrail.stage.mozaws.net/index.php?/suites/view/3177&group_by=cases:section_id&group_order=asc Improve keyboard shortcut support]<br />
<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 05-22-2018 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | 05-21-2018 || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/about:addons_non-xul_rebuild&diff=1207740Add-ons/QA/Testplan/about:addons non-xul rebuild2019-02-14T09:46:10Z<p>CosminB: /* what's in scope? */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 02/14/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
<br />
*Building new design and functionality in HTML will allow us to iterate faster, without the burden of dealing with XUL quirks and limitations. <br />
*It would help to avoid investing deeper in a technology that is deprecated.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* The XUL interface from about:addons converted to HTML which will iterate faster.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mstriemer@mozilla.com Mark Striemer]; irc nick :mstriemer <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.allowPrivateBrowsingByDefault from about:config<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.allowPrivateBrowsingByDefault from about:config<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 01-29-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | 12-15-2018 || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - [https://docs.google.com/document/d/1TJ9ncwORJ3nkQ4te78NU6lFmR-3kKTcKCqX7Gq1wQyo/edit]<br />
Install flow - N/A<br />
<br />
<br />
<br />
* {{Bug|1460738}} - [META] improve support for Private Browsing<br />
<bugzilla><br />
{<br />
"blocks":[1460738],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
* {{Bug|1380809}} - [META] manifest incognito feature support<br />
<bugzilla><br />
{<br />
"blocks":[1380809],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [https://testrail.stage.mozaws.net/index.php?/suites/view/3166&group_by=cases:section_id&group_order=asc Improve support for private browsing mode]<br />
<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 05-22-2018 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | 05-21-2018 || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/about:addons_non-xul_rebuild&diff=1207739Add-ons/QA/Testplan/about:addons non-xul rebuild2019-02-14T09:44:21Z<p>CosminB: </p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 02/14/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
<br />
*Building new design and functionality in HTML will allow us to iterate faster, without the burden of dealing with XUL quirks and limitations. <br />
*It would help to avoid investing deeper in a technology that is deprecated.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* The behavior of the storage.local JSONFile backend and storage.local IndexedDB backend when the extensions use them.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mstriemer@mozilla.com Mark Striemer]; irc nick :mstriemer <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.allowPrivateBrowsingByDefault from about:config<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.allowPrivateBrowsingByDefault from about:config<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 01-29-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | 12-15-2018 || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - [https://docs.google.com/document/d/1TJ9ncwORJ3nkQ4te78NU6lFmR-3kKTcKCqX7Gq1wQyo/edit]<br />
Install flow - N/A<br />
<br />
<br />
<br />
* {{Bug|1460738}} - [META] improve support for Private Browsing<br />
<bugzilla><br />
{<br />
"blocks":[1460738],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
* {{Bug|1380809}} - [META] manifest incognito feature support<br />
<bugzilla><br />
{<br />
"blocks":[1380809],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [https://testrail.stage.mozaws.net/index.php?/suites/view/3166&group_by=cases:section_id&group_order=asc Improve support for private browsing mode]<br />
<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 05-22-2018 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | 05-21-2018 || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Improve_keyboard_shortcut_support&diff=1207738Add-ons/QA/Testplan/Improve keyboard shortcut support2019-02-14T09:43:49Z<p>CosminB: /* what's in scope? */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 01/29/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
Allow the user to understand what the current keyboard mapping is, highlighting keyboard mappings that are overriden by add-ons, and providing a UI for users to adjust keyboard shortcuts used by web extensions.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All the required Telemetry is in place<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* The assigned shortcut keys work as expected when they are assigned from the extension.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mstriemer@mozilla.com Mark Striemer]; irc nick :mstriemer <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 01-29-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | 12-15-2018 || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - [https://docs.google.com/document/d/1D0XRkNaO5YgwwSPg6JZn5lsdV-EyrlPbvZMqr4jeOVo/edit]<br />
Install flow - [https://mozilla.invisionapp.com/share/Y2MK3ZS8ZUP#/screens/299111761]<br />
<br />
<br />
<br />
* {{Bug|1215061}} - [META] Better keyboard shortcut support<br />
<bugzilla><br />
{<br />
"blocks":[1215061],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
* {{Bug|1303384}} - UI for re-assigning an extension's command shortcut<br />
<bugzilla><br />
{<br />
"blocks":[1303384],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [https://testrail.stage.mozaws.net/index.php?/suites/view/3177&group_by=cases:section_id&group_order=asc Improve keyboard shortcut support]<br />
<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 05-22-2018 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | 05-21-2018 || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Improve_support_for_private_browsing_mode&diff=1207737Add-ons/QA/Testplan/Improve support for private browsing mode2019-02-14T09:42:11Z<p>CosminB: /* what's in scope? */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 01/29/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
There are two aspects to extensions running in private browsing tabs: user choice, and developer preference.<br />
<br />
*User Choice: the user should be able to determine which, if any, extensions should be able to run within private browsing tabs.<br />
*Developer Preference: the developer should be able to specify if their extension can or cannot be run in private browsing tabs.<br />
<br />
Mozilla and independent research strongly support the idea that user control of extensions in private browsing is an important part of protecting users online.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All the required Telemetry is in place<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* The behavior is as expected for the extensions in the Private Browsing.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mixedpuppy@gmail.com Shane Caraveo]; irc nick :mixedpuppy <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.allowPrivateBrowsingByDefault from about:config<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.allowPrivateBrowsingByDefault from about:config<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 01-29-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | 12-15-2018 || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - [https://docs.google.com/document/d/1TJ9ncwORJ3nkQ4te78NU6lFmR-3kKTcKCqX7Gq1wQyo/edit]<br />
Install flow - N/A<br />
<br />
<br />
<br />
* {{Bug|1460738}} - [META] improve support for Private Browsing<br />
<bugzilla><br />
{<br />
"blocks":[1460738],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
* {{Bug|1380809}} - [META] manifest incognito feature support<br />
<bugzilla><br />
{<br />
"blocks":[1380809],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [https://testrail.stage.mozaws.net/index.php?/suites/view/3166&group_by=cases:section_id&group_order=asc Improve support for private browsing mode]<br />
<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 05-22-2018 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | 05-21-2018 || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplans&diff=1207736Add-ons/QA/Testplans2019-02-14T09:35:40Z<p>CosminB: /* QA Contacts */</p>
<hr />
<div>== QA Contacts == <br />
<br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Add-ons QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
== WebExtensions TestPlans==<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/about:addons_non-xul_rebuild about:addons_non-xul_rebuild]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_ContentScript Add-ons ContentScript]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-on_Compatibility_Check Add-on Compatibility Check]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Android_Webextension_API Android Webextension API]<br />
* [https://wiki.mozilla.org/WebExtensions/chrome.storage.sync chrome.storage.sync]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/CSP CSP]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Context_Menu_Updates Context Menu Updates]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Content_handler_API Content handler API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_Storage_Local_API Improve Storage Local API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_keyboard_shortcut_support Improve_keyboard_shortcut_support]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_support_for_private_browsing_mode Improve_support_for_private_browsing_mode]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Limited_Tab_Hiding Limited_Tab_Hiding]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Language_Pack_Signing Language Pack Signing]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/OOP-WebExtensions OOP-Webextensions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Pre-installed_extension_in_Firefox Pre-installed extension in Firefox]<br />
* [https://wiki.mozilla.org/Add-ons/Permissions/Testplan Permissions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/SHA SHA]<br />
* [https://wiki.mozilla.org/QA/Theming/Testplan Theming]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Telemetry_for_add-ons_manager Telemetry_for_add-ons_manager]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Permissions_Android Webextensions Permissions Android]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Language_Packs Webextensions Language Packs]<br />
* [https://wiki.mozilla.org/QA/WebExtensions WebExtensions-Plaform]<br />
* [https://wiki.mozilla.org/Add-ons/Webextensions_Cutover/Testplan WebExtensions Cutover]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Cutover_Android Webextensions Cutover Android]<br />
<br />
== AMO TestPlans==<br />
* [https://wiki.mozilla.org/QA/AMO_WebExtensions Webextension-AMO]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Activity Activity Mail]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Listed-Unlisted_Submissions New Submissions Flow]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Restrict_Legacy_Add-ons_Submission Restrict Legacy Add-ons Submission]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/NewAMOMobile New Mobile Frontend]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Auto-Approvals Auto-Approvals]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_Post_Reviews_Process Post Reviews Process]</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplans&diff=1207735Add-ons/QA/Testplans2019-02-14T09:33:21Z<p>CosminB: /* WebExtensions TestPlans */</p>
<hr />
<div>== QA Contacts == <br />
<br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
WebExtensions QA: [mailto:marius.santa@softvision.ro Marius Santa]; irc nick :Santa<br /><br />
WebExtensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
Add-ons QA: [mailto:valentina.peleskei@softvision.ro Valentina Peleskei]; irc nick :ValentinaV<br /><br />
<br />
== WebExtensions TestPlans==<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/about:addons_non-xul_rebuild about:addons_non-xul_rebuild]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_ContentScript Add-ons ContentScript]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-on_Compatibility_Check Add-on Compatibility Check]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Android_Webextension_API Android Webextension API]<br />
* [https://wiki.mozilla.org/WebExtensions/chrome.storage.sync chrome.storage.sync]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/CSP CSP]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Context_Menu_Updates Context Menu Updates]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Content_handler_API Content handler API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_Storage_Local_API Improve Storage Local API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_keyboard_shortcut_support Improve_keyboard_shortcut_support]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_support_for_private_browsing_mode Improve_support_for_private_browsing_mode]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Limited_Tab_Hiding Limited_Tab_Hiding]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Language_Pack_Signing Language Pack Signing]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/OOP-WebExtensions OOP-Webextensions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Pre-installed_extension_in_Firefox Pre-installed extension in Firefox]<br />
* [https://wiki.mozilla.org/Add-ons/Permissions/Testplan Permissions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/SHA SHA]<br />
* [https://wiki.mozilla.org/QA/Theming/Testplan Theming]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Telemetry_for_add-ons_manager Telemetry_for_add-ons_manager]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Permissions_Android Webextensions Permissions Android]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Language_Packs Webextensions Language Packs]<br />
* [https://wiki.mozilla.org/QA/WebExtensions WebExtensions-Plaform]<br />
* [https://wiki.mozilla.org/Add-ons/Webextensions_Cutover/Testplan WebExtensions Cutover]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Cutover_Android Webextensions Cutover Android]<br />
<br />
== AMO TestPlans==<br />
* [https://wiki.mozilla.org/QA/AMO_WebExtensions Webextension-AMO]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Activity Activity Mail]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Listed-Unlisted_Submissions New Submissions Flow]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Restrict_Legacy_Add-ons_Submission Restrict Legacy Add-ons Submission]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/NewAMOMobile New Mobile Frontend]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Auto-Approvals Auto-Approvals]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_Post_Reviews_Process Post Reviews Process]</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/about:addons_non-xul_rebuild&diff=1207734Add-ons/QA/Testplan/about:addons non-xul rebuild2019-02-14T09:32:58Z<p>CosminB: Created blank page</p>
<hr />
<div></div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplans&diff=1206848Add-ons/QA/Testplans2019-01-29T13:45:12Z<p>CosminB: /* WebExtensions TestPlans */</p>
<hr />
<div>== QA Contacts == <br />
<br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
WebExtensions QA: [mailto:marius.santa@softvision.ro Marius Santa]; irc nick :Santa<br /><br />
WebExtensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
Add-ons QA: [mailto:valentina.peleskei@softvision.ro Valentina Peleskei]; irc nick :ValentinaV<br /><br />
<br />
== WebExtensions TestPlans==<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_ContentScript Add-ons ContentScript]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-on_Compatibility_Check Add-on Compatibility Check]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Android_Webextension_API Android Webextension API]<br />
* [https://wiki.mozilla.org/WebExtensions/chrome.storage.sync chrome.storage.sync]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/CSP CSP]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Context_Menu_Updates Context Menu Updates]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Content_handler_API Content handler API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_Storage_Local_API Improve Storage Local API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_keyboard_shortcut_support Improve_keyboard_shortcut_support]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_support_for_private_browsing_mode Improve_support_for_private_browsing_mode]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Limited_Tab_Hiding Limited_Tab_Hiding]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Language_Pack_Signing Language Pack Signing]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/OOP-WebExtensions OOP-Webextensions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Pre-installed_extension_in_Firefox Pre-installed extension in Firefox]<br />
* [https://wiki.mozilla.org/Add-ons/Permissions/Testplan Permissions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/SHA SHA]<br />
* [https://wiki.mozilla.org/QA/Theming/Testplan Theming]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Telemetry_for_add-ons_manager Telemetry_for_add-ons_manager]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Permissions_Android Webextensions Permissions Android]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Language_Packs Webextensions Language Packs]<br />
* [https://wiki.mozilla.org/QA/WebExtensions WebExtensions-Plaform]<br />
* [https://wiki.mozilla.org/Add-ons/Webextensions_Cutover/Testplan WebExtensions Cutover]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Cutover_Android Webextensions Cutover Android]<br />
<br />
== AMO TestPlans==<br />
* [https://wiki.mozilla.org/QA/AMO_WebExtensions Webextension-AMO]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Activity Activity Mail]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Listed-Unlisted_Submissions New Submissions Flow]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Restrict_Legacy_Add-ons_Submission Restrict Legacy Add-ons Submission]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/NewAMOMobile New Mobile Frontend]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Auto-Approvals Auto-Approvals]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_Post_Reviews_Process Post Reviews Process]</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Improve_support_for_private_browsing_mode&diff=1206847Add-ons/QA/Testplan/Improve support for private browsing mode2019-01-29T13:42:28Z<p>CosminB: Created page with "'''Revision History''' {| class="wikitable" style="width:60%" |- ! Date !! Version !! Author !! Description |- | 01/29/2018 || 1.0 || Cosmin Badescu || Created first draft |..."</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 01/29/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
There are two aspects to extensions running in private browsing tabs: user choice, and developer preference.<br />
<br />
*User Choice: the user should be able to determine which, if any, extensions should be able to run within private browsing tabs.<br />
*Developer Preference: the developer should be able to specify if their extension can or cannot be run in private browsing tabs.<br />
<br />
Mozilla and independent research strongly support the idea that user control of extensions in private browsing is an important part of protecting users online.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All the required Telemetry is in place<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* The behavior of the storage.local JSONFile backend and storage.local IndexedDB backend when the extensions use them.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mixedpuppy@gmail.com Shane Caraveo]; irc nick :mixedpuppy <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.allowPrivateBrowsingByDefault from about:config<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.allowPrivateBrowsingByDefault from about:config<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 01-29-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | 12-15-2018 || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - [https://docs.google.com/document/d/1TJ9ncwORJ3nkQ4te78NU6lFmR-3kKTcKCqX7Gq1wQyo/edit]<br />
Install flow - N/A<br />
<br />
<br />
<br />
* {{Bug|1460738}} - [META] improve support for Private Browsing<br />
<bugzilla><br />
{<br />
"blocks":[1460738],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
* {{Bug|1380809}} - [META] manifest incognito feature support<br />
<bugzilla><br />
{<br />
"blocks":[1380809],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [https://testrail.stage.mozaws.net/index.php?/suites/view/3166&group_by=cases:section_id&group_order=asc Improve support for private browsing mode]<br />
<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 05-22-2018 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | 05-21-2018 || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplans&diff=1206846Add-ons/QA/Testplans2019-01-29T13:31:03Z<p>CosminB: /* WebExtensions TestPlans */</p>
<hr />
<div>== QA Contacts == <br />
<br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
WebExtensions QA: [mailto:marius.santa@softvision.ro Marius Santa]; irc nick :Santa<br /><br />
WebExtensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
Add-ons QA: [mailto:valentina.peleskei@softvision.ro Valentina Peleskei]; irc nick :ValentinaV<br /><br />
<br />
== WebExtensions TestPlans==<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_ContentScript Add-ons ContentScript]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-on_Compatibility_Check Add-on Compatibility Check]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Android_Webextension_API Android Webextension API]<br />
* [https://wiki.mozilla.org/WebExtensions/chrome.storage.sync chrome.storage.sync]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/CSP CSP]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Context_Menu_Updates Context Menu Updates]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Content_handler_API Content handler API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_Storage_Local_API Improve Storage Local API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_keyboard_shortcut_support Improve_keyboard_shortcut_support]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Limited_Tab_Hiding Limited_Tab_Hiding]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Language_Pack_Signing Language Pack Signing]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/OOP-WebExtensions OOP-Webextensions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Pre-installed_extension_in_Firefox Pre-installed extension in Firefox]<br />
* [https://wiki.mozilla.org/Add-ons/Permissions/Testplan Permissions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/SHA SHA]<br />
* [https://wiki.mozilla.org/QA/Theming/Testplan Theming]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Telemetry_for_add-ons_manager Telemetry_for_add-ons_manager]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Permissions_Android Webextensions Permissions Android]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Language_Packs Webextensions Language Packs]<br />
* [https://wiki.mozilla.org/QA/WebExtensions WebExtensions-Plaform]<br />
* [https://wiki.mozilla.org/Add-ons/Webextensions_Cutover/Testplan WebExtensions Cutover]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Cutover_Android Webextensions Cutover Android]<br />
<br />
== AMO TestPlans==<br />
* [https://wiki.mozilla.org/QA/AMO_WebExtensions Webextension-AMO]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Activity Activity Mail]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Listed-Unlisted_Submissions New Submissions Flow]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Restrict_Legacy_Add-ons_Submission Restrict Legacy Add-ons Submission]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/NewAMOMobile New Mobile Frontend]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Auto-Approvals Auto-Approvals]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_Post_Reviews_Process Post Reviews Process]</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Improve_keyboard_shortcut_support&diff=1206845Add-ons/QA/Testplan/Improve keyboard shortcut support2019-01-29T13:30:03Z<p>CosminB: Created page with "'''Revision History''' {| class="wikitable" style="width:60%" |- ! Date !! Version !! Author !! Description |- | 01/29/2018 || 1.0 || Cosmin Badescu || Created first draft |..."</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 01/29/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
Allow the user to understand what the current keyboard mapping is, highlighting keyboard mappings that are overriden by add-ons, and providing a UI for users to adjust keyboard shortcuts used by web extensions.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All the required Telemetry is in place<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* The behavior of the storage.local JSONFile backend and storage.local IndexedDB backend when the extensions use them.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:mstriemer@mozilla.com Mark Striemer]; irc nick :mstriemer <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*N/A<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 01-29-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | 12-15-2018 || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - [https://docs.google.com/document/d/1D0XRkNaO5YgwwSPg6JZn5lsdV-EyrlPbvZMqr4jeOVo/edit]<br />
Install flow - [https://mozilla.invisionapp.com/share/Y2MK3ZS8ZUP#/screens/299111761]<br />
<br />
<br />
<br />
* {{Bug|1215061}} - [META] Better keyboard shortcut support<br />
<bugzilla><br />
{<br />
"blocks":[1215061],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
* {{Bug|1303384}} - UI for re-assigning an extension's command shortcut<br />
<bugzilla><br />
{<br />
"blocks":[1303384],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [https://testrail.stage.mozaws.net/index.php?/suites/view/3177&group_by=cases:section_id&group_order=asc Improve keyboard shortcut support]<br />
<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 05-22-2018 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | 05-21-2018 || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplans&diff=1203587Add-ons/QA/Testplans2018-11-08T10:10:22Z<p>CosminB: /* WebExtensions TestPlans */</p>
<hr />
<div>== QA Contacts == <br />
<br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
WebExtensions QA: [mailto:marius.santa@softvision.ro Marius Santa]; irc nick :Santa<br /><br />
WebExtensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
Add-ons QA: [mailto:valentina.peleskei@softvision.ro Valentina Peleskei]; irc nick :ValentinaV<br /><br />
<br />
== WebExtensions TestPlans==<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_ContentScript Add-ons ContentScript]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-on_Compatibility_Check Add-on Compatibility Check]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Android_Webextension_API Android Webextension API]<br />
* [https://wiki.mozilla.org/WebExtensions/chrome.storage.sync chrome.storage.sync]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/CSP CSP]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Context_Menu_Updates Context Menu Updates]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Content_handler_API Content handler API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_Storage_Local_API Improve Storage Local API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Limited_Tab_Hiding Limited_Tab_Hiding]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Language_Pack_Signing Language Pack Signing]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/OOP-WebExtensions OOP-Webextensions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Pre-installed_extension_in_Firefox Pre-installed extension in Firefox]<br />
* [https://wiki.mozilla.org/Add-ons/Permissions/Testplan Permissions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/SHA SHA]<br />
* [https://wiki.mozilla.org/QA/Theming/Testplan Theming]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Telemetry_for_add-ons_manager Telemetry_for_add-ons_manager]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Permissions_Android Webextensions Permissions Android]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Language_Packs Webextensions Language Packs]<br />
* [https://wiki.mozilla.org/QA/WebExtensions WebExtensions-Plaform]<br />
* [https://wiki.mozilla.org/Add-ons/Webextensions_Cutover/Testplan WebExtensions Cutover]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Cutover_Android Webextensions Cutover Android]<br />
<br />
== AMO TestPlans==<br />
* [https://wiki.mozilla.org/QA/AMO_WebExtensions Webextension-AMO]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Activity Activity Mail]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Listed-Unlisted_Submissions New Submissions Flow]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Restrict_Legacy_Add-ons_Submission Restrict Legacy Add-ons Submission]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/NewAMOMobile New Mobile Frontend]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Auto-Approvals Auto-Approvals]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_Post_Reviews_Process Post Reviews Process]</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplans&diff=1203584Add-ons/QA/Testplans2018-11-08T10:09:33Z<p>CosminB: /* WebExtensions TestPlans */</p>
<hr />
<div>== QA Contacts == <br />
<br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
WebExtensions QA: [mailto:marius.santa@softvision.ro Marius Santa]; irc nick :Santa<br /><br />
WebExtensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
Add-ons QA: [mailto:valentina.peleskei@softvision.ro Valentina Peleskei]; irc nick :ValentinaV<br /><br />
<br />
== WebExtensions TestPlans==<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_ContentScript Add-ons ContentScript]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-on_Compatibility_Check Add-on Compatibility Check]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Android_Webextension_API Android Webextension API]<br />
* [https://wiki.mozilla.org/WebExtensions/chrome.storage.sync chrome.storage.sync]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/CSP CSP]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Context_Menu_Updates Context Menu Updates]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Content_handler_API Content handler API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_Storage_Local_API Improve Storage Local API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Limited_Tab_Hiding Limited_Tab_Hiding]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Language_Pack_Signing Language Pack Signing]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/OOP-WebExtensions OOP-Webextensions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Pre-installed_extension_in_Firefox Pre-installed extension in Firefox]<br />
* [https://wiki.mozilla.org/Add-ons/Permissions/Testplan Permissions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/SHA SHA]<br />
* [https://wiki.mozilla.org/QA/Theming/Testplan Theming]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Permissions_Android Webextensions Permissions Android]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Language_Packs Webextensions Language Packs]<br />
* [https://wiki.mozilla.org/QA/WebExtensions WebExtensions-Plaform]<br />
* [https://wiki.mozilla.org/Add-ons/Webextensions_Cutover/Testplan WebExtensions Cutover]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Telemetry_for_add-ons_manager Telemetry_for_add-ons_manager]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Cutover_Android Webextensions Cutover Android]<br />
<br />
== AMO TestPlans==<br />
* [https://wiki.mozilla.org/QA/AMO_WebExtensions Webextension-AMO]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Activity Activity Mail]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Listed-Unlisted_Submissions New Submissions Flow]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Restrict_Legacy_Add-ons_Submission Restrict Legacy Add-ons Submission]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/NewAMOMobile New Mobile Frontend]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Auto-Approvals Auto-Approvals]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_Post_Reviews_Process Post Reviews Process]</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Telemetry_for_add-ons_manager&diff=1202262Add-ons/QA/Testplan/Telemetry for add-ons manager2018-10-12T08:32:52Z<p>CosminB: /* Test suite */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 10/12/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
The improvement of the extensions install process also involves the telemetry part, where we can measure how successfully people navigate that flow.<br />
Therefore, implementation over the number of started, successful, user abort, time to success / abortion, duration of download and number of permissions listed, have been added with this feature.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Context Menu Updates and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All the required Telemetry is in place<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* Extensions actions, like install/update/uninstall for telemetry events are collected, displayed, measured etc. as expected.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:lgreco@mozilla.com Luca Greco]; irc nick :rpl<br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*TBD<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*TBD<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 10-12-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [TestRail]<br />
|-<br />
| Bugs management || [Bugzilla] / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - TBD<br />
Install flow - TBD<br />
<br />
<br />
<br />
* {{Bug|1465143}} - [META] Telemetry completion<br />
<bugzilla><br />
{<br />
"blocks":[1465143],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
* {{Bug|1316074}} - get telemetry on started and completed extension/theme installs<br />
<bugzilla><br />
{<br />
"blocks":[1316074],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [https://testrail.stage.mozaws.net/index.php?/suites/view/2950&group_by=cases:section_id&group_order=asc TestRail tests]<br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 10-12-2018 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Telemetry_for_add-ons_manager&diff=1202261Add-ons/QA/Testplan/Telemetry for add-ons manager2018-10-12T08:32:32Z<p>CosminB: /* Testing Tools */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 10/12/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
The improvement of the extensions install process also involves the telemetry part, where we can measure how successfully people navigate that flow.<br />
Therefore, implementation over the number of started, successful, user abort, time to success / abortion, duration of download and number of permissions listed, have been added with this feature.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Context Menu Updates and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All the required Telemetry is in place<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* Extensions actions, like install/update/uninstall for telemetry events are collected, displayed, measured etc. as expected.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:lgreco@mozilla.com Luca Greco]; irc nick :rpl<br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*TBD<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*TBD<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 10-12-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [TestRail]<br />
|-<br />
| Bugs management || [Bugzilla] / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - TBD<br />
Install flow - TBD<br />
<br />
<br />
<br />
* {{Bug|1465143}} - [META] Telemetry completion<br />
<bugzilla><br />
{<br />
"blocks":[1465143],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
* {{Bug|1316074}} - get telemetry on started and completed extension/theme installs<br />
<bugzilla><br />
{<br />
"blocks":[1316074],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [ TestRail tests]<br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 10-12-2018 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Telemetry_for_add-ons_manager&diff=1202260Add-ons/QA/Testplan/Telemetry for add-ons manager2018-10-12T08:30:24Z<p>CosminB: /* what's in scope? */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 10/12/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
The improvement of the extensions install process also involves the telemetry part, where we can measure how successfully people navigate that flow.<br />
Therefore, implementation over the number of started, successful, user abort, time to success / abortion, duration of download and number of permissions listed, have been added with this feature.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Context Menu Updates and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All the required Telemetry is in place<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* Extensions actions, like install/update/uninstall for telemetry events are collected, displayed, measured etc. as expected.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:lgreco@mozilla.com Luca Greco]; irc nick :rpl<br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*TBD<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*TBD<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 10-12-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [https://testrail.stage.mozaws.net/index.php?/suites/view/2950&group_by=cases:section_id&group_order=asc TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [https://testrail.stage.mozaws.net/index.php?/suites/view/2950&group_by=cases:section_id&group_order=asc TestRail]<br />
|-<br />
| Bugs management || [https://bugzilla.mozilla.org/show_bug.cgi?id=1316074 Bugzilla] / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - TBD<br />
Install flow - TBD<br />
<br />
<br />
<br />
* {{Bug|1465143}} - [META] Telemetry completion<br />
<bugzilla><br />
{<br />
"blocks":[1465143],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
* {{Bug|1316074}} - get telemetry on started and completed extension/theme installs<br />
<bugzilla><br />
{<br />
"blocks":[1316074],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [ TestRail tests]<br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 10-12-2018 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Telemetry_for_add-ons_manager&diff=1202259Add-ons/QA/Testplan/Telemetry for add-ons manager2018-10-12T08:30:10Z<p>CosminB: /* Overview */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 10/12/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
The improvement of the extensions install process also involves the telemetry part, where we can measure how successfully people navigate that flow.<br />
Therefore, implementation over the number of started, successful, user abort, time to success / abortion, duration of download and number of permissions listed, have been added with this feature.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Context Menu Updates and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All the required Telemetry is in place<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* Extensions actions like install/update/uninstall for telemetry events are collected, displayed, measured etc. as expected.<br />
<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:lgreco@mozilla.com Luca Greco]; irc nick :rpl<br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*TBD<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*TBD<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 10-12-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [https://testrail.stage.mozaws.net/index.php?/suites/view/2950&group_by=cases:section_id&group_order=asc TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [https://testrail.stage.mozaws.net/index.php?/suites/view/2950&group_by=cases:section_id&group_order=asc TestRail]<br />
|-<br />
| Bugs management || [https://bugzilla.mozilla.org/show_bug.cgi?id=1316074 Bugzilla] / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - TBD<br />
Install flow - TBD<br />
<br />
<br />
<br />
* {{Bug|1465143}} - [META] Telemetry completion<br />
<bugzilla><br />
{<br />
"blocks":[1465143],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
* {{Bug|1316074}} - get telemetry on started and completed extension/theme installs<br />
<bugzilla><br />
{<br />
"blocks":[1316074],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [ TestRail tests]<br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 10-12-2018 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Telemetry_for_add-ons_manager&diff=1202258Add-ons/QA/Testplan/Telemetry for add-ons manager2018-10-12T08:27:09Z<p>CosminB: /* Overview */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 10/12/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
The improvement of the add-on install process also involves the telemetry part, where we can measure how successfully people navigate that flow.<br />
Therefore, implementation over the number of started, successful, user abort, time to success / abortion, duration of download and number of permissions listed, have been added with this feature.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Context Menu Updates and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All the required Telemetry is in place<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* TBD.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:lgreco@mozilla.com Luca Greco]; irc nick :rpl<br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*TBD<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*TBD<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 10-12-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [https://testrail.stage.mozaws.net/index.php?/suites/view/2950&group_by=cases:section_id&group_order=asc TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [https://testrail.stage.mozaws.net/index.php?/suites/view/2950&group_by=cases:section_id&group_order=asc TestRail]<br />
|-<br />
| Bugs management || [https://bugzilla.mozilla.org/show_bug.cgi?id=1316074 Bugzilla] / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - TBD<br />
Install flow - TBD<br />
<br />
<br />
<br />
* {{Bug|1465143}} - [META] Telemetry completion<br />
<bugzilla><br />
{<br />
"blocks":[1465143],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
* {{Bug|1316074}} - get telemetry on started and completed extension/theme installs<br />
<bugzilla><br />
{<br />
"blocks":[1316074],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [ TestRail tests]<br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 10-12-2018 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Telemetry_for_add-ons_manager&diff=1202257Add-ons/QA/Testplan/Telemetry for add-ons manager2018-10-12T08:13:56Z<p>CosminB: /* Testing Tools */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 10/12/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
TBD.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Context Menu Updates and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All the required Telemetry is in place<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* TBD.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:lgreco@mozilla.com Luca Greco]; irc nick :rpl<br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*TBD<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*TBD<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 10-12-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [https://testrail.stage.mozaws.net/index.php?/suites/view/2950&group_by=cases:section_id&group_order=asc TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [https://testrail.stage.mozaws.net/index.php?/suites/view/2950&group_by=cases:section_id&group_order=asc TestRail]<br />
|-<br />
| Bugs management || [https://bugzilla.mozilla.org/show_bug.cgi?id=1316074 Bugzilla] / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - TBD<br />
Install flow - TBD<br />
<br />
<br />
<br />
* {{Bug|1465143}} - [META] Telemetry completion<br />
<bugzilla><br />
{<br />
"blocks":[1465143],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
* {{Bug|1316074}} - get telemetry on started and completed extension/theme installs<br />
<bugzilla><br />
{<br />
"blocks":[1316074],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [ TestRail tests]<br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 10-12-2018 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Telemetry_for_add-ons_manager&diff=1202256Add-ons/QA/Testplan/Telemetry for add-ons manager2018-10-12T08:12:32Z<p>CosminB: /* Bug Work */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 10/12/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
TBD.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Context Menu Updates and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All the required Telemetry is in place<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* TBD.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:lgreco@mozilla.com Luca Greco]; irc nick :rpl<br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*TBD<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*TBD<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 10-12-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [https://testrail.stage.mozaws.net/index.php?/suites/view/2950&group_by=cases:section_id&group_order=asc TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [https://testrail.stage.mozaws.net/index.php?/suites/view/2950&group_by=cases:section_id&group_order=asc TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - TBD<br />
Install flow - TBD<br />
<br />
<br />
<br />
* {{Bug|1465143}} - [META] Telemetry completion<br />
<bugzilla><br />
{<br />
"blocks":[1465143],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
* {{Bug|1316074}} - get telemetry on started and completed extension/theme installs<br />
<bugzilla><br />
{<br />
"blocks":[1316074],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [ TestRail tests]<br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 10-12-2018 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Telemetry_for_add-ons_manager&diff=1202255Add-ons/QA/Testplan/Telemetry for add-ons manager2018-10-12T08:11:17Z<p>CosminB: </p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 10/12/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
TBD.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Context Menu Updates and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All the required Telemetry is in place<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* TBD.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:lgreco@mozilla.com Luca Greco]; irc nick :rpl<br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*TBD<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*TBD<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 10-12-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [https://testrail.stage.mozaws.net/index.php?/suites/view/2950&group_by=cases:section_id&group_order=asc TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [https://testrail.stage.mozaws.net/index.php?/suites/view/2950&group_by=cases:section_id&group_order=asc TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - TBD<br />
Install flow - TBD<br />
<br />
<br />
<br />
* {{Bug|1465143}} - [META] Telemetry completion<br />
<bugzilla><br />
{<br />
"blocks":[1465143],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
* {{Bug|1316074}} - get telemetry on started and completed extension/theme installs<br />
<bugzilla><br />
{<br />
"blocks":[1316074],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [ TestRail tests]<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 10-12-2018 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplans&diff=1202254Add-ons/QA/Testplans2018-10-12T08:02:43Z<p>CosminB: /* WebExtensions TestPlans */</p>
<hr />
<div>== QA Contacts == <br />
<br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
WebExtensions QA: [mailto:marius.santa@softvision.ro Marius Santa]; irc nick :Santa<br /><br />
WebExtensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
Add-ons QA: [mailto:valentina.peleskei@softvision.ro Valentina Peleskei]; irc nick :ValentinaV<br /><br />
<br />
== WebExtensions TestPlans==<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_ContentScript Add-ons ContentScript]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-on_Compatibility_Check Add-on Compatibility Check]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Android_Webextension_API Android Webextension API]<br />
* [https://wiki.mozilla.org/WebExtensions/chrome.storage.sync chrome.storage.sync]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/CSP CSP]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Context_Menu_Updates Context Menu Updates]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_Storage_Local_API Improve Storage Local API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Limited_Tab_Hiding Limited_Tab_Hiding]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Language_Pack_Signing Language Pack Signing]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/OOP-WebExtensions OOP-Webextensions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Pre-installed_extension_in_Firefox Pre-installed extension in Firefox]<br />
* [https://wiki.mozilla.org/Add-ons/Permissions/Testplan Permissions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/SHA SHA]<br />
* [https://wiki.mozilla.org/QA/Theming/Testplan Theming]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Permissions_Android Webextensions Permissions Android]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Language_Packs Webextensions Language Packs]<br />
* [https://wiki.mozilla.org/QA/WebExtensions WebExtensions-Plaform]<br />
* [https://wiki.mozilla.org/Add-ons/Webextensions_Cutover/Testplan WebExtensions Cutover]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Cutover_Android Webextensions Cutover Android]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Content_handler_API Content handler API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Telemetry_for_add-ons_manager Telemetry_for_add-ons_manager]<br />
<br />
== AMO TestPlans==<br />
* [https://wiki.mozilla.org/QA/AMO_WebExtensions Webextension-AMO]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Activity Activity Mail]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Listed-Unlisted_Submissions New Submissions Flow]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Restrict_Legacy_Add-ons_Submission Restrict Legacy Add-ons Submission]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/NewAMOMobile New Mobile Frontend]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Auto-Approvals Auto-Approvals]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_Post_Reviews_Process Post Reviews Process]</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplans&diff=1202253Add-ons/QA/Testplans2018-10-12T08:02:17Z<p>CosminB: /* WebExtensions TestPlans */</p>
<hr />
<div>== QA Contacts == <br />
<br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
WebExtensions QA: [mailto:marius.santa@softvision.ro Marius Santa]; irc nick :Santa<br /><br />
WebExtensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
Add-ons QA: [mailto:valentina.peleskei@softvision.ro Valentina Peleskei]; irc nick :ValentinaV<br /><br />
<br />
== WebExtensions TestPlans==<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_ContentScript Add-ons ContentScript]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-on_Compatibility_Check Add-on Compatibility Check]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Android_Webextension_API Android Webextension API]<br />
* [https://wiki.mozilla.org/WebExtensions/chrome.storage.sync chrome.storage.sync]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/CSP CSP]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Context_Menu_Updates Context Menu Updates]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Improve_Storage_Local_API Improve Storage Local API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Limited_Tab_Hiding Limited_Tab_Hiding]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Language_Pack_Signing Language Pack Signing]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/OOP-WebExtensions OOP-Webextensions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Pre-installed_extension_in_Firefox Pre-installed extension in Firefox]<br />
* [https://wiki.mozilla.org/Add-ons/Permissions/Testplan Permissions]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/SHA SHA]<br />
* [https://wiki.mozilla.org/QA/Theming/Testplan Theming]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Permissions_Android Webextensions Permissions Android]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Language_Packs Webextensions Language Packs]<br />
* [https://wiki.mozilla.org/QA/WebExtensions WebExtensions-Plaform]<br />
* [https://wiki.mozilla.org/Add-ons/Webextensions_Cutover/Testplan WebExtensions Cutover]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Webextensions_Cutover_Android Webextensions Cutover Android]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Content_handler_API Content handler API]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Telemetry_for_add-ons_manager]<br />
<br />
== AMO TestPlans==<br />
* [https://wiki.mozilla.org/QA/AMO_WebExtensions Webextension-AMO]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Activity Activity Mail]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Listed-Unlisted_Submissions New Submissions Flow]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Restrict_Legacy_Add-ons_Submission Restrict Legacy Add-ons Submission]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/NewAMOMobile New Mobile Frontend]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Auto-Approvals Auto-Approvals]<br />
* [https://wiki.mozilla.org/Add-ons/QA/Testplan/Add-ons_Post_Reviews_Process Post Reviews Process]</div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Telemetry_for_add-ons_manager&diff=1202252Add-ons/QA/Testplan/Telemetry for add-ons manager2018-10-12T08:01:44Z<p>CosminB: Created blank page</p>
<hr />
<div></div>CosminBhttps://wiki.mozilla.org/index.php?title=Add-ons/QA/Testplan/Improve_Storage_Local_API&diff=1199264Add-ons/QA/Testplan/Improve Storage Local API2018-08-10T13:14:10Z<p>CosminB: /* Test suite */</p>
<hr />
<div>'''Revision History'''<br />
<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Date !! Version !! Author !! Description <br />
|-<br />
| 05/22/2018 || 1.0 || Cosmin Badescu || Created first draft<br />
|-<br />
|}<br />
<br />
<br />
= Overview =<br />
The purpose of this feature is to check that the migration from storage.local JSONFile backend to storage.local IndexedDB backend works correctly.<br />
<br />
== Purpose ==<br />
This document purports to detail the test approach for the improvements of Local Storage API and includes Entry/Exit criteria, Scope for testing, links to test cases etc<br />
<br />
=== Entry Criteria ===<br />
* QA has access to all the PRDs, mocks and related documents<br />
* The feature has landed on Nightly<br />
* AMO parts has landed on dev<br />
<br />
=== Exit Criteria ===<br />
* All the bugs against the feature have been triaged<br />
* All the P1/P2 bugs have been fixed<br />
* All the resolved bugs have been verified by QA<br />
* The find/fixed rate is going down over a predefined period of time<br />
<br />
=== Acceptance Criteria ===<br />
This section broadly outlines when the product is ready to ship<br />
* QA has signed off<br />
* All the required Telemetry is in place<br />
* All info is localized at least for a pre-defined set of locales<br />
* All the necessary PR/blogposts have been sent out<br />
<br />
== Scope ==<br />
This section describes what parts of the feature will be tested and what parts won't be.<br />
<br />
===what's in scope?===<br />
* The behavior of the storage.local JSONFile backend and storage.local IndexedDB backend when the extensions use them.<br />
<br />
===what's out of scope?===<br />
* Security testing<br />
<br />
== Ownership ==<br />
Dev Lead: [mailto:lgreco@mozilla.com Luca Greco]; irc nick :rpl <br /><br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br /> <br />
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br /><br />
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br /><br />
<br />
<br />
= Requirements for testing =<br />
== Environments ==<br />
OSes covered: Windows, Mac OS X, Linux<br />
<br />
== Channel dependent settings (configs) and environment setups ==<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
=== Nightly ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.webextensions.ExtensionStorageIDB.enabled from about:config<br />
</div><br />
<br />
=== Beta ===<br />
<div class="mw-collapsible-content"><br />
*Enable extensions.webextensions.ExtensionStorageIDB.enabled from about:config<br />
</div><br />
<br />
=== Release ===<br />
<div class="mw-collapsible-content"><br />
The feature is enabled by default<br />
</div><br />
</div><br />
<br />
= Test Strategy = <br />
<br />
== Test Objectives ==<br />
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.<br />
This could be documented in bullet form or in a table similar to the one below.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Ref !! Function !! Test Objective !! Test Type !! Owners <br />
|-<br />
| TO-1 || Installing from AMO || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-2 || Installing from local files || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-3 || Add-on updates || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|-<br />
| TO-4 || Sideloading || To verify that the extension uses the API correctly || Manual || Add-ons QA Team<br />
|}<br />
<br />
== Builds ==<br />
This section should contain links for builds with the feature - <br />
* [http://archive.mozilla.org/pub/mobile/nightly/latest-date-android-api-16/ Link] for Nightly builds<br />
* [http://archive.mozilla.org/pub/mobile/candidates/ Link] for Beta builds<br />
* [http://archive.mozilla.org/pub/mobile/releases/ Link] for Release builds<br />
<br />
== Test Execution Schedule ==<br />
The following table identifies the anticipated testing period available for test execution.<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Project phase !! Start Date !! End Date<br />
|-<br />
| Start project <br />
|style="text-align:center;" | || <br />
|-<br />
| Study documentation/specs received from developers<br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Test plan creation <br />
|style="text-align:center;" | 05-22-2018 || <br />
|-<br />
| QA - Test cases/Env preparation <br />
|style="text-align:center;" | 05-21-2018 || N/A<br />
|-<br />
| QA - Nightly Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| QA - Beta Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| Release Date <br />
|style="text-align:center;" | ||<br />
|}<br />
<br />
== Testing Tools ==<br />
Detail the tools to be used for testing, for example see the following table:<br />
{| class="wikitable" style="width:50%"<br />
|-<br />
! Process !! Tool<br />
|-<br />
| Test plan creation || Mozilla wiki<br />
|-<br />
| Test case creation || [ Docs] / [ TestRail]<br />
|-<br />
| Test case execution || [ Docs] / [ TestRail]<br />
|-<br />
| Bugs management || Bugzilla / Github<br />
|}<br />
<br />
= Status = <br />
== Overview ==<br />
Track the dates and build number where feature was released to Nightly<br />
Track the dates and build number where feature was merged to Release/Beta<br />
<br />
== Risk analysis ==<br />
Identify the high-risk assumptions<br />
Identify existing bugs on the feature with high risk<br />
Identify if other areas are affected by the fix<br />
<br />
= References =<br />
* List and links for specs<br />
PRD - TBD<br />
Install flow - [https://docs.google.com/document/d/1IDNHoneiARuw8PT6KIQGp03WxFmaQYijpluZA0C-E7I/edit Link]<br />
<br />
<br />
<br />
* {{Bug|1406181}} - Use indexedDB as the backend for storage.local (pref Off)<br />
<bugzilla><br />
{<br />
"blocks":[1406181],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
* {{Bug|1474562}} - Enable ExtensionStorageIDB backend on Nightly (pref ON)<br />
<bugzilla><br />
{<br />
"blocks":[1474562],<br />
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"<br />
}<br />
</bugzilla><br />
<br />
= Testcases = <br />
== Overview ==<br />
Summary of testing scenarios<br />
== Test Areas ==<br />
{| class="wikitable" style="width:80%"<br />
|-<br />
! Test Areas !! Covered !! Details<br />
|-<br />
| '''Installing from AMO'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from local files'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Installing from thirdparty'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Add-on updates'''<br />
|style="text-align:center;" | || <br />
|-<br />
| '''Sideloading''' <br />
|| || <br />
|-<br />
| '''Other''' <br />
|style="text-align:center;" | ||<br />
|-<br />
|}<br />
<br />
== Test suite ==<br />
* Link for the [ Initial test planning]<br />
* Link for the [ Google doc tests]<br />
* Link for the [https://testrail.stage.mozaws.net/index.php?/suites/view/2712&group_by=cases:section_id&group_order=asc TestRail tests pref'd OFF]<br />
* Link for the [https://testrail.stage.mozaws.net/index.php?/suites/view/2602&group_by=cases:section_id&group_order=asc TestRail tests pref'd ON]<br />
<br />
= Bug Work =<br />
Tracking bug - []<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Bug fix verification ======<br />
<div class="mw-collapsible-content"><br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Display permissions prompt for webextensions installed using mozAddonManager<br />
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms<br />
<font color="green">'''[Verified]'''</font> [ Bug xxxxxxx] - Prompt users with permissions for third-party webextensions installs<br />
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms<br />
</div></div><br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"><br />
<br />
====== Logged bugs ======<br />
<div class="mw-collapsible-content"><br />
[ Bug xxxxxxx] - Misaligned icon and webextension name in permissions doorhanger<br/> <br />
<br />
</div><br />
</div><br />
<br />
= Sign off =<br />
== Criteria ==<br />
Check list<br />
* All test cases should be executed<br />
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan<br />
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)<br />
<br />
== Results ==<br />
'''Nightly testing'''<br /><br />
<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite, use template from []<br />
<br />
<br />
'''Merge to Beta Sign-off'''<br />
List of OSes that will be covered by testing<br /><br />
*Link for the tests run<br />
** Full Test suite<br />
<br />
== Checklist ==<br />
{| class="wikitable" style="width:60%"<br />
|-<br />
! Exit Criteria !! Status !! Notes/Details<br />
|-<br />
| Testing Prerequisites (specs, use cases) <br />
| style="text-align:center;" | <br />
| style="text-align:center;" | <br />
|-<br />
| Testing Infrastructure setup <br />
|style="text-align:center;" | || <br />
|-<br />
| Test Plan Creation <br />
| style="text-align:center;" | 05-22-2018 || <br />
|-<br />
| Test Cases Creation <br />
|style="text-align:center;" | 05-21-2018 || N/A<br />
|-<br />
| Full Functional Tests Execution <br />
|style="text-align:center;" | <br />
|style="text-align:center;" | <br />
|-<br />
| Automation Coverage ||<br />
|style="text-align:center;" | <br />
|-<br />
| Performance Testing <br />
|style="text-align:center;" | || <br />
|-<br />
| All Defects Logged || || <br />
|-<br />
| Critical/Blockers Fixed and Verified || || <br />
|-<br />
| Metrics/Telemetry|| <br />
|style="text-align:center;" | <br />
|-<br />
| QA Signoff - Nightly Release|| <br />
|style="text-align:center;" | Email to be sent <br />
|-<br />
| QA Beta - Full Testing<br />
|style="text-align:center;" | || <br />
|-<br />
| QA Signoff - Beta Release || <br />
|style="text-align:center;" | Email to be sent <br />
|}</div>CosminB