Add-ons/Permissions/Testplan: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
 
(49 intermediate revisions by 3 users not shown)
Line 7: Line 7:
| 01/09/2017 || 1.0 || Krupa Raj || Created first draft
| 01/09/2017 || 1.0 || Krupa Raj || Created first draft
|-
|-
|  
| 01/20/2017 || 1.1 || Vasilica Mihasca || Updates
|-
| 01/23/2017 || 1.2 || Krupa Raj || Added Acceptance Criteria
|-
| 03/22/2017 || 1.3 || Vasilica Mihasca || Added TestRail links
|-
| 08/23/2017 || 1.4 || Cosmin Badescu || Updates: References, Overview, Purpose,Requirements for testing, Ownership
|-
|}
|}


= Overview =
= Overview =
Permissions is a feature which allows webextensions to have certain special powers. The permissions key specified in the manifest.json will grant special powers for your extension. This key is an array of strings, and each string is a request for a permission.
If you request permissions using this key, then the browser may inform the user at install time that the extension is requesting certain privileges, and ask them to confirm that they are happy to grant these privileges. The browser may also allow the user to inspect an extension's privileges after installation.
* As part of this feature, the optional permissions are used when they are needed for optional features in your extension.
== Purpose ==
== Purpose ==
Detail the purpose of this document. For example:
This document purports to detail the test approach to Webextension Permissions and Optional permissions including Entry/Exit criteria, Scope for testing, links to testcases etc
* The test scope, focus areas and objectives
 
* The test responsibilities
=== Entry Criteria ===
* The test strategy for the levels and types of test for this release
* QA has access to all the PRDs, mocks and related documents
* The entry and exit criteria
* The feature has landed on Nightly
* The basis of the test estimates
* AMO parts has landed on dev
* Any risks, issues, assumptions and test dependencies
 
* The test schedule and major milestones
=== Exit Criteria ===
* The test deliverables
* All the bugs against the feature have been triaged
* All the P1/P2 bugs have been fixed
* All the resolved bugs have been verified by QA
* The find/fixed rate is going down over a predefined period of time
 
=== Acceptance Criteria ===
This section broadly outlines when the product is ready to ship
* QA has signed off
* All the required Telemetry is in place
* All info is localized at least for a pre-defined set of locales
* All the necessary PR/blogposts have been sent out


== Scope ==
== Scope ==
This wiki details the testing that will be performed by the project team for the <project name> project. It defines the overall testing requirements and provides an integrated view of the project test activities. Its purpose is to document:
This section describes what parts of the feature will be tested and what parts won't be.
* What will be tested
 
* How testing will be performed
===what's in scope?===
* Installation flow from AMO
* Installation flow from Add-ons Manager
* Installation flow from Discopane
* Installation flow from Third-party websites
* Add-on updates
* Add-on Validation related to permissions during submission
* Sideloading(?)
* Webextensions have access to the permissions requested
 
===what's out of scope?===
* Security testing
* Performance testing


== Ownership ==
== Ownership ==
What resources are needed, and when (developers, QA, PMs working on the feature)
Dev Lead: [mailto:aswan@mozilla.com Andrew Swan]; irc nick :aswan<br />
QA Manager: [mailto:kraj@mozilla.com Krupa Raj]; irc nick :krupa<br />
= Testing summary =
QA Lead: [mailto:vcarciu@softvision.ro Victor Carciu]; irc nick :victorc<br />
== Scope of Testing ==
Webextensions QA: [mailto:vasilica.mihasca@softvision.ro Vasilica Mihasca]; irc nick :vasilica_mihasca<br />
=== In Scope ===
Webextensions QA: [mailto:cosmin.badescu@softvision.ro Cosmin Badescu]; irc nick :CosminB<br />
Detail what is in scope from a testing perspective for the project team.
Add-ons QA: [mailto:valentina.peleskei@softvision.ro Valentina Virlics]; irc nick :ValentinaV<br />
 
=== Out of Scope ===
Detail what is out of scope from a testing perspective for the project team. Note: if usability testing is not in the scope of testing feature.


= Requirements for testing =
= Requirements for testing =
== Environments ==
== Environments ==
Specify OSes that need to be covered
OSes covered: Windows, MacOSX, Linux
Specify Devices that need to be covered
Specify other configuration/environmental setup needed
 


== Channel dependent settings (configs) and environment setups ==
== Channel dependent settings (configs) and environment setups ==
Line 50: Line 80:
=== Nightly ===
=== Nightly ===
<div class="mw-collapsible-content">
<div class="mw-collapsible-content">
text
* Enable extensions.webextPermissionPrompts from about:config
* Enable extensions.webextOptionalPermissionPrompts from about:config
* Disable check for signing if testing on AMO dev or stage
</div>
</div>


=== Aurora ===
=== Aurora ===
<div class="mw-collapsible-content">
<div class="mw-collapsible-content">
text
* Enable extensions.webextPermissionPrompts from about:config
* Enable extensions.webextOptionalPermissionPrompts from about:config
* Disable check for signing if testing on AMO dev or stage
</div>
</div>


=== Beta ===
=== Beta ===
<div class="mw-collapsible-content">
<div class="mw-collapsible-content">
text
* Enable extensions.webextPermissionPrompts from about:config
* Enable extensions.webextOptionalPermissionPrompts from about:config
* [Unbranded build] Disable check for signing if testing on AMO dev or stage
 
</div>
</div>


=== Post Beta / Release ===
=== Post Beta / Release ===
<div class="mw-collapsible-content">
<div class="mw-collapsible-content">
text
The feature is enabled by default.
</div>
</div>
</div>
</div>


= Test Strategy =  
= Test Strategy =  
== Test Objectives ==
== Test Objectives ==
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master.
Line 76: Line 114:
{| class="wikitable"
{| class="wikitable"
|-
|-
! Ref !! Function !! Test Objective !! Evaluation Criteria !! Test Type !! Owners  
! Ref !! Function !! Test Objective !! Test Type !! Owners  
|-
| TO-1 || Validation during AMO || To verify that linter raise the proper error messages || Manual || Add-ons QA Team
|-
| TO-2 || AMO public pages || To verify that permissions are correctly displayed || Manual || Add-ons QA Team
|-
| TO-3 || Installing from AMO || To verify that the permissions flow is fully respected || Manual || Add-ons QA Team
|-
| TO-4 || Installing from Discopane || To verify that the permissions flow is fully respected || Manual || Add-ons QA Team
|-
| TO-5 || Installing from Add-ons manager || To verify that the permissions flow is fully respected || Manual || Add-ons QA Team
|-
| TO-6 || Installing via sideloading || To verify that the permissions flow is fully respected || Manual || Add-ons QA Team
|-
|-
| 1 || Name of the feature or sub-function being tested || The objective the test is trying to demonstrate || The criteria that will be evaluated to demonstrate the test is successful || Manual/ Automation/ Regression/ Performance/ Usability/ Security/ Telemetry || Eng Team
| TO-7 || Installing from local files|| To verify that the permissions flow is fully respected || Manual || Add-ons QA Team
|-
|-
| 2 || Repeat for each feature/sub-function || || || ||
| TO-8 || Installing from thirdparty || To verify that the permissions flow is fully respected || Manual || Add-ons QA Team
|-
|-
| 3 || || || || ||
| TO-9 || Add-on updates || To verify that the permissions flow is fully respected || Manual || Add-ons QA Team
|}
|}


== Builds ==
== Builds ==
This section should contain links for builds with the feature -  
This section should contain links for builds with the feature -  
* Links for Nightly builds
* [https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central/ Link] for Nightly builds
* Links for Aurora builds
* [https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-aurora/ Link] for Aurora builds
* Links for Beta builds
* [https://archive.mozilla.org/pub/firefox/candidates/ Link] for Beta builds


== Test Execution Schedule ==
== Test Execution Schedule ==
Line 105: Line 154:
|-
|-
| QA - Test plan creation  
| QA - Test plan creation  
|style="text-align:center;" | ||  
|style="text-align:center;" | 01-09-2017 || 01-11-2017
|-
|-
| QA - Test cases/Env preparation  
| QA - Test cases/Env preparation  
|style="text-align:center;" |  ||  
|style="text-align:center;" |01-09-2017 ||  
|-
|-
| QA - Nightly Testing  
| QA - Nightly Testing  
|style="text-align:center;" |  ||  
|style="text-align:center;" |01-11-2017 ||  
|-
|-
| QA - Aurora Testing
| QA - Aurora Testing
Line 131: Line 180:
| Test plan creation || Mozilla wiki
| Test plan creation || Mozilla wiki
|-
|-
| Test case creation || [https://testrail.stage.mozaws.net/index.php TestRail]/ Google docs
| Test case creation || [https://testrail.stage.mozaws.net/index.php TestRail] / [https://docs.google.com/spreadsheets/d/1b_W1iVVuo-MRtwilKeHt_Vjbemy3j1Pbiq4slL_BhE8/edit#gid=36647849 Google docs] / [https://public.etherpad-mozilla.org/p/webext-permissions etherpad]
|-
|-
| Test case execution || [https://testrail.stage.mozaws.net/index.php TestRail]
| Test case execution || [https://testrail.stage.mozaws.net/index.php TestRail]
|-
|-
| Bugs management || Bugzilla
| Bugs management || Bugzilla / Github
|}
|}


Line 148: Line 197:
  Identify existing bugs on the feature with high risk
  Identify existing bugs on the feature with high risk
  Identify if other areas are affected by the fix
  Identify if other areas are affected by the fix
= References =
* List and links for specs
  PRD - https://docs.google.com/document/d/1jZQUeIfelisRXBMaTuPTo2hXiIcPT3IpdYwQXDpzEMI/edit#heading=h.jyc2dow656pq
  Install flow - https://www.figma.com/file/HrLiKUwoLQZsIUIVBKM8Wnnu/Install-Flow-showing-Permissions
  Permissions Copy - https://docs.google.com/document/d/1-aLncxcKpinCNpvksmknwWZhrM2bEChDmQ3C422C6Vc/edit#heading=h.cwg9zx7py4zk
   
   
= References =
* {{Bug|1308292}} - (webext-permissions) (tracking) Webextensions required permissions handling
* PRD - https://docs.google.com/document/d/1jZQUeIfelisRXBMaTuPTo2hXiIcPT3IpdYwQXDpzEMI/edit#heading=h.jyc2dow656pq
<bugzilla>
* Install flow - https://www.figma.com/file/HrLiKUwoLQZsIUIVBKM8Wnnu/Install-Flow-showing-Permissions
    {
* Permissions Copy - https://docs.google.com/document/d/1-aLncxcKpinCNpvksmknwWZhrM2bEChDmQ3C422C6Vc/edit#heading=h.cwg9zx7py4zk
        "blocks":[1308292],
* Tracker bug - [https://bugzilla.mozilla.org/show_bug.cgi?id=1308292 bug 1308292]
        "include_fields": "id, priority, component, assigned_to, summary, status, target_milestone"
    }
</bugzilla>
 
* {{Bug|1197420}} - Implement permissions API and optional_permissions manifest property
<bugzilla>
    {
        "blocks":[1197420],
        "include_fields": "id, priority, component, assigned_to, summary, status, target_milestone"
    }
</bugzilla>


= Testcases =  
= Testcases =  
Line 163: Line 228:
! Test Areas !! Covered !! Details
! Test Areas !! Covered !! Details
|-
|-
| Private Window
| '''Installation from AMO'''
|style="text-align:center;" | ||  
|style="text-align:center;" | ||  
|-
|-
| Multi-Process Enabled
| '''Installation from Discopane'''
|style="text-align:center;" |  ||
|-
| Multi-process Disabled
|style="text-align:center;" |  ||
|-
| Theme (high contrast)
|style="text-align:center;" |  ||
|-
| '''UI'''  
||  ||
|-
| Mouse-only operation 
|style="text-align:center;" |  ||
|-
| Keyboard-only operation 
|style="text-align:center;" |  ||
|-
| Display (HiDPI)
|style="text-align:center;" | ||
|-
| Interraction (scroll, zoom)
|style="text-align:center;" | ||
|-
| Usable with a screen reader 
|style="text-align:center;" |  || e.g. with NVDA
|-
| Usability and/or discoverability testing 
|style="text-align:center;" |  || Is this feature user friendly
|-
| RTL build testing 
|style="text-align:center;" |  ||
|-
| '''Help/Support'''
||  ||
|-
| Help/support interface required 
|style="text-align:center;" |  || Make sure link to support/help page exist and is easy reachable.
|-
| Support documents planned(written) 
|style="text-align:center;" |  || Make sure support documents are written and are correct.
 
|-
| '''Install/Upgrade'''
||  ||
|-
| Feature upgrades/downgrades data as expected 
|style="text-align:center;" |  ||
|-
| Does sync work across upgrades 
|style="text-align:center;" |  ||  
|style="text-align:center;" |  ||  
|-
|-
| Requires install testing   
| '''Installation from Add-ons Manager'''
|style="text-align:center;" |  || separate feature/application installation needed (not only Firefox)
|-
| Affects first-run or onboarding   
|style="text-align:center;" |  || Florin/Lawrence are investigating if there is a dedicated QA for this, or we should test? Should be an yes/no and if is yes should add in detail column the team/person assigned.
|-
| Does this affect partner builds? Partner build testing 
|style="text-align:center;" |  || yes/no options, add comment with details about who will lead testing
 
|-
| ''' Enterprise '''  
||  ||  Raise up the topic to developers to see if they are expecting to work different on ESR builds
|-
| Enterprise administration 
|style="text-align:center;" |  ||  
|style="text-align:center;" |  ||  
|-
|-
| Network proxies/autoconfig 
| '''Installation from Third-party'''
|style="text-align:center;" |  ||  
|style="text-align:center;" |  ||  
|-
|-
| ESR behavior changes 
| '''Add-on updates'''  
|style="text-align:center;" |  ||
|-
| Locked preferences 
|style="text-align:center;" |  ||
 
|-
| ''' Data Monitoring '''  
||  ||  
||  ||  
|-
|-
| Temporary or permanent telemetry monitoring 
| '''Sideloading'''  
|style="text-align:center;" | || List of error conditions to monitor
|-
| Telemetry correctness testing 
|style="text-align:center;" |  ||  
|style="text-align:center;" |  ||  
|-
|-
| Server integration testing 
| '''Other'''  
|style="text-align:center;" | ||
|-
| Offline and server failure testing 
|style="text-align:center;" |  ||
|-
| Load testing 
|style="text-align:center;" |  ||
|style="text-align:center;" |  ||
|-
| ''' Add-ons '''
||  || If add-ons are available for testing feature, or is current feature will affect some add-ons, then API testing should be done for the add-on.
|-
| Addon API required? 
|style="text-align:center;" |  ||
|-
| Comprehensive API testing 
|style="text-align:center;" |  ||
|-
| Permissions 
|style="text-align:center;" |  ||
|-
| Testing with existing/popular addons
|style="text-align:center;" |  ||
|-
| ''' Security '''
||  || Security is in charge of Matt Wobensmith. We should contact his team to see if security testing is necessary for current feature.
|-
| 3rd-party security review 
|style="text-align:center;" |  ||
|-
| Privilege escalation testing
|style="text-align:center;" |  ||
|-
| Fuzzing 
|style="text-align:center;" |  ||
|-
|-
| ''' Web Compatibility '''
||  || depends on the feature
|-
| Testing against target sites 
|style="text-align:center;" |  ||
|-
| Survey of many sites for compatibility 
|style="text-align:center;" |  ||
|-
| ''' Interoperability  '''
||  || depends on the feature
|-
| Common protocol/data format with other software: specification available. Interop testing with other common clients or servers. 
|style="text-align:center;" |  ||
|-
| Coordinated testing/interop across the Firefoxes: Desktop, Android, iOS 
|style="text-align:center;" |  ||
|-
| Interaction of this feature with other browser features 
|style="text-align:center;" |  ||
|}
|}


== Test suite ==
== Test suite ==
Full Test suite - Link with the gdoc, follow the format from [https://docs.google.com/spreadsheets/d/1N8dGK_uenhfcM9cEPC3HPHmi0jFN98FWtL2DuszrfLY/edit#gid=0 link]
* Link for the [https://public.etherpad-mozilla.org/p/webext-permissions Initial test planning]
Smoke Test suite - Link with the gdoc, follow the format from [https://docs.google.com/spreadsheets/d/1N8dGK_uenhfcM9cEPC3HPHmi0jFN98FWtL2DuszrfLY/edit#gid=0 link]
* Link for the [https://docs.google.com/spreadsheets/d/1b_W1iVVuo-MRtwilKeHt_Vjbemy3j1Pbiq4slL_BhE8/edit#gid=1522764364 Google doc tests]
Regression Test suite - Link with the gdoc - if available/needed.
* Link for the [https://testrail.stage.mozaws.net/index.php?/suites/view/747&group_by=cases:section_id&group_id=2394&group_order=asc TestRail tests]


= Bug Work =
= Bug Work =
Tracking bug – meta bug
Tracking bug - [https://bugzilla.mozilla.org/show_bug.cgi?id=1308292 bug 1308292]


<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
Line 328: Line 263:
====== Bug fix verification ======
====== Bug fix verification ======
<div class="mw-collapsible-content">
<div class="mw-collapsible-content">
{| class="wikitable" style="width:50%"
<font color="green">'''[Verified]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1308309 Bug 1308309] - Display permissions prompt for webextensions installed using mozAddonManager
|-
:↳ <small>'''2017-01-10'''</small>: verified fixed on 53.0a1 across platforms
! Bug No !! Summary !! Status !! Firefox Verion
<font color="green">'''[Verified]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1308295 Bug 1308295] - Prompt users with permissions for third-party webextensions installs
|-
:↳ <small>'''2015-04-21'''</small>: verified fixed on 53.0a1 across platforms
|  123 || bug summary || NEW || Nighly 45
|-
| || || ||
|}
</div></div>
</div></div>


Line 342: Line 273:
====== Logged bugs ======
====== Logged bugs ======
<div class="mw-collapsible-content">
<div class="mw-collapsible-content">
Bug 111111<br/>
[https://bugzilla.mozilla.org/show_bug.cgi?id=1329942 Bug 1329942] - Misaligned icon and webextension name in permissions doorhanger<br/>  
Bug 211111<br/>
 
</div>
</div>
</div>
</div>


= Sign off =
= Sign off =
Line 360: Line 290:
List of OSes that will be covered by testing<br />
List of OSes that will be covered by testing<br />
*Link for the tests run
*Link for the tests run
** Daily Smoke, use template from [https://docs.google.com/spreadsheets/d/1ch0OubyOWn1W-_JLF-38nvSvVB76Wmo1hkLccEtU5hk/edit?usp=sharing link]
** Full Test suite, use template from [https://testrail.stage.mozaws.net/index.php?/suites/view/747&group_by=cases:section_id&group_id=2394&group_order=asc link]
** Full Test suite, use template from [https://docs.google.com/spreadsheets/d/1ch0OubyOWn1W-_JLF-38nvSvVB76Wmo1hkLccEtU5hk/edit?usp=sharing link]
 
** Regression Test suite, if needed/available


'''Merge to Aurora Sign-off'''
'''Merge to Aurora Sign-off'''

Latest revision as of 07:08, 2 October 2017

Revision History

Date Version Author Description
01/09/2017 1.0 Krupa Raj Created first draft
01/20/2017 1.1 Vasilica Mihasca Updates
01/23/2017 1.2 Krupa Raj Added Acceptance Criteria
03/22/2017 1.3 Vasilica Mihasca Added TestRail links
08/23/2017 1.4 Cosmin Badescu Updates: References, Overview, Purpose,Requirements for testing, Ownership


Overview

Permissions is a feature which allows webextensions to have certain special powers. The permissions key specified in the manifest.json will grant special powers for your extension. This key is an array of strings, and each string is a request for a permission.

If you request permissions using this key, then the browser may inform the user at install time that the extension is requesting certain privileges, and ask them to confirm that they are happy to grant these privileges. The browser may also allow the user to inspect an extension's privileges after installation.

  • As part of this feature, the optional permissions are used when they are needed for optional features in your extension.

Purpose

This document purports to detail the test approach to Webextension Permissions and Optional permissions including Entry/Exit criteria, Scope for testing, links to testcases etc

Entry Criteria

  • QA has access to all the PRDs, mocks and related documents
  • The feature has landed on Nightly
  • AMO parts has landed on dev

Exit Criteria

  • All the bugs against the feature have been triaged
  • All the P1/P2 bugs have been fixed
  • All the resolved bugs have been verified by QA
  • The find/fixed rate is going down over a predefined period of time

Acceptance Criteria

This section broadly outlines when the product is ready to ship

  • QA has signed off
  • All the required Telemetry is in place
  • All info is localized at least for a pre-defined set of locales
  • All the necessary PR/blogposts have been sent out

Scope

This section describes what parts of the feature will be tested and what parts won't be.

what's in scope?

  • Installation flow from AMO
  • Installation flow from Add-ons Manager
  • Installation flow from Discopane
  • Installation flow from Third-party websites
  • Add-on updates
  • Add-on Validation related to permissions during submission
  • Sideloading(?)
  • Webextensions have access to the permissions requested

what's out of scope?

  • Security testing
  • Performance testing

Ownership

Dev Lead: Andrew Swan; irc nick :aswan
QA Manager: Krupa Raj; irc nick :krupa
QA Lead: Victor Carciu; irc nick :victorc
Webextensions QA: Vasilica Mihasca; irc nick :vasilica_mihasca
Webextensions QA: Cosmin Badescu; irc nick :CosminB
Add-ons QA: Valentina Virlics; irc nick :ValentinaV

Requirements for testing

Environments

OSes covered: Windows, MacOSX, Linux

Channel dependent settings (configs) and environment setups

Nightly

  • Enable extensions.webextPermissionPrompts from about:config
  • Enable extensions.webextOptionalPermissionPrompts from about:config
  • Disable check for signing if testing on AMO dev or stage

Aurora

  • Enable extensions.webextPermissionPrompts from about:config
  • Enable extensions.webextOptionalPermissionPrompts from about:config
  • Disable check for signing if testing on AMO dev or stage

Beta

  • Enable extensions.webextPermissionPrompts from about:config
  • Enable extensions.webextOptionalPermissionPrompts from about:config
  • [Unbranded build] Disable check for signing if testing on AMO dev or stage

Post Beta / Release

The feature is enabled by default.

Test Strategy

Test Objectives

This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master. This could be documented in bullet form or in a table similar to the one below.

Ref Function Test Objective Test Type Owners
TO-1 Validation during AMO To verify that linter raise the proper error messages Manual Add-ons QA Team
TO-2 AMO public pages To verify that permissions are correctly displayed Manual Add-ons QA Team
TO-3 Installing from AMO To verify that the permissions flow is fully respected Manual Add-ons QA Team
TO-4 Installing from Discopane To verify that the permissions flow is fully respected Manual Add-ons QA Team
TO-5 Installing from Add-ons manager To verify that the permissions flow is fully respected Manual Add-ons QA Team
TO-6 Installing via sideloading To verify that the permissions flow is fully respected Manual Add-ons QA Team
TO-7 Installing from local files To verify that the permissions flow is fully respected Manual Add-ons QA Team
TO-8 Installing from thirdparty To verify that the permissions flow is fully respected Manual Add-ons QA Team
TO-9 Add-on updates To verify that the permissions flow is fully respected Manual Add-ons QA Team

Builds

This section should contain links for builds with the feature -

  • Link for Nightly builds
  • Link for Aurora builds
  • Link for Beta builds

Test Execution Schedule

The following table identifies the anticipated testing period available for test execution.

Project phase Start Date End Date
Start project
Study documentation/specs received from developers
QA - Test plan creation 01-09-2017 01-11-2017
QA - Test cases/Env preparation 01-09-2017
QA - Nightly Testing 01-11-2017
QA - Aurora Testing
QA - Beta Testing
Release Date

Testing Tools

Detail the tools to be used for testing, for example see the following table:

Process Tool
Test plan creation Mozilla wiki
Test case creation TestRail / Google docs / etherpad
Test case execution TestRail
Bugs management Bugzilla / Github

Status

Overview

Track the dates and build number where feature was released to Nightly
Track the dates and build number where feature was merged to Aurora
Track the dates and build number where feature was merged to Release/Beta

Risk analysis

Identify the high-risk assumptions
Identify existing bugs on the feature with high risk
Identify if other areas are affected by the fix

References

* List and links for specs
  PRD - https://docs.google.com/document/d/1jZQUeIfelisRXBMaTuPTo2hXiIcPT3IpdYwQXDpzEMI/edit#heading=h.jyc2dow656pq
  Install flow - https://www.figma.com/file/HrLiKUwoLQZsIUIVBKM8Wnnu/Install-Flow-showing-Permissions
  Permissions Copy - https://docs.google.com/document/d/1-aLncxcKpinCNpvksmknwWZhrM2bEChDmQ3C422C6Vc/edit#heading=h.cwg9zx7py4zk

* bug 1308292 - (webext-permissions) (tracking) Webextensions required permissions handling
Full Query
ID Priority Component Assigned to Summary Status Target milestone
1308295 P2 General Andrew Swan [:aswan] Prompt users with permissions for third-party webextensions installs VERIFIED mozilla53
1308296 P5 General Andrew Swan [:aswan] Display "your add-on is ready" popup after installing a webextension VERIFIED ---
1308309 P2 General Andrew Swan [:aswan] Display permissions prompt for webextensions installed using mozAddonManager VERIFIED mozilla53
1308310 P3 General Andrew Swan [:aswan] Display "your add-on is ready" popup after installing with mozAddonManager VERIFIED mozilla54
1311815 P1 Request Handling Tomislav Jovanovic :zombie webRequest doesn't check host permissions RESOLVED mozilla54
1316996 P1 General Andrew Swan [:aswan] Create initial text for webextensions permissions RESOLVED mozilla53
1317000 P1 Add-ons Manager Andrew Swan [:aswan] Display permissions prompt for webextensions installed locally from about:addons VERIFIED mozilla54
1317363 P1 Add-ons Manager Andrew Swan [:aswan] Implement the new sideloading flow VERIFIED mozilla53
1317470 P1 Add-ons Manager Andrew Swan [:aswan] Display permission prompt for webextensions background updates VERIFIED mozilla53
1317590 P1 Add-ons Manager Andrew Swan [:aswan] AddonManager support for webextensions permissions RESOLVED mozilla53
1329942 P3 Frontend Mark Striemer [:mstriemer] Misaligned icon and webextension name in permissions doorhanger VERIFIED mozilla55
1330823 P2 Add-ons Manager Andrew Swan [:aswan] Display permissions prompt for interactive webextensions updates VERIFIED mozilla54
1332061 P1 Add-ons Manager Andrew Swan [:aswan] Add unsigned warning to webextension permission dialog VERIFIED mozilla55
1333168 P1 Add-ons Manager Andrew Swan [:aswan] Bold text in permission notifications is breaking lines unnecessarily VERIFIED mozilla54
1333262 -- General Andrew Swan [:aswan] On re-installing an add-on get the update permission prompt RESOLVED mozilla54
1333620 P1 Add-ons Manager Andrew Swan [:aswan] Doorhanger message to restart the browser is missing when Permissions is enabled VERIFIED mozilla54
1333790 -- General Long webextension name causes pop-up layout issue RESOLVED ---
1334010 P1 Add-ons Manager Andrew Swan [:aswan] sideload/update visual fixes VERIFIED mozilla54
1334076 -- Add-ons Manager Andrew Swan [:aswan] No puzzle icon displayed in installation pop-up if there is no icon specified in manifest.json VERIFIED mozilla54
1334085 P1 Add-ons Manager Andrew Swan [:aswan] Sideloading button text is not left-aligned VERIFIED mozilla54
1334096 P1 Add-ons Manager Andrew Swan [:aswan] Sideloading button should be removed if it not used to enable to extension VERIFIED mozilla54
1334354 P1 Add-ons Manager Andrew Swan [:aswan] Post-install notifications are dismissed immediately if AMO redirects to a contribute page VERIFIED mozilla54
1334404 P1 Add-ons Manager Andrew Swan [:aswan] Replace [name of add-on] with [your extension] / [your add-on] in permission confirmation pop-up VERIFIED mozilla54
1334479 P1 Add-ons Manager Andrew Swan [:aswan] Legacy add-ons which display a confirmation tab have no icon in confirmation doorhanger VERIFIED mozilla54
1335333 -- Add-ons Manager Andrew Swan [:aswan] Update pop-ups have no puzzle icon displayed if there is no icon specified in manifest.json VERIFIED mozilla54
1335697 P1 Add-ons Manager Andrew Swan [:aswan] No pop-ups displayed while installing from Addons Manager search list VERIFIED mozilla54
1335703 -- Add-ons Manager Re-enable permission doorhanger not displayed RESOLVED ---
1335720 P2 Add-ons Manager Andrew Swan [:aswan] Replace "Disable" with "Cancel" button in sideloaded-install pop-up VERIFIED mozilla54
1335985 P1 Add-ons Manager Florian Quèze [:florian] Add delay before an action can be taken on a persistent notification when the window is reactivated VERIFIED mozilla54
1336085 P2 Add-ons Manager Andrew Swan [:aswan] Don't display pending update if there are no new promptable permissions VERIFIED mozilla54
1337870 -- Add-ons Manager Andrew Swan [:aswan] Stop truncating add-on names in sideload/update menu items VERIFIED mozilla54
1337951 P3 Add-ons Manager Mark Striemer [:mstriemer] Add separator above menu items for sideloaded/updated addons VERIFIED mozilla54
1338713 P1 Add-ons Manager Andrew Swan [:aswan] Basic telemetry for permissions notifications VERIFIED mozilla55
1339552 P1 Add-ons Manager Andrew Swan [:aswan] Do not prompt with permissions when upgrading from a legacy add-on to a WebExtension version VERIFIED mozilla54
1339952 P3 General Andrew Swan [:aswan] List permissions in descending order of priority in the install prompts VERIFIED mozilla54
1340078 -- Add-ons Manager Andrew Swan [:aswan] Old permissions are reposted in interactive webextension update pop-up VERIFIED mozilla54
1340135 -- Add-ons Manager Andrew Swan [:aswan] Permission pop-up does not display the correct word for the update button when background update is prompted. VERIFIED mozilla54
1340220 -- Graphic Commandline and Toolbar Can enable a sideloaded add-on skipping permissions using gcli RESOLVED ---
1340443 P1 Add-ons Manager Andrew Swan [:aswan] “nativeMessaging” prevents the installation of a WebExtension VERIFIED mozilla54
1340501 P3 Add-ons Manager Andrew Swan [:aswan] Incorrect text in sideloading-install pop-up for several types of add-ons VERIFIED mozilla54
1341240 -- Add-ons Manager “history” and “topSites” permissions display the same text in installation pop-up RESOLVED ---
1341273 -- Add-ons Manager Add permission string for "unlimitedStorage" permission RESOLVED ---
1341286 -- Add-ons Manager Permissions pop-up does not disappear after navigating to a new url RESOLVED ---
1342133 P3 Untriaged Support non-permission properties in manifest as permissions RESOLVED ---
1342142 -- Add-ons Manager Andrew Swan [:aswan] enable webextension permissions prompts by default VERIFIED mozilla55
1342350 P5 Add-ons Manager Canceling the permissions pop-up removes the webextension from the search results list RESOLVED ---
1342426 -- Add-ons Manager No icon displayed for a webextension if the manifest references a nonexistent image RESOLVED ---
1342896 P1 Add-ons Manager Andrew Swan [:aswan] Pending update displayed for non-promptable permissions while upgrading via interactive method VERIFIED mozilla54
1342914 P3 Add-ons Manager Add-ons Manager ”Enable” button should not be clickable as long as permissions pop-up is displayed RESOLVED ---
1343498 P2 Add-ons Manager Andrew Swan [:aswan] WebExtensions with invalid match pattern are not installed VERIFIED mozilla55
1343571 -- Notifications and Alerts Florian Quèze [:florian] The delay is not applied and the action is triggered from the first click on a notification when the window is refocus VERIFIED mozilla55
1344214 P3 Add-ons Manager Andrew Swan [:aswan] Status "waiting for download" does not contain the correct button in the pop-up VERIFIED mozilla55
1346138 P5 Add-ons Manager Icon is not displayed in the sideloading pop-up after upgrading from a disabled legacy add-on to a WebExtension version RESOLVED ---
1347063 -- Site Identity The pop-up persists on the desktop screen after the web page is out of focus (minimized) RESOLVED ---
1347478 P5 Add-ons Manager After the update for a WebExtension that is installed via sideloading method a user can enable the first version while the last version will be disabled in about:addons RESOLVED ---
1348854 -- Add-ons Manager Andrew Swan [:aswan] Webextension pop-up style displayed while installing legacy add-ons from about:addons search list VERIFIED mozilla55
1349189 P5 Add-ons Manager The icon from the confirmation pop-up is not displayed after the WebExtension is removed from about:addons. RESOLVED ---
1350277 P3 Add-ons Manager Andy McKay http://*/* and https://*/* permissions display identical descriptions VERIFIED mozilla57
1358431 P2 Frontend Mark Striemer [:mstriemer] Add-on name bolded with delay in permissions pop-up VERIFIED mozilla56

59 Total; 0 Open (0%); 18 Resolved (30.51%); 41 Verified (69.49%);


  • bug 1197420 - Implement permissions API and optional_permissions manifest property
Full Query
ID Priority Component Assigned to Summary Status Target milestone
1201896 P1 Site Identity :Paolo Amadini Update popup notifications for add-on installation VERIFIED Firefox 53
1229230 P3 Untriaged Define how Add-on Permissions help users (and how they may be surfaced to them) RESOLVED ---
1333235 -- Frontend Markus Jaritz [:markus] (UX) ux/copy/visual design for optional permissions dialog RESOLVED ---
1333477 -- General Kris Maglione [:kmag] Support injecting and invalidating schema wrappers dynamically RESOLVED mozilla55
1350759 -- Frontend Andrew Swan [:aswan] Warn but accept illegal values in optional_permissions RESOLVED mozilla55
1375485 P5 General Ian Moody [:Kwan] (UTC+0) urls or <all_urls> can be requested as "permissions" instead of "origins" VERIFIED mozilla58
1382031 -- Frontend Andrew Swan [:aswan] Optional permissions dialog keyboard shortcut RESOLVED mozilla56
1444294 P1 General Shane Caraveo (:mixedpuppy) Implement permissions API events RESOLVED mozilla77

8 Total; 0 Open (0%); 6 Resolved (75%); 2 Verified (25%);


Testcases

Overview

Summary of testing scenarios

Test Areas

Test Areas Covered Details
Installation from AMO
Installation from Discopane
Installation from Add-ons Manager
Installation from Third-party
Add-on updates
Sideloading
Other

Test suite

Bug Work

Tracking bug - bug 1308292

Bug fix verification

[Verified] Bug 1308309 - Display permissions prompt for webextensions installed using mozAddonManager

↳ 2017-01-10: verified fixed on 53.0a1 across platforms

[Verified] Bug 1308295 - Prompt users with permissions for third-party webextensions installs

↳ 2015-04-21: verified fixed on 53.0a1 across platforms
Logged bugs

Bug 1329942 - Misaligned icon and webextension name in permissions doorhanger

Sign off

Criteria

Check list

  • All test cases should be executed
  • Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan
  • All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)

Results

Nightly testing

List of OSes that will be covered by testing

  • Link for the tests run
    • Full Test suite, use template from link


Merge to Aurora Sign-off List of OSes that will be covered by testing

  • Link for the tests run
    • Full Test suite

Checklist

Exit Criteria Status Notes/Details
Testing Prerequisites (specs, use cases)
Testing Infrastructure setup
Test Plan Creation
Test Cases Creation
Full Functional Tests Execution
Automation Coverage
Performance Testing
All Defects Logged
Critical/Blockers Fixed and Verified
Metrics/Telemetry
QA Signoff - Nightly Release Email to be sent
QA Aurora - Full Testing
QA Signoff - Aurora Release Email to be sent
QA Beta - Full Testing
QA Signoff - Beta Release Email to be sent
Retrieved from "https://wiki.mozilla.org/index.php?title=Add-ons/Permissions/Testplan&oldid=1181530"