Add-ons/QA/Testplan/CSP
From MozillaWiki
< Add-ons
Revision History
Date | Version | Author | Description |
---|---|---|---|
11/01/2017 | 1.0 | Marius Santa | Created first draft |
Contents
Overview
- All content injected into web content pages is currently subject to the same Content Security Policy, regardless of who injected it. For privileged callers, such as extension content scripts, this means that some functionality can behave erratically, depending on the page they're running on.
- The plan here is to apply a separate CSP to content injected by certain privileged callers, rather than subjecting it to page CSP. Content from system URLs (like moz-extension:) is already immune to CSP. This change will extend that immunity to any content injected by those callers.
Purpose
- This document's purpose is to detail the test approach to the CSP for content scripts, including Entry/Exit criteria, Scope for testing, links to testcases etc
Entry Criteria
- QA has access to all the PRDs, mocks and related documents
- The feature has landed on Nightly
- AMO parts has landed on dev
Exit Criteria
- All the bugs against the feature have been triaged
- All the P1/P2 bugs have been fixed
- All the resolved bugs have been verified by QA
- The find/fixed rate is going down over a predefined period of time
Scope
This section describes what parts of the feature will be tested and what parts won't be.
what's in scope?
- Apply a separate CSP to content injected by certain privileged callers
- Extend immunity to any content injected by those privileged callers
what's out of scope?
- Performance testing
Ownership
Product Manager: Jorge Villalobos; irc nick :jorgev
QA Manager: Krupa Raj; irc nick :krupa
QA Lead: Victor Carciu; irc nick :victorc
Add-ons QA: Valentina Virlics; irc nick :ValentinaV
Webextensions QA: Marius Santa; irc nick :Santa
Requirements for testing
Environments
- Windows
- Mac OS
- Linux
Servers
- Stage: https://addons.allizom.org/en-US/
- Dev: https://addons-dev.allizom.org/en-US/
- Production: https://addons.mozilla.org/en-US/
Channel dependent settings (configs) and environment setups
- Nightly
- Beta
- Release
Test Strategy
Builds
This section should contain links for builds with the feature -
- Nightly builds
- Beta builds
- Beta Unbranded builds:
- Release builds
- Release Unbranded builds:
Test Execution Schedule
The following table identifies the anticipated testing period available for test execution.
Project phase | Start Date | End Date |
---|---|---|
Start project | ||
Study PRD/mocks received | ||
QA - Test plan creation | 11-01-2017 | |
QA - Test cases preparation | ||
QA - Test cases execution | ||
Release Date |
Testing Tools
Process | Tool |
---|---|
Test plan creation | Mozilla wiki |
Test case creation | TestRail / Google docs / etherpad |
Test case execution | TestRail |
Bugs management | Github |
References
* List and links for specs
* bug 1267027 - (webext-permissions) (tracking) Webextensions required permissions handling
10 Total; 5 Open (50%); 5 Resolved (50%); 0 Verified (0%);
Testcases
Test Areas
- Submission/Approvals/Installation of extension that inject stuff into a page with CSP
Test Areas
Test Areas | Covered | Details |
---|---|---|
Pages with restrictive CSP | ||
Content script injection |
Bug Work
Tracking bug - bug 1267027
Bug fix verification
Logged bugs
Sign off
Criteria
Check list
- All test cases should be executed
- All blockers must be fixed and verified or have an agreed-upon timeline for being fixed
Checklist
Exit Criteria | Status | Notes/Details |
---|---|---|
Testing Prerequisites (specs, use cases) | ||
Testing Infrastructure setup | ||
Test Plan Creation | 11-01-2017 | |
Test Cases Creation | ||
Full Functional Tests Execution | ||
Automation Coverage | ||
Performance Testing | ||
All Defects Logged | ||
Critical/Blockers Fixed and Verified | ||
Metrics/Telemetry | ||
QA Signoff - Nightly Release | ||
QA Beta - Full Testing | ||
QA Signoff - Beta Release |