Add-ons/QA/Testplan/declarativeContent API
Revision History
Date | Version | Author | Description |
---|---|---|---|
10/23/2018 | 1.0 | Vlad Jiman | Created first draft |
10/26/2018 | 1.0 | Vlad Jiman | Updated sections, added more details |
10/09/2019 | 1.1 | Alex Cornestean | Updated sections |
Please note, this document is still under development and parts have not been modified from the template cloning, other sections are placeholders
Contents
Overview
Purpose
This document proposes to detail the test approach to the "declarativeContent API" feature, including Entry/Exit/Acceptance criteria, Scope for testing, links to test cases etc.
Entry Criteria
- QA has access to all the PRDs, mocks and related documents
- The feature has landed on Nightly
Exit Criteria
- All the bugs against the feature have been triaged
- All the P1/P2 bugs have been fixed
- All the resolved bugs have been verified by QA
- The find/fixed bug ratio shows a descending trend over a defined time period
Acceptance Criteria
This section broadly outlines when the product is ready to ship
- QA has signed off
Scope
This section describes what parts of the feature will be tested and what parts won't be.
What's in scope?
- Validation of the entire "declarativeContent API" feature, including all related UI features.
What's out of scope?
- Performance testing
- Security testing
Ownership
Dev Lead: Rob Wu ; irc nick: robwu_nl
QA Manager: Krupa Raj; irc nick: krupa
QA Lead: Victor Carciu; irc nick: victorc
Webextensions QA: Alex Cornestean;
Requirements for testing
Environments
OSes covered: Windows, Mac OS X, Linux, Mobile (TBD)
Channel dependent settings (configs) and environment setups
Nightly
none
Beta
none
Release
none
Test Strategy
Test Objectives
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master. This could be documented in bullet form or in a table similar to the one below.
Ref | Function | Test Objective | Test Type | Owners |
---|---|---|---|---|
TO-1 | Match using URL | To verify that the extension will correctly display its icon when matching by URL | Manual | Add-ons QA Team |
TO-2 | Match using CSS | To verify that the extension will correctly display its icon when matching by CSS | Manual | Add-ons QA Team |
TO-3 | Match using CSS + URL | To verify that the extension will correctly display its icon when matching by CSS + URL | Manual | Add-ons QA Team |
TO-4 | Browser restart/crash scenarios | To verify that the extension will correctly work in previous scenarios ever after restart/crash | Manual | Add-ons QA Team |
TO-5 | Sideloading, Debug | To verify that the extension will correctly work after being loaded by sideload or as temporary addon | Manual | Add-ons QA Team |
Builds
This section should contain links for builds with the feature -
Test Execution Schedule
The following table identifies the anticipated testing period available for test execution.
Project phase | Start Date | End Date |
---|---|---|
Start project | ||
Study documentation/specs received from developers | 10/23/2018 | |
QA - Test plan creation | 10/23/2018 | |
QA - Test cases/Env preparation | 10/25/2018 | |
QA - Nightly Testing | ||
QA - Beta Testing | ||
Release Date |
Testing Tools
Detail the tools to be used for testing, for example see the following table:
Process | Tool |
---|---|
Test plan creation | Mozilla wiki |
Test case creation | TestRail |
Test case execution | TestRail |
Bugs management | Bugzilla |
Status
Overview
Track the dates and build number where feature was released to Nightly Track the dates and build number where feature was merged to Release/Beta
Risk analysis
Identify the high-risk assumptions Identify existing bugs on the feature with high risk Identify if other areas are affected by the fix
References
* List and links for specs PRD - chromeDev * bug 1435864 - [META] Implement declarativeContent API
No results.
0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);
Testcases
Overview
The test suite proposes a series of 3 test cases devised to cover several scenarios in order to properly validate the feature.
- Validating that an addon will properly show its icon when loading a CSS match page, but hide its icon when loading a URL or CSS+URL match page.
- Validating that an addon will properly show its icon when loading a CSS+URL match page, but hide its icon when loading a URL or CSS match page.
- Validating that an addon will properly show its icon when loading a URL match page, but hide its icon when loading a CSS or CSS+URL match page.
Test Areas
Test Areas | Covered | Details |
---|---|---|
Addon A (matching by CSS) will show its icon when matching a CSS page, but hide the icon when loading a URL or CSS+URL match page | ||
Addon B (matching by CSS+URL) will show its icon when matching a CSS+URL page, but hide the icon when loading a CSS or URL match page | ||
Addon C (matching by URL) will show its icon when matching a URL page, but hide the icon when loading a CSS or CSS+URL match page |
Test suite
- [ TestRail]
Sign off
Criteria
Check list
- All test cases should be executed
- Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan
- All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)
Checklist
Exit Criteria | Status | Notes/Details |
---|---|---|
Test Plan Creation | 10/23/2018 | Test Plan submitted for review |
Test Cases Creation | 05/15/2019 | Test Suite submitted for review |
Critical/Blockers Fixed and Verified | - | Feature not landed |
QA Signoff - Nightly Release | - | Feature does not require Nightly testing |
QA Beta - Full Testing | - | Feature not landed |
QA Signoff - Beta Release | - | Feature not landed |