|10/23/2018||1.0||Vlad Jiman||Created first draft|
|10/26/2018||1.0||Vlad Jiman||Updated sections, added more details|
|10/09/2019||1.1||Alex Cornestean||Updated sections|
Please note, this document is still under development and parts have not been modified from the template cloning, other sections are placeholders
- 1 Overview
- 2 Requirements for testing
- 3 Test Strategy
- 4 Status
- 5 References
- 6 Testcases
- 7 Sign off
This document proposes to detail the test approach to the "declarativeContent API" feature, including Entry/Exit/Acceptance criteria, Scope for testing, links to test cases etc.
- QA has access to all the PRDs, mocks and related documents
- The feature has landed on Nightly
- All the bugs against the feature have been triaged
- All the P1/P2 bugs have been fixed
- All the resolved bugs have been verified by QA
- The find/fixed bug ratio shows a descending trend over a defined time period
This section broadly outlines when the product is ready to ship
- QA has signed off
This section describes what parts of the feature will be tested and what parts won't be.
What's in scope?
- Validation of the entire "declarativeContent API" feature, including all related UI features.
What's out of scope?
- Performance testing
- Security testing
Requirements for testing
OSes covered: Windows, Mac OS X, Linux, Mobile (TBD)
Channel dependent settings (configs) and environment setups
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master. This could be documented in bullet form or in a table similar to the one below.
|Ref||Function||Test Objective||Test Type||Owners|
|TO-1||Match using URL||To verify that the extension will correctly display its icon when matching by URL||Manual||Add-ons QA Team|
|TO-2||Match using CSS||To verify that the extension will correctly display its icon when matching by CSS||Manual||Add-ons QA Team|
|TO-3||Match using CSS + URL||To verify that the extension will correctly display its icon when matching by CSS + URL||Manual||Add-ons QA Team|
|TO-4||Browser restart/crash scenarios||To verify that the extension will correctly work in previous scenarios ever after restart/crash||Manual||Add-ons QA Team|
|TO-5||Sideloading, Debug||To verify that the extension will correctly work after being loaded by sideload or as temporary addon||Manual||Add-ons QA Team|
This section should contain links for builds with the feature -
Test Execution Schedule
The following table identifies the anticipated testing period available for test execution.
|Project phase||Start Date||End Date|
|Study documentation/specs received from developers||10/23/2018|
|QA - Test plan creation||10/23/2018|
|QA - Test cases/Env preparation||10/25/2018|
|QA - Nightly Testing|
|QA - Beta Testing|
Detail the tools to be used for testing, for example see the following table:
|Test plan creation||Mozilla wiki|
|Test case creation||TestRail|
|Test case execution||TestRail|
Track the dates and build number where feature was released to Nightly Track the dates and build number where feature was merged to Release/Beta
Identify the high-risk assumptions Identify existing bugs on the feature with high risk Identify if other areas are affected by the fix
* List and links for specs PRD - chromeDev * bug 1435864 - [META] Implement declarativeContent API
|ID||Priority||Component||Assigned to||Summary||Status||Target milestone|
|1315829||P2||General||Rob Wu [:robwu]||Intermittent TEST-UNEXPECTED-TIMEOUT | toolkit/components/extensions/test/xpcshell/test_ext_runtime_onInstalled_and_onStartup.js | Test timed out||RESOLVED||mozilla66|
|1323433||P2||General||Implement a declarative content script API||RESOLVED||---|
|1432647||P5||General||Support themes with declarativeContent API||REOPENED||---|
|1465994||P3||General||Implement setIcon as declarativeContent API||NEW||---|
|1465995||P1||General||Rob Wu [:robwu]||Implement ShowPageAction for declarativeContent API||ASSIGNED||---|
|1540215||P3||General||Rob Wu [:robwu]||Implement declarativeContent.IgnoreRules action||NEW||---|
6 Total; 4 Open (66.67%); 2 Resolved (33.33%); 0 Verified (0%);
The test suite proposes a series of 3 test cases devised to cover several scenarios in order to properly validate the feature.
- Validating that an addon will properly show its icon when loading a CSS match page, but hide its icon when loading a URL or CSS+URL match page.
- Validating that an addon will properly show its icon when loading a CSS+URL match page, but hide its icon when loading a URL or CSS match page.
- Validating that an addon will properly show its icon when loading a URL match page, but hide its icon when loading a CSS or CSS+URL match page.
|Addon A (matching by CSS) will show its icon when matching a CSS page, but hide the icon when loading a URL or CSS+URL match page|
|Addon B (matching by CSS+URL) will show its icon when matching a CSS+URL page, but hide the icon when loading a CSS or URL match page|
|Addon C (matching by URL) will show its icon when matching a URL page, but hide the icon when loading a CSS or CSS+URL match page|
- [ TestRail]
- All test cases should be executed
- Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan
- All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)
|Test Plan Creation||10/23/2018||Test Plan submitted for review|
|Test Cases Creation||05/15/2019||Test Suite submitted for review|
|Critical/Blockers Fixed and Verified||-||Feature not landed|
|QA Signoff - Nightly Release||-||Feature does not require Nightly testing|
|QA Beta - Full Testing||-||Feature not landed|
|QA Signoff - Beta Release||-||Feature not landed|