Add-ons/QA/Testplan/Recommendations on about:addons (themes & extensions)
Revision History
| Date | Version | Author | Description |
|---|---|---|---|
| 02-04-2019 | 2.0 | Vlad Jiman | Updated all sections |
Please note: This document is still under works and sections might contain placeholders and not accurate data
Overview
The purpose of this feature is to add recommended extensions/themes on the appropriate tabs. As it is currently, suggestions are only shown on the 'Get Add-ons' tab. This feature is dependent on the modifications that will occur on the about:addons HTML page and any QA process will only begin once that is implemented.
Purpose
This document proposes to detail a test approach for the “Recommendations on about:addons” feature, which includes Entry/Exit/Acceptance criteria, Testing scope, references to testcases, etc.
Entry Criteria
- QA has access to all PRDs, mocks and related documentation
- The feature has landed on Nightly
Exit Criteria
- All feature related bugs have been triaged
- All P1/P2 bugs have been fixed
- All resolved bugs have been verified by QA
- The find/fixed bug ratio shows a descending trend over a defined time period
Acceptance Criteria
This section proposes to highlight the criteria concerning the shipment readiness status of the product.
- QA has signed off
- All the required Telemetry is in place
Scope
This section outlines which parts of the new implemented feature will or will not be tested.
What is in scope
- Validation of the entire ‘Recommendations on about:addons’ feature, including all related UI features
What is out of scope
- Security testing
- Device testing
- Performance testing
Ownership
Dev Lead: Mark Striemer ; irc nick:mstriemer
QA Manager: Krupa Raj; irc nick :krupa
QA Lead: Victor Carciu; irc nick :victorc
Webextensions QA: Vlad Jiman; irc nick :CosminB
Requirements for testing
Environments
Covered OSes: Windows, Mac OS X, Linux
Test Strategy
Test Objectives
This section details the progression test objectives that will be covered.
| Ref | Function | Test Objective | Test Type | Owners |
|---|---|---|---|---|
| TO1 | Private Browsing Testing | Validate if default recommendations are displayed | Manual | Add-ons QA Team |
| TO2 | Profile Testing | Validate if default/personalized recommendations are displayed for new profiles/profiles with installed add-ons/browsing history/bookmarks | Manual | Add-ons QA Team |
| TO3 | Feature On/Off Toggling | Validate if activating/deactivating the feature yields default/personalized recommendations | Manual | Add-ons QA Team |
Test Profiles
| Ref | Profile Objective |
|---|---|
| TP1 | Validate if default recommendations are displayed for new profiles |
| TP2 | Validate if personalized recommendations are displayed for profiles that have both bookmarks and navigation history |
| TP3 | Validate if personalized recommendations are displayed for profiles that have installed add-ons (both extensions and themes) |
| TP4 | Validate if personalized recommendations are displayed for profiles that have both installed add-ons and bookmarks |
| TP5 | Validate if default recommendations are displayed when the feature is deactivated |
| TP6 | Validate if ‘Create your own theme’ link is displayed and works as intended |
Builds
This section should contain links for builds with the feature -
Test Execution Schedule
The below table outlines the anticipated testing time frame available for test execution.
| Project phase | Start Date | End Date |
|---|---|---|
| Start project | 10/18/2018 | |
| Study documentation/specs received from developers | 11/02/2018 | |
| QA - Test plan creation | 12/23/2018 | |
| QA - Test cases/Env preparation | 01/03/2019 | |
| QA - Nightly Testing | ||
| QA - Beta Testing | ||
| Release Date |
Testing Tools
Exemplifies the tools used for test suite creation/execution.
| Process | Tool |
|---|---|
| Test plan creation | Mozilla wiki |
| Test case creation | TestRail |
| Test case execution | TestRail |
| Bugs management | Bugzilla |
References
* List and links for specs OnePager - Gdocs Invision - Presentation * bug 1523406 - [meta] Recommendations on about:addons in Extensions & Themes
8 Total; 1 Open (12.5%); 5 Resolved (62.5%); 2 Verified (25%);
Testcases
Overview
The test suite proposes a series of test cases devised to cover scenarios where default or recommended extensions will be displayed, in order to validate the feature.
- Validation of default/personalized recommendations being displayed based on profile usage
- Validation of default recommendations being displayed during private browsing
Test Areas
| Test Areas | Covered | Details |
|---|---|---|
| Private Browsing | ||
| Profile Testing | ||
| Feature On/Off Toggling |
Sign off
Criteria
Check list
- All test cases should be executed
- All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)
Checklist
| Exit Criteria | Status | Notes/Details |
|---|---|---|
| Testing Prerequisites (specs, use cases) | ||
| Testing Infrastructure setup | ||
| Test Plan Creation | 04-01-2019 | |
| Test Cases Creation | ||
| Full Functional Tests Execution | ||
| Automation Coverage | ||
| Performance Testing | ||
| All Defects Logged | ||
| Critical/Blockers Fixed and Verified | ||
| Metrics/Telemetry | ||
| QA Signoff - Nightly Release | Email to be sent | |
| QA Beta - Full Testing | ||
| QA Signoff - Beta Release | Email to be sent |
