QA/Photon Performance
Revision History
This section describes the modifications that have been made to this wiki page. A new row has been completed each time the content of this document is updated (small corrections for typographical errors do not need to be recorded). The description of the modification contains the differences from the prior version, in terms of what sections were updated and to what extent.
| Date | Version | Author | Description |
|---|---|---|---|
| 08/11/2017 | 1.0 | Adrian Florinescu | Initial Draft |
Purpose
Overview
Photon is the codename for the Firefox UI refresh on the Firefox front end to be the user facing component of Quantum. Photon Performance is front-end specific performance improvements that complement Quantum work. Summing up, Photon Performance consists of performance improvements and user perceived performance improvements.
Ownership
Engineering lead Justin Dolske
Engineering Team Florian Quèze Mike Conley
QA Adrian Florinescu
References
Testing summary
Scope of Testing
In Scope
- what's expected from the manual QA point of view is to ensure the Photon Performance changes do not introduce regressions and the areas impacted are covered by exploratory/smoke/regression testing;
- the depth of varies based on the FF area that shall be checked;
- validating user perceived improvements;
Out of Scope
- validating the actual performance improvements - excepting the visual ones (user perceived improvements)
Environments
Windows 7, 8.1, 10 Ubuntu 16.04 Mac 10.12, 10.10
Test Strategy
Risk Assessment and Coverage
| ID | Description / Threat Description | Covered by Test Objective | Magnitude | Probability | Priority | Impact Score |
|---|---|---|---|---|---|---|
| RAC-1 | TO-1 | 2-Moderate | 1-Unlikely | 1-Low | 4 |
Values:
- Magnitude: 1- Low , 2-Moderate, 3-High
- Probability: 1-Unlikely, 2-Possible, 3-Almost Certain
- Priority: 1 - Low, 2-Medium, 3-High
Impact Score Breakdown:
- An impact value of 1, 2, 3, 4 would describe an area which although should be covered there aren't expected any discoveries of critical issues.
- An impact value of 6, 8, 9, 12 would describe an area in which we expect to find issues but those issues are not expected to be critical.
- An impact value of 18 or 27 would describe an area on which it is likely to find issues and those issues to be critical or blockers.
Test Objectives
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master. This could be documented in bullet form or in a table similar to the one below.
| Main area | Sub-areas | Additional Details | Enviroments | TestSuite Coverage (%) | RAC |
|---|---|---|---|---|---|
| Captive Portal | basic functionality | w-l-m | 100% | ||
| Active/Inactive CP detection | w-l-m | 100% | |||
| CP managers(OSX/Win) | focused on startup | w-m | 100% | ||
| Set FF Default | w-l-m | 0% | |||
| set from preferences / startup | w-l-m | 0% | |||
| set from command line | w-l-m | 0% | |||
| Tab labels behavior | tab label display and transitions during operations | w-l-m | 0% | ||
| session restore | w-l-m | 0% | |||
| general tab label transitions | w-l-m | 0% | |||
| Window animation suppression | w-l-m | 0% | |||
| FF open with params | OS animation is suppresed | w-l-m | 0% | ||
| FF open from external | OS animation is suppresed | w-l-m | 0% | ||
| FF scroll & layout | |||||
| FF basic scroll | UI not content (tab strip, library, long autocomplete pop-ups -eg. autofill) | w-l-m | 5% | ||
| touch/trackpads scrolling | w-l-m | 10% | |||
| Search panel | pozition and size of search panel is recalculated corectly on FF actions | w-l-m | 0% | ||
| Customized position | w-l-m | 0% | |||
| FF resize | w-l-m | 0% | |||
| drag and drop with resize | w-l-m | 0% | |||
| DPI changes | w-l-m | 0% | |||
| HiDPI | w-l-m | 0% | |||
| MousePos flush (1307134 ) | status bar is swaped from L to R and other way around | w-l-m | 0% | ||
| Restore session - windows size and positions | w-l-m | 0% | |||
| Themes | Activate/deactivate window/tab | w-l-m | 0% | ||
| Tab switch | w-l-m | 0% |
Test Execution Schedule
The following table identifies the anticipated testing period available for test execution.
| Project phase | Start Date | End Date |
|---|---|---|
| Start project | ||
| Study documentation/specs received from developers | ||
| QA - Test plan creation | ||
| QA - Test cases/Env preparation | ||
| QA - Nightly Testing | ||
| QA - Beta Testing | ||
| Release Date |
Testing Tools
Detail the tools to be used for testing, for example see the following table:
| Process | Tool |
|---|---|
| Test plan creation | Mozilla wiki |
| Test case creation | TestRail |
| Test case execution | TestRail |
| Bugs management | [ Bugzilla] |
Sign off Criteria
TBD