QA/Photon Performance
Revision History
This section describes the modifications that have been made to this wiki page. A new row has been completed each time the content of this document is updated (small corrections for typographical errors do not need to be recorded). The description of the modification contains the differences from the prior version, in terms of what sections were updated and to what extent.
| Date | Version | Author | Description |
|---|---|---|---|
| 08/11/2017 | 1.0 | Adrian Florinescu | Initial Draft |
| 08/17/2017 | 2.0 | Adrian Florinescu | RAC/TO updates |
Purpose
Overview
Photon is the codename for the Firefox UI refresh on the Firefox front end to be the user facing component of Quantum. Photon Performance is front-end specific performance improvements that complement Quantum work. Summing up, Photon Performance consists of performance improvements and user perceived performance improvements.
Ownership
Engineering lead Justin Dolske
Engineering Team Florian Quèze Mike Conley
QA Adrian Florinescu
References
Testing summary
Scope of Testing
In Scope
- what's expected from the manual QA point of view is to ensure the Photon Performance changes do not introduce regressions and the areas impacted are covered by exploratory/smoke/regression testing;
- the depth of testing vary from case to case, based on the FF area that shall be checked;
- validating user perceived improvements;
Out of Scope
- validating the actual performance improvements - excepting the visual ones (user perceived improvements)
Environments
Windows 7, 8.1, 10 Ubuntu 16.04 Mac 10.12, 10.10
Test Strategy
Risk Assessment and Coverage
| ID | Description / Threat Description | Covered by Test Objective | Magnitude | Probability | Priority | Impact Score |
|---|---|---|---|---|---|---|
| RAC-1 | (Start-up)Captive Portal detection is not done at all or in fashionable time | TO-1, TO-2, TO-3 | 3-High | 2-Possible | 3-High | 18 |
| RAC-2 | (Start-up) Captive Portal detection and OS native captive portal agents(MAC/Win) do not integrate | TO-1, TO-2, TO-3 | 2-Moderate | 2-Possible | 3-High | 12 |
| RAC-3 | FF cannot be set as default from preferences, at startup or by console command | TO-4, To-5, TO-6 | 3-High | 2-Possible | 3-High | 18 |
| RAC-4 | Tab labels are not correctly displayed | TO-7, TO-8, TO-9 | 2-Moderate | 2-Possible | 3-High | 12 |
| RAC-5 | Window animation are enabled for FF (win/mac) | TO-10, TO-11, TO-12 | 2-Moderate | 2-Possible | 2-Medium | 8 |
| RAC-6 | Firefox scroll is not correctly enabled and handled | TO-14 | 3-High | 2-Possible | 2-Medium | 12 |
| RAC-7 | Firefox scrolling is not working smoothly when using trackpad | TO-15 | 2-Moderate | 2-Possible | 1-Low | 4 |
| RAC-8 | Firefox scrolling is not working smoothly when using touch | TO-15 | 2-Moderate | 2-Possible | 1-Low | 4 |
| RAC-9 | Search panel position is not correctly calculated or remembered on basic scenarios | TO-16 | 3-High | 1-Unlikely | 2-Medium | 6 |
| RAC-10 | Search panel position is not correctly calculated or remembered on HiDPI, customized position, FF resizes, drag and drop, multi screen | TO-17, TO-18, TO-19, TO-20, To-21 | 3-High | 2-Possible | 1-Low | 6 |
| RAC-11 | Windows size and position not remembered correctly after session restore | TO-23 | 2-Moderate | 3-Almost Certain | 2-Medium | 12 |
Values:
- Magnitude: 1- Low , 2-Moderate, 3-High
- Probability: 1-Unlikely, 2-Possible, 3-Almost Certain
- Priority: 1 - Low, 2-Medium, 3-High
Impact Score Breakdown:
- An impact value of 1, 2, 3, 4 would describe an area which although should be covered there aren't expected any discoveries of critical issues.
- An impact value of 6, 8, 9, 12 would describe an area in which we expect to find issues but those issues are not expected to be critical.
- An impact value of 18 or 27 would describe an area on which it is likely to find issues and those issues to be critical or blockers.
Test Objectives
This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master. This could be documented in bullet form or in a table similar to the one below.
| TO | Main area | Sub-areas | Additional Details | Enviroments | TestSuite Coverage (%) | RAC |
|---|---|---|---|---|---|---|
| 1 | Captive Portal | basic functionality | w-l-m | 100% | RAC1, RAC2 | |
| 2 | Active/Inactive CP detection | w-l-m | 100% | RAC1, RAC2 | ||
| 3 | CP managers(OSX/Win) | focused on startup | w-m | 100% | RAC1, RAC2 | |
| 4 | Set FF Default | w-l-m | 0% | RAC3 | ||
| 5 | set from preferences / startup | w-l-m | 0% | RAC3 | ||
| 6 | set from command line | w-l-m | 0% | RAC3 | ||
| 7 | Tab labels behavior | tab label display and transitions during operations | w-l-m | 60% | RAC4 | |
| 8 | session restore | w-l-m | 40% | RAC4 | ||
| 9 | general tab label transitions | w-l-m | 60% | RAC4 | ||
| 10 | Window animation suppression | w-l-m | 0% | RAC5 | ||
| 11 | FF open with params | OS animation is suppresed | w-l-m | 0% | RAC5 | |
| 12 | FF open from external | OS animation is suppresed | w-l-m | 0% | RAC5 | |
| 13 | FF scroll & layout | w-l-m | 0% | |||
| 14 | FF basic scroll | UI not content (tab strip, library, long autocomplete pop-ups -eg. autofill) | w-l-m | 70% | RAC6 | |
| 15 | touch/trackpads scrolling | w-l-m | 70% | RAC7 | ||
| 16 | Search panel | pozition and size of search panel is recalculated corectly on FF actions | w-l-m | 0% | RAC8 | |
| 17 | Customized position | w-l-m | 0% | RAC10 | ||
| 18 | FF resize | w-l-m | 0% | RAC10 | ||
| 19 | drag and drop with resize | w-l-m | 0% | RAC10 | ||
| 20 | DPI changes | w-l-m | 0% | RAC10 | ||
| 21 | HiDPI | w-l-m | 0% | RAC10 | ||
| 22 | MousePos flush (1307134 ) | status bar is swaped from L to R and other way around | w-l-m | 0% | ||
| 23 | Restore session - windows size and positions | w-l-m | 60% | RAC11 | ||
| 24 | Themes | Activate/deactivate window/tab | w-l-m | 70% | ||
| 25 | Tab switch | w-l-m | 0% | |||
| 26 | Startup | w-l-m | 0% |
Test Execution Schedule
The following table identifies the anticipated testing period available for test execution.
| Project phase | Start Date | End Date |
|---|---|---|
| Start project | 03/30/1017 | |
| Study documentation/specs received from developers | 03/30/1017 | |
| QA - Test plan creation | 08/11/2017 | |
| QA - Test cases/Env preparation | - | |
| QA - Nightly Testing | ||
| QA - Beta Testing | ||
| Release Date |
Testing Tools
Detail the tools to be used for testing, for example see the following table:
| Process | Tool |
|---|---|
| Test plan creation | Mozilla wiki |
| Test case creation | TestRail |
| Test case execution | TestRail |
| Bugs management | [ Bugzilla] |
Bug Work
Meta bug: 1348289 - (photon-performance-triage) [meta Photon - Improve perceived UI performance in Firefox]
Bugzilla query error
Array ( [type] => error [message] => http-bad-status [params] => Array ( [0] => 429 [1] => Unknown Error ) ) 1
Sign off Criteria
TBD
Approvals Required / Received
The following individuals are required to/have approved this Test Plan:
| Name | Title | Department | Approval Date | Method |
|---|---|---|---|---|
| Lawrence Mandel/Ryan VanderMeulen | QA Reviewer | Product Integrity | Date | |
| Florian Quèze | Engineering | Engineering | Date | |
| Marco Mucci | Product Management | Product Management | Date |