QA/Platform/Graphics
Jump to navigation
Jump to search
Consult this page for more information.
Understanding the Problem Space
First order of business for my transition to the Graphics team is to understand the problem space so I can understand the immediate needs of the team and make the best impact I can in the shortest amount of time.
- What are the key problems/challenges facing the Graphics team in terms of quality?
- discrepancy in environments between testers and release users
- discoverability of bugs pre-release
- ?...
- Where can QA add value/support to the Graphics team?
- improving pre-release discoverability of bugs
- closing the gap between tester and release systems
- helping with bug triage, particularly with bugs hiding in general components
- representation in crashkill
- improving code coverage and/or identifying gaps in code coverage
- identifying ways to improve participation in the graphics team (events, projects, One & Done, etc)
- documentation of tools, testing processes, etc
- building out the lab in Toronto
- continuing to drive Betabreakers testing every 6 weeks
- verifying bug fixes (what does this look like)?
- profiling areas of risk (eg. troublesome configs)
- conducting root cause analysis for regressions
- understanding problems outside of our control (eg. driver resets)
- feature testing and upcoming priorities (e10s, Windows 10, El Capitain, Android, B2G, etc)
- What does QA need to know to be effective?
- key components of an actionable Graphics bug
- fundamentals/technologies that should be learned
- how to distinguish a graphics crash from a non-graphics crash with a graphics signature
- meetings, mailing lists, bugzilla components to watch, blogs, IRC channels to join, etc
- who is each member of the team (incl. contributors) and what do they do
- where does graphics code reside in the tree?
- what role does Unified Telemetry in graphics quality?
- what are the prefs to enable/disable different functionalities?
- we need a database of known-troublesome hardware/driver configurations to inform testing, hardware acquisitions, and blocklisting
Sanity Checking
- Desktop
- Boot 2 Gecko (No-Jun Park)
- Android
- Telemetry
Stability
How do we identify a graphics crash?
- by signature: gfx, layers, D2D, D3D, ?...
- by topmost filename: gfx, ?...
- ?...
How do we prioritize graphics crashes?
- Overall topcrashes in release > beta > aurora > nightly
- Gfx crashes in release > beta > aurora > nightly
- Explosive crashes in release > beta > aurora > nightly
What tools do we have at our disposal to investigate crashes?
- Bughunter for investigating crashes correlated to a URL
- KaiRo's reports for identifying crashes that are new or escalating quickly
- Socorro for getting detailed information about crash reports
What information is needed to make a crash actionable by developers?
- Correlations to particular hardware, driver, add-on, 3rd-party software, or library
- ?...
Features
- Gecko 39: OOM driver issues
- Gecko 40: OMTC on all platforms
- Gecko 41: WebGL 2, E10S M3
- Gecko 42: Desktop Tiling, Desktop APZ, Desktop Silk
Participation
- Sanity checking via One & Done
- Meetups to connect testers/users with devs
- Testdays to teach people about graphics testing
- Documentation and translation of documentation
- Engaging on community spaces (Discourse, Reddit, Facebook, Twitter, etc)
Betabreakers
Testing:
- [DONE] Firefox 38: MSE stress test
- [DONE] Firefox 39: beta sanity check
- [DONE] Firefox 40: WebGL with e10s
- Firefox 41: exploratory testing Windows 10 in Aurora, gfx-noted bugs (eg. [1], [2], [3], [4])
- Firefox 42: to be determined
- Firefox 43: to be determined
- Firefox 44: to be determined
Risks:
- Betabreakers will not have Windows 10 deployed to machines until after it is officially released. However, they can deploy Preview to select machines upon request. We need to develop a set of requirements for Windows 10 testing, particularly machine specifications for any upcoming testrun that targets Windows 10.
- We need to select hardware for testing based on data from past testruns and known-troublesome hardware
- We need to identify gaps in test coverage and investigate whether they can fill these gaps for us
Guides
Interesting Initiatives
Device Inventory
- To Do
- Catalog Vancouver hardware
- Catalog mobile hardware
- Correlate to Telemtry
Mobile devices can be found here
| System | GPU 1 | Device ID | GPU 2 | Device ID | Operating System | Memory |
|---|---|---|---|---|---|---|
| Asus G73JH | AMD Mobility™ Radeon® HD 5870 | 0x1002:? | ||||
| Dell Precision T1600 | AMD Radeon R7 240 | 0x1002:? | Intel HD 3000 | 0x8086:? | ||
| Dell Precision T3500 | AMD Radeon HD 5670 | 0x1002:68d8 | ||||
| Dell Vostro | NVidia GeForce GTS 450 | 0x10de:? | ||||
| Dell Vostro 240s | Intel G45 | 0x8086:2e22 | ||||
| Dell XPS L501X | NVidia GeForce GT 420M | 0x10de:046e | Intel HD Graphics (Core i3) | 0x8086:0046 | ||
| Hawkman | Intel HD 4600 | 0x8086:0412 | ||||
| Macbook Pro (?,?) | NVidia GeForce 8600M | 0x10de:0407 | ||||
| PoisonIvy | Intel HD 4000 | 0x8086:? | ||||
| Silly Box | Intel G41 | 0x8086:2e32 | ||||
| Thinkpad G500 | AMD Radeon HD 8570M | 0x1002:? | Intel HD 4000 | 0x8086:0166 | ||
| Thinkpad G575 | AMD Radeon HD 6250 | 0x1002:9804 | ||||
| Thinkpad R32 | ||||||
| Thinkpad T500 | AMD Radeon HD 3650 | 0x1002:? | ||||
| Thinkpad T510 | NVidia NVS 3100M | 0x10de:0a6c | ||||
| Thinkpad W510 | NVidia Quadro 880M | 0x10de:? | ||||
| Thinkpad x220 | Intel HD 3000 | 0x8086:? |
- Loose Graphics Cards
- AMD Radeon X1600
- PowerColor AMD Radeon HD 5770
Phones
| Device | CPU | GPU | Memory | Platform | Location |
|---|---|---|---|---|---|
| Alcatel OneTouch Fire | Qualcomm MSM7227A Snapdragon S1 | Adreno 200 | 256 MB | B2G | Toronto (Kartikaya Gupta) |
| Flame | ? | ? | ? | B2G | Toronto (Kartikaya Gupta) |
| Geeksphone Peak | Qualcomm Snapdragon S4 8225 | ? | 512 MB | B2G | Toronto (Kartikaya Gupta) |
| Huawei Ascend Y300 | Qualcomm MSM8225 Snapdragon S4 Play | Adreno 203 | 512 MB | Toronto (Botond Ballo) | |
| LG Nexus 4 | Qualcomm APQ8064 Snapdragon S4 Pro | Adreno 320 | 2048 MB | B2G | Toronto (Botond Ballo) |
| LG Nexus 4 | Qualcomm APQ8064 Snapdragon S4 Pro | Adreno 320 | 2048 MB | Android 4.2.2 | Toronto (Kartikaya Gupta) |
| Motorola RAZR i | Intel Atom Z2460 | PowerVR SGX540 | 2048 MB | Android 4.0.4 | Toronto (Botond Ballo) |
| Samsung GT-I5510M | Qualcomm MSM7227 Snapdragon S1 | Adreno 200 | 160 MB | Android 2.2.2 | Toronto (Botond Ballo) |
| Samsung Galaxy Nexus | TI OMAP 4460 (ARMv7A Cortex-A9) | PowerVR SGX540 | 1024 MB | Android 4.2.2 | Toronto (Kartikaya Gupta) |
| Samsung Galaxy Q SGH-T589W | ARMv6 800 MHz | ? | 270 MB | Android 2.3.6 | Toronto (Kartikaya Gupta) |
| Sony Xperia Z3C | Qualcomm Snapdragon 2.5 GHz | Adreno 330 | 2048 MB | Toronto (Kartikaya Gupta) |
List of devices held by the Fennec team (may be outdated)