QA/Platform/Graphics/Features/GPU Process

Overview

Purpose

The purpose of this document is to describe the GPU Process feature from a testing perspective, including the following details:

  • Scope, focus areas and objectives of testing
  • Owners and points of contact for areas of responsibility
  • Strategy and types of testing
  • The entry and exit criteria
  • The basis of the test estimates
  • Any risks, issues, assumptions and test dependencies
  • The test schedule and major milestones
  • The test deliverables

Ownership

Developers
QA
Management

Scope

The following is in scope from a testing perspective for the project team.

  • GPU process startup time
  • Page load time
  • GPU process restart time (in case of process kill)
  • Real world performance with various common usecases (webgl, canvas2d, flash).
  • Plugin startup time
  • Antivirus conflicts
  • Memory footprint
  • Shutdown time
  • Overall product stability
  • Scrolling performance and correctness

The following is not in scope from a testing perspective for the project team.

  • to be determined

Schedule

Project phase Date Started Date Completed
Planning
Development planning & Spec documentation
Test planning & Spec review
Development
Code lands in Nightly
Test case development
Test environment preparation
QA
Nightly testing & sign-off
Aurora testing & sign-off
Beta testing & sign-off
RC testing & sign-off
Release

Outstanding Questions

What are the coverage requirements (platforms, devices, etc)?
  • David Anderson
    • Machines that have (1) Windows 7 with the Platform Update, or Windows 8 and higher, and (2) Direct3D 11 compositing before the GPU process, should now be using a Direct3D 11 in the compositor GPU process.
    • We don't really care about machines on Windows 7 w/o PU or lower, or machines that don't get D3D11. They should not be using the GPU process. [ashughes: we should probably test for this]
What are the release criteria for each milestone and how will this criteria be measured?
  • David Anderson
    • I might be missing some things here, but we should have evidence that UI crashes on covered systems have moved to the GPU process. Of course, we should also know that we have not regressed acceleration coverage, performance/startup time or rendering quality.
    • At this point there's only one real milestone and it's "ride the trains for 53".
What are the use cases that need to be covered?
What can/should be covered by automation vs manual testing and what test infrastructure will be used?
  • David Anderson
    • Automated testing has done a good job making sure haven't had any obvious regressions. A big open problem is that we don't have any testing for when the GPU process crashes. We have no testing that we recover properly. Currently I'm doing this by hand. I have a patch that I send to try about twice a day that simulates random GPU crashes. I then sort through all the columns and see what failures come up.
    • Having some way to automate this would be amazing. Having at least some deterministic tests on infrastructure would be great too. I hope to work on the latter soon now that current manual testing is looking good.
    • Another area we lack in testing is device resets. I don't know how to approach this yet, but it's hard to do on try.
    • There is one testing hole that will be patched up soon: after a device reset we can wind up with a software compositor in the GPU process. I requested a new reftest column to test this condition on TreeHerder (it'll run on Windows 7 machines by forcing the GPU process on).
What are the specific things we want to test with Telemetry Experiments vs System Addons?
What are the areas of risk? As an example, are there things that Apple/Google/Microsoft, AMD/Intel/NVIDIA, or other third-party vendors could do that would impact the release criteria and/or schedule?
  • David Anderson
    • We've gotten an upset user complaining about Skia fonts in the UI now. I don't know how to evaluate that risk. I guess the biggest risk is we mess up and a lot of users stop getting D3D11, but we haven't seen that yet.
    • One thing I'm not too concerned about, to an extent, is whether moving to the GPU process *causes* more acceleration crashes. Unless it's a horrific amount of extra crashes, that's a fine result.

Test Coverage

Environments

  • Platforms: Windows, Mac, Linux, Android, iOS versions
  • Devices: AMD/Intel/NVIDIA/etc; Single/Multi/Hybrid/etc; Internal/External; Current/Beta/Legacy drivers
  • Other: 3rd-party software (Antivirus, Security, WindowsBlinds, etc), Preference changes
  • Channel Specific: Nightly, Aurora, Beta, Release

Risks

High-risk Assumptions
High-risk Bugs
Other Areas of Risk

Builds

This section should contain links for builds with the feature -

  • Links for Nightly builds
  • Links for Aurora builds
  • Links for Beta builds

Tools

Detail the tools to be used for testing, for example see the following table:

Process Tool
Test plan creation Mozilla wiki
Test case creation TestRail/ Google docs
Test case execution TestRail
Automated test execution Talos, etc...?
Bugs management Bugzilla
Experimentation Telemetry Experiments

Use Cases

Test Areas Covered Details
Private Window
Multi-Process Enabled
Multi-process Disabled
Theme (high contrast)
UI
Mouse-only operation
Keyboard-only operation
Display (HiDPI)
Interraction (scroll, zoom)
Usable with a screen reader e.g. with NVDA
Usability and/or discoverability testing Is this feature user friendly
RTL build testing
Help/Support
Help/support interface required Make sure link to support/help page exist and is easy reachable.
Support documents planned(written) Make sure support documents are written and are correct.
Install/Upgrade
Feature upgrades/downgrades data as expected
Does sync work across upgrades
Requires install testing separate feature/application installation needed (not only Firefox)
Affects first-run or onboarding Florin/Lawrence are investigating if there is a dedicated QA for this, or we should test? Should be an yes/no and if is yes should add in detail column the team/person assigned.
Does this affect partner builds? Partner build testing yes/no options, add comment with details about who will lead testing
Enterprise Raise up the topic to developers to see if they are expecting to work different on ESR builds
Enterprise administration
Network proxies/autoconfig
ESR behavior changes
Locked preferences
Data Monitoring
Temporary or permanent telemetry monitoring List of error conditions to monitor
Telemetry correctness testing
Server integration testing
Offline and server failure testing
Load testing
Add-ons If add-ons are available for testing feature, or is current feature will affect some add-ons, then API testing should be done for the add-on.
Addon API required?
Comprehensive API testing
Permissions
Testing with existing/popular addons
Security Security is in charge of Matt Wobensmith. We should contact his team to see if security testing is necessary for current feature.
3rd-party security review
Privilege escalation testing
Fuzzing
Web Compatibility depends on the feature
Testing against target sites
Survey of many sites for compatibility
Interoperability depends on the feature
Common protocol/data format with other software: specification available. Interop testing with other common clients or servers.
Coordinated testing/interop across the Firefoxes: Desktop, Android, iOS
Interaction of this feature with other browser features
Interaction of this feature with third-party software (Antivirus, Security, Games, Theming, etc)

Test Suites

Full Test suite - Link with the gdoc, follow the format from link
Smoke Test suite - Link with the gdoc, follow the format from link
Regression Test suite - Link with the gdoc - if available/needed.

Links for the tests run

    • Daily Smoke, use template from link
    • Full Test suite, use template from link
    • Regression Test suite, if needed/available

Experiments

Experiment Contact Tracking Bug Status Start Date End Date
A/B test layers.gpu-process.dev.enabled in Nightly 53 Anthony Hughes (:ashughes) bug 1319206 In Development 2016-11-28 2016-12-12
A/B test media.gpu-process-decoder in Nightly 53 Anthony Hughes (:ashughes)

Nightly 53 A/B Test

Purpose:

  • Compare crash rates for users with GPU process enabled vs users with GPU process disabled

Dependencies:

  • How do we deal with the fact that the pref is set early during startup? (emailed felipeg)
  • Telemetry probes to measure crash rates
  • Telemetry probe to measure proportion of users in each cohort

Measurements:

Bugs

Meta Bug - Dependency Tree
Full Query
ID Summary Status Resolution Depends on
1158076 Use light GTK theme for web content RESOLVED FIXED 1303310, 232227, 1216658, 1234279, 1282753, 1286000
1218576 Aggregate content process Telemetry in the main process VERIFIED FIXED 1468096, 1277504, 1282196, 1285006, 1286951, 1287101, 1290027, 1304009, 1304269, 1304519, 1304660, 1306856, 1306884, 1307031, 1307696, 1312778, 1356898
1255843 Develop a tool to measure resource usage of each subprocess RESOLVED FIXED 1277608, 1300850, 1300852, 1300853, 1300855, 1300856
1264545 Restrict Compositor access to nsIWidget RESOLVED FIXED
1265975 Remove compositor widget access on Windows RESOLVED FIXED 1264545, 1266536
1269037 Change CompositorVsyncScheduler to use widget proxies RESOLVED FIXED 1264545
1269653 Remove LayerManagerComposite access to nsIWidget RESOLVED FIXED 1271749
1271180 Introduce skeleton framework for the GPU process RESOLVED FIXED 1268559, 1277997
1271870 Remove compositor widget access on MacOS RESOLVED WONTFIX 1281686
1271872 Add process manager tool to developer tool. RESOLVED DUPLICATE
1272472 Isolate widget access to CompositorBridgeParent RESOLVED FIXED 1377415, 1273017
1274149 Restrict main-thread access to CompositorBridgeParent RESOLVED FIXED
1277068 Ignore SIGINT for all subprocesses RESOLVED FIXED
1277439 Add a non-Parent/Child protocol model RESOLVED WONTFIX
1280822 Make gfxPrefs usable in the GPU process RESOLVED FIXED
1281268 Measure overhead of GPU process on APZ RESOLVED FIXED
1281998 Implement IPDL version of CompositorWidgetProxy RESOLVED FIXED 1282343
1282348 Bootstrap out-of-process PCompositorBridge RESOLVED FIXED 1281575, 1281998, 1283744
1282364 Split up gfxWindowsPlatform RESOLVED FIXED 1288259, 1297182
1285625 Remote vsync notifications to the GPU process RESOLVED FIXED 1282348, 1285364
1287362 Don't use Preferences from CompositorLRU RESOLVED FIXED
1287366 Add ImageBridge support to the GPU process RESOLVED FIXED
1287597 Fix VRManager to work with the GPU process RESOLVED FIXED
1287917 Ensure ScrollMetadata::sNullMetadata is initialized in the GPU process RESOLVED FIXED
1287984 Content process cannot open GPU process DIBs RESOLVED FIXED
1288822 Create CompositorBridgeParent with a vsync rate RESOLVED FIXED
1289205 Fix typo in GPUChild::Init RESOLVED FIXED
1289251 Create a CompositorWidget for GTK+X11 Platform RESOLVED FIXED 1304199, 1304659
1289895 Add the GPU process as a broker peer RESOLVED FIXED
1294339 Don't use nsIGfxInfo in CompositorD3D11 RESOLVED FIXED
1294343 Remove fallback-to-WARP in the compositor RESOLVED FIXED
1294347 Always use DirectWrite if available RESOLVED FIXED
1294350 Fix GPUProcess startup issues RESOLVED FIXED
1294986 Disable DirectX in the UI process when using a GPU process RESOLVED FIXED
1294988 Enable Direct3D11 in the GPU process RESOLVED FIXED 1294986
1296749 Fix DirectX shutdown order RESOLVED FIXED
1297182 Add D3D9 support to the GPU process RESOLVED FIXED
1297251 [meta] Instant GPU Process Restarts NEW 1297260, 1337149, 1280715, 1297565, 1297567, 1297568, 1298938, 1300936, 1301780, 1302236, 1302611, 1303426, 1303451, 1303452, 1305628, 1305829, 1305897, 1305941, 1311715, 1311786, 1313770, 1314349, 1314375, 1314399, 1314408, 1314425, 1314427, 1314520, 1316419, 1316473, 1316476, 1316599, 1319213, 1319310, 1319557
1298507 Support tab-switching in the GPU process RESOLVED FIXED 1279086
1301266 Fix checkerboard reporting in the GPU process RESOLVED FIXED
1303820 Fix root scroll delta override when APZ is out of process RESOLVED FIXED
1304210 Fix crashes in ShadowLayerForwarder when the GPU process dies RESOLVED WONTFIX
1304494 Handle Telemetry::Accumulate in the GPU process RESOLVED FIXED
1305198 Fix null-deref when APZ is disabled in the compositor process RESOLVED FIXED
1307578 Let OOP compositor ride the trains for Windows RESOLVED FIXED 1294348, 1297790, 1297792, 1297822, 1297826, 1297828, 1297843, 1303451, 1307972, 1308398, 1309398, 1309714, 1309890, 1310110, 1310111, 1310397, 1310443, 1311716, 1311770, 1311786, 1312606, 1312835, 1313170, 1313199, 1313260, 1313499, 1313763, 1313978, 1314127, 1314133, 1314186, 1314349, 1314375, 1314399, 1314408, 1314425, 1314427, 1314520, 1314803, 1315699, 1315886, 1316419, 1316473, 1316481, 1316599, 1316632, 1317441, 1317796, 1318618, 1318845, 1319213, 1319310, 1319557, 1324095, 1325212, 1331005, 1339839, 1339852, 1343818
1314173 Add GPU process type to existing metrics RESOLVED FIXED 1314227, 1314231, 1314626
1314711 Killing the GPU process on Linux causes the parent process to terminate NEW
1314768 Tracking bug for all GPU process testing NEW 1297260, 1314426
1314804 D3D9DXVAManager still initializes in the UI process when the GPU process is enabled NEW
1317972 Work out what to do for crash reports with the GPU process NEW
1319206 Run a Telemetry experiment to vet GPU Process on Windows RESOLVED FIXED 1323304, 1325786
1321492 Add the GPU process to about:memory RESOLVED FIXED
1333880 D3D11 for h264 decoding fails since Firefox 53 when layers.gpu-process.enabled is true UNCONFIRMED
1338694 Add a telemetry probe for GPU_PROCESS_STATUS, similar to E10S_STATUS NEW
1343650 Annotate GPU Process in App Notes field RESOLVED FIXED
1350092 Run a Telemetry experiment to vet GPU Process on Windows prior to release in Firefox 53 RESOLVED FIXED
1352081 Google Maps experiences flickering when ending the GPU Process RESOLVED WONTFIX
1357744 No GPUProcessPid and GPIPRocess entries In the Graphics section of about:support in FIrefox 53 stable RESOLVED INCOMPLETE
1358304 Include GPUProcessPid in about:support for all release branches RESOLVED DUPLICATE
1362166 Enable IPDL timeout for communicating to the GPU process RESOLVED FIXED
1373088 Don't use a timeout for compositor process startup when debugging RESOLVED FIXED
1380144 Collect hang information from the compositor process RESOLVED DUPLICATE
1390488 GPU process crashes don't contain crash message (including rust panic messages) VERIFIED FIXED
1393703 Crash caused by Tab2QR add-on with GPU Process enabled NEW
1733680 [Linux/GPU] Crash at GPUProcessManager::SimulateDeviceReset() RESOLVED FIXED

65 Total; 8 Open (12.31%); 55 Resolved (84.62%); 2 Verified (3.08%);


Sign off

Nightly

Criteria Status Notes/Details
Testing Prerequisites (specs, use cases)
Testing Infrastructure setup
Test Plan Creation
Test Cases Creation
Full Functional Tests Execution
Automation Coverage
Performance Testing
All Defects Logged
Critical/Blockers Fixed and Verified
Metrics/Telemetry
QA Signoff - Okay to Merge Email to be sent
QA Signoff - Aurora Migration Email to be sent
QA Signoff - Verify Aurora Release Notes Email to be sent

Aurora

Criteria Status Notes/Details
Testing Prerequisites (specs, use cases)
Testing Infrastructure setup
Test Plan Creation
Test Cases Creation
Full Functional Tests Execution
Automation Coverage
Performance Testing
All Defects Logged
Critical/Blockers Fixed and Verified
Metrics/Telemetry
QA Signoff - Okay to Merge Email to be sent
QA Signoff - Beta Migration Email to be sent
QA Signoff - Verify Beta Release Notes Email to be sent

Beta

Criteria Status Notes/Details
Testing Prerequisites (specs, use cases)
Testing Infrastructure setup
Test Plan Creation
Test Cases Creation
Full Functional Tests Execution
Automation Coverage
Performance Testing
All Defects Logged
Critical/Blockers Fixed and Verified
Metrics/Telemetry
QA Signoff - Beta 1..n Email to be sent
QA Signoff - Release Migration Email to be sent

Release

Criteria Status Notes/Details
Testing Prerequisites (specs, use cases)
Testing Infrastructure setup
Test Plan Creation
Test Cases Creation
Full Functional Tests Execution
Automation Coverage
Performance Testing
All Defects Logged
Critical/Blockers Fixed and Verified
Metrics/Telemetry
QA Signoff - Okay to Release Email to be sent
QA Signoff - Verify Release Notes Email to be sent

References

Revision History

This section describes the modifications that have been made to this wiki page. A new row has been completed each time the content of this document is updated (small corrections for typographical errors do not need to be recorded). The description of the modification contains the differences from the prior version, in terms of what sections were updated and to what extent.

Date Version Author Description
2016-11-02 1.0 Anthony Hughes Created first draft