QA/Platform/Graphics/Features/GPU Process

From MozillaWiki
< QA‎ | Platform‎ | Graphics
Revision as of 22:16, 8 November 2016 by Ashughes (talk | contribs)
Jump to navigation Jump to search

Overview

Purpose

The purpose of this document is to describe the GPU Process feature from a testing perspective, including the following details:

  • Scope, focus areas and objectives of testing
  • Owners and points of contact for areas of responsibility
  • Strategy and types of testing
  • The entry and exit criteria
  • The basis of the test estimates
  • Any risks, issues, assumptions and test dependencies
  • The test schedule and major milestones
  • The test deliverables

Ownership

Developers
QA
Management

Scope

The following is in scope from a testing perspective for the project team.

  • GPU process startup time
  • Page load time
  • GPU process restart time (in case of process kill)
  • Real world performance with various common usecases (webgl, canvas2d, flash).
  • Plugin startup time
  • Antivirus conflicts
  • Memory footprint
  • Shutdown time
  • Overall product stability
  • Scrolling performance and correctness

The following is not in scope from a testing perspective for the project team.

  • to be determined

Schedule

Project phase Date Started Date Completed
Planning
Development planning & Spec documentation
Test planning & Spec review
Development
Code lands in Nightly
Test case development
Test environment preparation
QA
Nightly testing & sign-off
Aurora testing & sign-off
Beta testing & sign-off
RC testing & sign-off
Release

Outstanding Questions

  • What testing is out of scope for this feature?
  • What platforms, hardware, drivers, renderers, etc need coverage?

Test Coverage

Environments

  • Platforms: Windows, Mac, Linux, Android, iOS versions
  • Devices: AMD/Intel/NVIDIA/etc; Single/Multi/Hybrid/etc; Internal/External; Current/Beta/Legacy drivers
  • Other: 3rd-party software (Antivirus, Security, WindowsBlinds, etc), Preference changes
  • Channel Specific: Nightly, Aurora, Beta, Release

Risks

High-risk Assumptions
High-risk Bugs
Other Areas of Risk

Builds

This section should contain links for builds with the feature -

  • Links for Nightly builds
  • Links for Aurora builds
  • Links for Beta builds

Tools

Detail the tools to be used for testing, for example see the following table:

Process Tool
Test plan creation Mozilla wiki
Test case creation TestRail/ Google docs
Test case execution TestRail
Automated test execution Talos, etc...?
Bugs management Bugzilla

Use Cases

Test Areas Covered Details
Private Window
Multi-Process Enabled
Multi-process Disabled
Theme (high contrast)
UI
Mouse-only operation
Keyboard-only operation
Display (HiDPI)
Interraction (scroll, zoom)
Usable with a screen reader e.g. with NVDA
Usability and/or discoverability testing Is this feature user friendly
RTL build testing
Help/Support
Help/support interface required Make sure link to support/help page exist and is easy reachable.
Support documents planned(written) Make sure support documents are written and are correct.
Install/Upgrade
Feature upgrades/downgrades data as expected
Does sync work across upgrades
Requires install testing separate feature/application installation needed (not only Firefox)
Affects first-run or onboarding Florin/Lawrence are investigating if there is a dedicated QA for this, or we should test? Should be an yes/no and if is yes should add in detail column the team/person assigned.
Does this affect partner builds? Partner build testing yes/no options, add comment with details about who will lead testing
Enterprise Raise up the topic to developers to see if they are expecting to work different on ESR builds
Enterprise administration
Network proxies/autoconfig
ESR behavior changes
Locked preferences
Data Monitoring
Temporary or permanent telemetry monitoring List of error conditions to monitor
Telemetry correctness testing
Server integration testing
Offline and server failure testing
Load testing
Add-ons If add-ons are available for testing feature, or is current feature will affect some add-ons, then API testing should be done for the add-on.
Addon API required?
Comprehensive API testing
Permissions
Testing with existing/popular addons
Security Security is in charge of Matt Wobensmith. We should contact his team to see if security testing is necessary for current feature.
3rd-party security review
Privilege escalation testing
Fuzzing
Web Compatibility depends on the feature
Testing against target sites
Survey of many sites for compatibility
Interoperability depends on the feature
Common protocol/data format with other software: specification available. Interop testing with other common clients or servers.
Coordinated testing/interop across the Firefoxes: Desktop, Android, iOS
Interaction of this feature with other browser features

Test Suites

Full Test suite - Link with the gdoc, follow the format from link
Smoke Test suite - Link with the gdoc, follow the format from link
Regression Test suite - Link with the gdoc - if available/needed.

Bugs

Meta Bug - Dependency Tree
Full Query
ID Summary Status Resolution Depends on
1158076 Use light GTK theme for web content RESOLVED FIXED 1303310, 232227, 1216658, 1234279, 1282753, 1286000
1218576 Aggregate content process Telemetry in the main process VERIFIED FIXED 1468096, 1277504, 1282196, 1285006, 1286951, 1287101, 1290027, 1304009, 1304269, 1304519, 1304660, 1306856, 1306884, 1307031, 1307696, 1312778, 1356898
1255843 Develop a tool to measure resource usage of each subprocess RESOLVED FIXED 1277608, 1300850, 1300852, 1300853, 1300855, 1300856
1264545 Restrict Compositor access to nsIWidget RESOLVED FIXED
1265975 Remove compositor widget access on Windows RESOLVED FIXED 1264545, 1266536
1269037 Change CompositorVsyncScheduler to use widget proxies RESOLVED FIXED 1264545
1269653 Remove LayerManagerComposite access to nsIWidget RESOLVED FIXED 1271749
1271180 Introduce skeleton framework for the GPU process RESOLVED FIXED 1268559, 1277997
1271870 Remove compositor widget access on MacOS RESOLVED WONTFIX 1281686
1271872 Add process manager tool to developer tool. RESOLVED DUPLICATE
1272472 Isolate widget access to CompositorBridgeParent RESOLVED FIXED 1377415, 1273017
1274149 Restrict main-thread access to CompositorBridgeParent RESOLVED FIXED
1277068 Ignore SIGINT for all subprocesses RESOLVED FIXED
1277439 Add a non-Parent/Child protocol model RESOLVED WONTFIX
1280822 Make gfxPrefs usable in the GPU process RESOLVED FIXED
1281268 Measure overhead of GPU process on APZ RESOLVED FIXED
1281998 Implement IPDL version of CompositorWidgetProxy RESOLVED FIXED 1282343
1282348 Bootstrap out-of-process PCompositorBridge RESOLVED FIXED 1281575, 1281998, 1283744
1282364 Split up gfxWindowsPlatform RESOLVED FIXED 1288259, 1297182
1285625 Remote vsync notifications to the GPU process RESOLVED FIXED 1282348, 1285364
1287362 Don't use Preferences from CompositorLRU RESOLVED FIXED
1287366 Add ImageBridge support to the GPU process RESOLVED FIXED
1287597 Fix VRManager to work with the GPU process RESOLVED FIXED
1287917 Ensure ScrollMetadata::sNullMetadata is initialized in the GPU process RESOLVED FIXED
1287984 Content process cannot open GPU process DIBs RESOLVED FIXED
1288822 Create CompositorBridgeParent with a vsync rate RESOLVED FIXED
1289205 Fix typo in GPUChild::Init RESOLVED FIXED
1289251 Create a CompositorWidget for GTK+X11 Platform RESOLVED FIXED 1304199, 1304659
1289895 Add the GPU process as a broker peer RESOLVED FIXED
1294339 Don't use nsIGfxInfo in CompositorD3D11 RESOLVED FIXED
1294343 Remove fallback-to-WARP in the compositor RESOLVED FIXED
1294347 Always use DirectWrite if available RESOLVED FIXED
1294350 Fix GPUProcess startup issues RESOLVED FIXED
1294986 Disable DirectX in the UI process when using a GPU process RESOLVED FIXED
1294988 Enable Direct3D11 in the GPU process RESOLVED FIXED 1294986
1296749 Fix DirectX shutdown order RESOLVED FIXED
1297182 Add D3D9 support to the GPU process RESOLVED FIXED
1297251 [meta] Instant GPU Process Restarts NEW 1297260, 1337149, 1280715, 1297565, 1297567, 1297568, 1298938, 1300936, 1301780, 1302236, 1302611, 1303426, 1303451, 1303452, 1305628, 1305829, 1305897, 1305941, 1311715, 1311786, 1313770, 1314349, 1314375, 1314399, 1314408, 1314425, 1314427, 1314520, 1316419, 1316473, 1316476, 1316599, 1319213, 1319310, 1319557
1298507 Support tab-switching in the GPU process RESOLVED FIXED 1279086
1301266 Fix checkerboard reporting in the GPU process RESOLVED FIXED
1303820 Fix root scroll delta override when APZ is out of process RESOLVED FIXED
1304210 Fix crashes in ShadowLayerForwarder when the GPU process dies RESOLVED WONTFIX
1304494 Handle Telemetry::Accumulate in the GPU process RESOLVED FIXED
1305198 Fix null-deref when APZ is disabled in the compositor process RESOLVED FIXED
1307578 Let OOP compositor ride the trains for Windows RESOLVED FIXED 1294348, 1297790, 1297792, 1297822, 1297826, 1297828, 1297843, 1303451, 1307972, 1308398, 1309398, 1309714, 1309890, 1310110, 1310111, 1310397, 1310443, 1311716, 1311770, 1311786, 1312606, 1312835, 1313170, 1313199, 1313260, 1313499, 1313763, 1313978, 1314127, 1314133, 1314186, 1314349, 1314375, 1314399, 1314408, 1314425, 1314427, 1314520, 1314803, 1315699, 1315886, 1316419, 1316473, 1316481, 1316599, 1316632, 1317441, 1317796, 1318618, 1318845, 1319213, 1319310, 1319557, 1324095, 1325212, 1331005, 1339839, 1339852, 1343818
1314173 Add GPU process type to existing metrics RESOLVED FIXED 1314227, 1314231, 1314626
1314711 Killing the GPU process on Linux causes the parent process to terminate NEW
1314768 Tracking bug for all GPU process testing NEW 1297260, 1314426
1314804 D3D9DXVAManager still initializes in the UI process when the GPU process is enabled NEW
1317972 Work out what to do for crash reports with the GPU process NEW
1319206 Run a Telemetry experiment to vet GPU Process on Windows RESOLVED FIXED 1323304, 1325786
1321492 Add the GPU process to about:memory RESOLVED FIXED
1333880 D3D11 for h264 decoding fails since Firefox 53 when layers.gpu-process.enabled is true UNCONFIRMED
1338694 Add a telemetry probe for GPU_PROCESS_STATUS, similar to E10S_STATUS NEW
1343650 Annotate GPU Process in App Notes field RESOLVED FIXED
1350092 Run a Telemetry experiment to vet GPU Process on Windows prior to release in Firefox 53 RESOLVED FIXED
1352081 Google Maps experiences flickering when ending the GPU Process RESOLVED WONTFIX
1357744 No GPUProcessPid and GPIPRocess entries In the Graphics section of about:support in FIrefox 53 stable RESOLVED INCOMPLETE
1358304 Include GPUProcessPid in about:support for all release branches RESOLVED DUPLICATE
1362166 Enable IPDL timeout for communicating to the GPU process RESOLVED FIXED
1373088 Don't use a timeout for compositor process startup when debugging RESOLVED FIXED
1380144 Collect hang information from the compositor process RESOLVED DUPLICATE
1390488 GPU process crashes don't contain crash message (including rust panic messages) VERIFIED FIXED
1393703 Crash caused by Tab2QR add-on with GPU Process enabled NEW
1733680 [Linux/GPU] Crash at GPUProcessManager::SimulateDeviceReset() RESOLVED FIXED

65 Total; 8 Open (12.31%); 55 Resolved (84.62%); 2 Verified (3.08%);


Sign off

Exit Criteria Status Notes/Details
Testing Prerequisites (specs, use cases)
Testing Infrastructure setup
Test Plan Creation
Test Cases Creation
Full Functional Tests Execution
Automation Coverage
Performance Testing
All Defects Logged
Critical/Blockers Fixed and Verified
Metrics/Telemetry
QA Signoff - Nightly Release Email to be sent
QA Aurora - Full Testing
QA Signoff - Aurora Release Email to be sent
QA Beta - Full Testing
QA Signoff - Beta Release Email to be sent

References

Revision History

This section describes the modifications that have been made to this wiki page. A new row has been completed each time the content of this document is updated (small corrections for typographical errors do not need to be recorded). The description of the modification contains the differences from the prior version, in terms of what sections were updated and to what extent.

Date Version Author Description
2016-11-02 1.0 Anthony Hughes Created first draft
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Platform/Graphics/Features/GPU_Process&oldid=1154070"