QA/Platform/Graphics

From MozillaWiki
< QA‎ | Platform
Jump to: navigation, search

Summary

This page holds important information related to testing and quality assurance for Gecko's graphics code.

Metrics

Tools

Device ID Catalogs:

Get Involved

Here are some ways you can help the Graphics team (and Mozilla):

Activities

One & Done Testing

DEPRECATION WARNING: One&Done is being decommissioned. In the meantime, all tasks have been archived to this wiki.


Betabreakers Lab Testing

Summary

There is a large discrepancy between the platform/hardware/driver configurations we see on Release versus what we see on our test branches. Additionally, the Graphics team and our testing community does not and can not provide sufficient test coverage to fill this gap. As a result we've historically been notified far too late about critical graphics issues triggering multiple chemspill releases and a loss of users.

In an effort to improve pre-release visibility of graphics regressions we have partnered with Betabreakers, a company that maintains a lab specifically designed to test various modern and legacy configurations of Windows versions, graphics hardware, and graphics drivers. We try to coordinate a graphics sanity testrun with them once per mozilla-aurora cycle, as detailed below.

More information about our relationship with Betabreakers can be found on mana.mozilla.org.

Test Runs

Bugs

Full Query
ID Summary Status
1144199 Test # 4: Bioshock Infinite video goes to infinite loading screen when resuming to play after putting computer to sleep RESOLVED
1144237 During Test 3 (Stress test) multiple videos buffer forever or just stop playing RESOLVED
1144242 Videos do not resume playing when coming out of sleep RESOLVED
1144257 DXVA: Firefox crashed when opening right click menu RESOLVED
1144258 Bioshock Ininite command video is distorted when setting quality to 1080p RESOLVED
1144272 Test # 3 (Stress Test) Video playback stutters when setting video resolution to max RESOLVED
1144282 DXVA crash with 3 HD video tabs open RESOLVED
1144283 Test # 5: (Canvas2d demo): No snowflakes are displayed RESOLVED
1144378 Stress test: Severe lag when when having multiple tabs open RESOLVED
1144411 Video corruption in 1080p RESOLVED
1144754 Some videos wont play, videos only play in 360p on Windows XP RESOLVED
1167621 Dead Trigger 2 Demo becomes laggy when multiple assets (enemies) are on screen RESOLVED
1167656 Audio stuttering on Auralux menu RESOLVED
1167681 Audio stutters while playing and persists after game ends in Dead Trigger 2 RESOLVED
1167700 Polycraft Demo does not play since Firefox 38 RESOLVED
1167778 X-Type game suffers from low frame rate RESOLVED
1168451 BSOD when playing Banana Bread game RESOLVED
1168467 Tanx game suffers from low framerate spikes RESOLVED
1172561 Mini Dell: Browser becomes laggy when the user scrolls the page up/down RESOLVED
1172596 Videos will not play from CNN.com VERIFIED
1172628 Setesh (System): Firefox becomes unresponsive when right clicking link, and selecting open in new tab RESOLVED
1172634 (Setesh) Firefox becomes unresponsive when selecting to play any CNN.com videos RESOLVED
1173000 Rajh (System) All videos load to black screen even after Flash is updated/installed RESOLVED
1173012 CNN videos won't play as the file is corrupt VERIFIED
1180317 some image black displayed with Firefox 40.0* RESOLVED
1184193 Some Windows 7 systems running WARP when they shouldn't be RESOLVED
1184618 Some images on flickr.com show striping artifacts RESOLVED
1185017 Crash when resizing browser while entering Google Street View after updating driver with browser open RESOLVED
1201126 1080p 60 FPS video stuttering and occasionally stopping playback RESOLVED
1217124 Expand, Minimize and Close buttons do not display RESOLVED
1217170 YouTube serves 360p VP8 video instead of Flash video to FF 40+ VERIFIED
1217240 Youtube Video Resolution has maximum resolution of 360p RESOLVED
1217547 Windows shell buttons (minimize/maximize/close) disappear when Windowblinds theme is applied if OMTC is enabled RESOLVED
1217566 Enhance untrusted connection pages with basic troubleshooting tips UNCONFIRMED
1217587 Youtube Video Resolution has maximum resolution of 360p RESOLVED
1217936 Max Youtube Video Resolution 720p RESOLVED
1217999 "You can skip to video in 5" countdown timer not moving, stuck watching ads RESOLVED
1218546 sec_error_ocsp_future_response error when loading various pages RESOLVED
1219425 "Let It Snow" Microsoft Edge demo acquires the incorrect window height of 32px NEW
1219959 Youtube videos max resolution is 360p Mozilla Firefox 41.0.2 for Ubuntu RESOLVED
1229171 Problem loading page, "The connection has timed out" when accessing www.zamunda.net RESOLVED
1229190 Content does not load when a new window is called RESOLVED
1229193 New windows do not retain firefox/windows themes correctly UNCONFIRMED
1229207 Close, minimize, etc, buttons do not show with blinds and firefox theme RESOLVED
1229872 Videos on Twitter homepage display "This browser does not support video playback" RESOLVED
1229904 vid.me videos fail to fallback to Flash playback on Windows XP RESOLVED
1229914 Black corruption appears when opening/closing tabs UNCONFIRMED
1229917 Incredibly slow performance while simultaneously loading several tabs RESOLVED
1229922 Error with tv.digikala.com video player in Firefox RESOLVED
1229970 Crash in `anonymous namespace''::ScriptLoaderRunnable::OnStartRequest while dragging tabs to their own window RESOLVED
1230625 Google Icons/Buttons do not render RESOLVED
1265809 Choppy video and sound on youtube.com on Shadowsong computer. RESOLVED
1265820 Choppy Video and Sound on youtube.com on Anetheron computer. RESOLVED
1265870 Choppy video on Youtube 360 at 2k and 4k on Asaad computer. RESOLVED
1265879 No video or audio on Twitch on Asaad computer VERIFIED
1265894 Choppy video on Youtube 360 on 4k and 1440 videos on Azeroth computer RESOLVED
1265901 Choppy video on Vimeo whenever full-screened on Azeroth computer RESOLVED
1265910 Choppy on all video when full-screened on Azeroth RESOLVED
1265918 Doesn't load 360 portion of youtube 360 and is choppy on 4k and 1440 on Lothar RESOLVED
1265933 Embedded facebook video shows no picture when on HD on Lothar computer. RESOLVED
1265935 Video on Vimeo gets choppy in 1080p and even choppier/lagging in fullscreen 1080p on Lothar computer RESOLVED
1265937 Any and all ads loaded are extremely choppy on Lothar computer RESOLVED
1266111 Choppy video on Vimeo at 1080P when full-screened on Shadowsong RESOLVED
1266114 Doesn't load 360 controls for Youtube 360 and gets choppy at 720p and up on Shadowsong computer RESOLVED
1266141 All remaining issues on Shadowsong computer RESOLVED
1266186 All remaining issues on Anetheron RESOLVED
1266208 Minor video/sound sync with Twitch at high quality on Ming computer. RESOLVED
1266246 All issues on the Centurion computer RESOLVED
1266482 All issus with Frostwolf computer RESOLVED
1266504 All issues with Silvermoon computer RESOLVED
1266560 Youtube 360 has some choppiness with hardware acceleration off and no 4k option with it on on Dradis computer. RESOLVED
1266580 All problems on Palpatine computer RESOLVED
1266787 All issues on Warlock computer RESOLVED
1266814 All issues on Onyxia computer RESOLVED
1266861 All issues on Dragonmaw computer RESOLVED
1266881 All issues on Frostmane computer RESOLVED
1266902 All issues on Nefarian computer RESOLVED
1267304 All issues on Marwyn computer RESOLVED
1267341 All issues on Eomer computer RESOLVED
1267366 All issues on Skullcrusher computer RESOLVED
1267424 All issues on Kal-El computer RESOLVED
1267478 All issues on Horridon computer RESOLVED
1267667 Choppy video on youtube RESOLVED
1267668 Choppy video on Youtube 360 RESOLVED
1267669 Choppy video on Vimeo UNCONFIRMED
1267672 Choppy video on Facebook UNCONFIRMED
1267673 Choppy video on DuckDuckGo UNCONFIRMED
1267674 Choppy video on Reddit UNCONFIRMED
1267676 Choppy video on Steam UNCONFIRMED
1267677 Choppy video on Twitch UNCONFIRMED
1267679 Choppy video on ABC News UNCONFIRMED
1267682 Choppy video on NFL.com UNCONFIRMED
1267700 360 controls on Youtube 360 don't load RESOLVED
1267714 Facebook HD video won't load RESOLVED
1267723 Audio/video sync issues on NFL.com RESOLVED
1267726 Audio/video sync issues on Twitch RESOLVED
1267728 Audio/video sync issues on ABC News RESOLVED
1269538 Vimeo not loading video on Talon RESOLVED
1269541 Some videos on Talon system require Flash to play RESOLVED

99 Total; 12 Open (12.12%); 83 Resolved (83.84%); 4 Verified (4.04%);

Triage

Triage currently focuses on two main areas: identifying critical stability regressions, and identifying issues which may have slipped through the cracks. We try to have a one hour triage session each weekday, usually starting around 10:00am Pacific. More information about this process can be found on the wiki and dashboard.

Retired Activities

Top Issues

Blockers

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);


Crashes

Full Query
ID Summary Status Last change time
1218607 [meta] Firefox on android crashing in [@ libGLESv2_adreno.so@ ...] NEW 2023-12-21T09:26:05Z
1535120 Crash in [@ __pthread_kill | abort | gpusGenerateCrashLog] REOPENED 2023-09-27T12:19:13Z
1605665 operator delete[] crashes on MemoryTextureHost NEW 2024-03-11T12:20:56Z
1711801 Crashes at gpusGenerateCrashLog with graphics kernel error '0xfffffff9/-7' REOPENED 2023-09-27T12:19:17Z
1711802 Crashes at gpusGenerateCrashLog with graphics kernel error '0xfffffffc/-4' REOPENED 2023-09-27T12:19:11Z
1711804 Crashes at gpusGenerateCrashLog with graphics kernel error '0xe00002c2' REOPENED 2023-09-27T12:19:06Z
1713230 Crashes at gpusGenerateCrashLog with customized graphics kernel errors '0x1be385f9' and '0x067900fc' REOPENED 2023-09-27T12:19:09Z
1714235 Crashes at gpusGenerateCrashLog with graphics hardware errors '0x00000813', '0x00000803', '0x00008013', '0x00000003' and '0x00008003' REOPENED 2023-09-27T12:19:16Z
1725571 Windows 7 x86 Crash in [@ OOM | large | mozalloc_abort | xul.dll | _PR_NativeRunThread | pr_root] NEW 2023-09-18T12:15:08Z
1737467 Startup Crash in [@ logging::LogMessage::~LogMessage] via mozilla::SandboxBroker::LaunchApp NEW 2024-03-07T18:02:58Z
1783212 macOS Crash in [@ pthread_kill | abort | gpusKillClientExt] NEW 2024-02-12T13:13:49Z
1806388 Thunderbird Crash in [@ OOM | large | mozalloc_abort | xul.dll | webrender::renderer::Renderer::render_impl] NEW 2024-03-24T21:08:12Z
1838649 Crashes often when scrolling on gitlab merge request NEW 2023-09-27T12:19:04Z

13 Total; 13 Open (100%); 0 Resolved (0%); 0 Verified (0%);


Features

Full Query
ID Summary Status Last change time
703217 Implement image() from CSS Image Values and Replaced Content Module Level 4 NEW 2022-10-11T00:04:16Z
790640 add Media Fragments for images (spatial dimensions) NEW 2023-02-16T08:13:30Z
952539 WebGL should support efficiently uploading a subrect of a canvas to a texture NEW 2022-10-11T15:15:54Z
1152974 Enable by default DEAA (Anti-aliasing) for Linux NEW 2023-08-11T18:19:59Z
1200864 Skip DrawElements buffer validation when we have robust_buffer_access NEW 2023-03-24T17:36:32Z
1223880 Implement initial-letter (from CSS Inline Layout Module Level 3) NEW 2023-05-05T23:30:43Z
1233569 Deprecate RGB8 from being renderable in WebGL 1 NEW 2022-10-11T17:53:19Z
1237489 [webvr] Add basic MOZ_texture_from_element to enable creating a texture from an arbitrary element NEW 2022-10-11T17:55:16Z
1244513 webm videos should be color corrected UNCONFIRMED 2022-10-11T17:59:31Z
1264543 [meta] Out-of-process compositing NEW 2023-06-15T10:49:09Z
1282074 Support very large canvases NEW 2024-02-14T05:08:37Z
1302685 [meta] Support OpenType Font Variations (Variable fonts) NEW 2023-01-10T15:19:10Z
1309309 Report memory used by ANGLE in about:memory as its own block? NEW 2022-10-11T18:39:33Z
1350292 Memory allocated by WebGL in WebGLBuffer::BufferData is not tracked in about:memory (detected by DMD in heap-unclassified) NEW 2024-03-05T08:06:06Z
1367003 Page slow to load when scrolling up and down. NEW 2022-10-11T19:19:15Z
1371101 Run color, border-color, and background-color animations off the main thread NEW 2022-10-11T19:21:49Z
1371108 WebGL: Make default back buffer default to 8x8 AA when available UNCONFIRMED 2022-10-11T19:21:50Z
1371627 GL_EXT_draw_buffers extension not supported UNCONFIRMED 2022-10-11T19:22:13Z
1371836 Provide a performance object for the WebGL canvas NEW 2022-10-11T19:22:42Z
1374710 GLBlitHelper should cache the various shaders NEW 2023-03-24T17:33:23Z
1381098 imageBitmap should provide an api to pull out raw pixel data (unmodified) without having to draw it to a canvas or a renderBuffer. This should be able to run in a webworker UNCONFIRMED 2023-08-07T12:24:48Z
1391345 Some emoji on the emojione demo page don't show up NEW 2022-10-11T19:36:02Z
1392080 Feature request : ɴᴘᴀᴘɪ being phased out : please provide an alternative for direct Vulkan/Direct3ᴅ12 access for viewing stereoscopic videos ! UNCONFIRMED 2023-07-27T18:52:44Z
1402293 Implement support for High Efficiency Image File Format (HEIF) NEW 2024-03-13T19:31:07Z
1403299 Support rendering fonts via FreeType on Windows NEW 2022-10-11T19:44:53Z
1403933 Support standard swizzle in D3D11 texture NEW 2022-10-11T19:45:25Z
1412350 Support Color Management on Android NEW 2023-10-25T12:40:48Z
1428954 Support encoding .ico files with multiple sizes NEW 2022-10-11T21:09:36Z
1443863 Implement support for AV1 Still Image File Format (AVIF) NEW 2024-02-11T22:56:37Z
1461360 [meta] Cross-platform dependencies for pinch-zooming on desktop NEW 2023-04-23T08:13:11Z
1476861 Implement FillRoundedRect for DrawTargetSkia ASSIGNED 2022-10-11T21:48:00Z
1477078 Implement PushClipRoundedRect for DrawTargetSkia ASSIGNED 2022-10-11T21:48:11Z
1480894 Implement wide-gamut support on Canvas NEW 2022-10-11T21:51:27Z
1516362 Selecting text using background-clip: text should probably change the text color NEW 2022-10-11T22:20:40Z
1521723 Implement the hyphenate-limit-chars property NEW 2022-10-22T20:46:01Z
1522397 Displayed content in active tab doesn't adapt to screen resolution changes when connecting/disconnecting external monitor NEW 2023-05-24T13:32:17Z
1532721 Re-enable D2D on Qualcomm devices when able NEW 2022-10-11T22:34:04Z
1536809 Add support for Canvas 2D "desynchronized" construction attribute UNCONFIRMED 2023-10-24T16:26:22Z
1539075 Implement support for JPEG XL (image/jxl) ASSIGNED 2024-03-28T05:17:06Z
1547612 Use Lanczos method for upscaling images for different zoom levels NEW 2023-01-14T09:05:28Z
1605234 [meta] Support elastic overscroll on desktop NEW 2023-05-10T17:29:46Z

41 Total; 41 Open (100%); 0 Resolved (0%); 0 Verified (0%);


New Issues

Full Query
ID Summary Last change time
1887212 `user-select: none` ignored when hidden element becomes visible 2024-03-23T04:12:00Z
1887301 Graph rendering is broken (gets fixed by disabling accelerated canvas) 2024-03-25T17:40:30Z
1887315 [driver bug] webrender blitting into backbuffer broken on adreno a3xx using Freedreno 2024-03-27T03:45:14Z
1887337 WebRender broken on Android Emulator when 'use_optimized_shaders' is false 2024-03-26T08:39:20Z
1887435 pointerup event isPrimary always true for touch 2024-03-29T02:30:44Z
1887562 Canvas flickering in 125.0b3 2024-03-27T19:45:54Z
1887575 font-display swap uses wrap fallback, ignoring local fonts 2024-03-26T01:49:23Z
1887577 credential.create rejecting rpId when the document is XML encoded 2024-03-25T12:15:07Z
1887847 Keyboard input very laggy on serveral sites 2024-03-29T00:52:27Z
1888009 Drag and Drop between Two Firefox Profile Instances Consistient Failure 2024-03-27T14:38:31Z
1888031 Colours of visited links not shown in Firefox 124 2024-03-28T04:22:15Z
1888396 Pie charts broken on Solaris SPARC 2024-03-28T15:56:33Z
1888449 Text wrapping with text-wrap: balance + float 2024-03-28T16:26:57Z
1888602 Image corrupt or truncated 1024 x 100247 png doesn't display at full size 2024-03-29T05:06:57Z

14 Total; 14 Open (100%); 0 Resolved (0%); 0 Verified (0%);


Understanding the Problem Space

First order of business for my transition to the Graphics team is to understand the problem space so I can understand the immediate needs of the team and make the best impact I can in the shortest amount of time.

What are the key problems/challenges facing the Graphics team in terms of quality?
  • discrepancy in environments between testers and release users
  • discoverability of bugs pre-release
  •  ?...
Where can QA add value/support to the Graphics team?
  • improving pre-release discoverability of bugs
  • closing the gap between tester and release systems
  • helping with bug triage, particularly with bugs hiding in general components
  • representation in crashkill
  • improving code coverage and/or identifying gaps in code coverage
  • identifying ways to improve participation in the graphics team (events, projects, One & Done, etc)
  • documentation of tools, testing processes, etc
  • building out the lab in Toronto
  • continuing to drive Betabreakers testing every 6 weeks
  • verifying bug fixes (what does this look like)?
  • profiling areas of risk (eg. troublesome configs)
  • conducting root cause analysis for regressions
  • understanding problems outside of our control (eg. driver resets)
  • feature testing and upcoming priorities (e10s, Windows 10, El Capitain, Android, B2G, etc)
What does QA need to know to be effective?
  • key components of an actionable Graphics bug
  • fundamentals/technologies that should be learned
  • how to distinguish a graphics crash from a non-graphics crash with a graphics signature
  • meetings, mailing lists, bugzilla components to watch, blogs, IRC channels to join, etc
  • who is each member of the team (incl. contributors) and what do they do
  • where does graphics code reside in the tree?
  • what role does Unified Telemetry in graphics quality?
  • what are the prefs to enable/disable different functionalities?
  • we need a database of known-troublesome hardware/driver configurations to inform testing, hardware acquisitions, and blocklisting
Understanding the Stability
  • How do we identify a graphics crash?
    • by signature: gfx, layers, D2D, D3D, ?...
    • by topmost filename: gfx, ?...
    • by driver (DLL, version, ?...)
    • by device/vendor ID?...
    •  ?...
  • How do we prioritize graphics crashes?
    • Overall topcrashes in release > beta > aurora > nightly
    • Gfx crashes in release > beta > aurora > nightly
    • Explosive crashes in release > beta > aurora > nightly
  • What tools do we have at our disposal to investigate crashes?
    • Bughunter for investigating crashes correlated to a URL
    • KaiRo's reports for identifying crashes that are new or escalating quickly
    • Socorro for getting detailed information about crash reports
  • What information is needed to make a crash actionable by developers?
    • Correlations to particular hardware, driver, add-on, 3rd-party software, or library
    •  ?...
Participation
  • Sanity checking via One & Done
  • Meetups to connect testers/users with devs
  • Testdays to teach people about graphics testing
  • Documentation and translation of documentation
  • Engaging on community spaces (Discourse, Reddit, Facebook, Twitter, etc)
Telemetry
  • COMPOSITE_TIME: time in CompositorParent::CompositeToTarget dispatching draw calls and calling SwapBuffers, but not texture upload (ie. complete composition)