Firefox/Shutdown Decoders
Contents
Overview
Background
Suspending video element's video decoder, when the video element is in background tabs or is invisible even in the foreground tab, is a Firefox feature that reduces CPU/GPU & memory usage.
The mechanism is that, when a video element is invisible, we replace its original video decoder with blank video decoder which only produces white frames with right resolution and right time information. The original video decoder is released and the black video decoder is light so that we reduce CPU/GPU & memory usage.
Trade-off
The trade-off is that, while the suspended-video-element is switched back to be visible again, we should resume its original video decoder. The resuming operation must be asynchronous and might be time-consuming which depends on the resolution of the video file and whether it contains audio tracks or not.
Working flow
The following is a step-by-step description of suspending decoder working flow.
- At the very beginning of the decoding framework, the raw media data is sent to demuxer.
- Demuxer helps to separate a combined signal, e.g., a streaming can be separated into audio data and video data by demuxer. After that, audio and video data are sent to audio and video decoder separately.
- In data decoding period, audio decoder keeps working as normal because a user may be listening to the music. But, Firefox uses a blank video decoder to replace current video decoder if the video element is invisible.
Overall Project Health
[GREEN]
Target Milestone
Firefox55
Engineer Owner
Tzuhao Kuo [:kaku]
QA Contact
Simona Badau <simona.marcu@softvision.ro>
Adrian Florinescu <adrian.florinescu@softvision.ro
MVP Scope-Bug Tracking
- bug 1293963 -[Meta] Suspend-video-decoder: phase-1 shipping
- tracking things only under the scope of phase-1 shipping.
17 Total; 1 Open (5.88%); 13 Resolved (76.47%); 3 Verified (17.65%);
- bug 1352007- [Meta] Suspend-video-decoder
- the highest level meta bug tracking all things related to the suspend-video-decoder feature.
18 Total; 9 Open (50%); 9 Resolved (50%); 0 Verified (0%);
- bug 1276556-[META] Tracking enable of background tab video decoder suspend
- tracking basic functionalities which are all completed
29 Total; 0 Open (0%); 26 Resolved (89.66%); 3 Verified (10.34%);
Signoff-Report issued on April 25 by Adrian Florinescu
Feature testing status: COMPLETED
Testing status: COMPLETED (100%). 83 passed (92%), 4 blocked (4%), 3 failed with known bugs (3%), 0 failed with new bugs (0%)
Overall feature status after testing: GREEN
Reason: 92% of our tests passed.
Recommendation from QE: SHIP IT
Reason: 1 existing issue was reopened during testing Bug1309494 Proposed course of action: let this feature ride Fx55.
QA Test Report
- Test Report: Test Report
Bugs tracking
| ID | Priority | Summary | Status | Resolution | Assigned to | Last change time |
|---|---|---|---|---|---|---|
| 1309494 | P3 | Use spinning loading UI if resuming video element's video decoder is longer than 250ms. | RESOLVED | FIXED | Tzuhao Kuo [:kaku] | 2017-05-17T09:08:05Z |
1 Total; 0 Open (0%); 1 Resolved (100%); 0 Verified (0%);
UX Spec
- UX Spec : UX Spec
Decoder resuming latency
While the suspended-video-element is switched back to be visible again, we should resume its original video decoder. The resuming operation must be asynchronous since we don't want to block the main thread and might be time-consuming which depends on the resolution of the video file and whether it contains audio tracks or not.
Currently, we have no way to boost the resuming time, however, we have telemetries for collecting the needed time of different resolutions on different platforms.
Power consumption (under experiment...)
Methodology
Used media file: https://drive.google.com/a/mozilla.com/file/d/0Bwk6-CqTcXSDRlYtTjVPZUpJRUU/view?usp=sharing
- mp4, H.264
- video only
- resolution: 1280 x 720
- duration is 2:4:22
Test scenario: use a local html file to play a local video file and put it into 4 cases.
- case 1: normal playback in foreground tab, full screen.
- case 2: normal playback if foreground tab with video decoder suspended, full screen.
- case 3: normal playback in background tab.
- case 4: normal playback if background tab with video decoder suspended.
Nexus 5
Normal playback in background tab: 3hr 59min 22sec.
Suspend playback in background tab: 6hr 10min 02sec.
Delta: 2hr 10min 40sec, 54%.
Nexus 6
Normal playback in background tab: 5hr 15min 19sec.
Suspend playback in background tab: 8hr 21min 37sec.
Delta: 3hr 6min 18sec, 59%.
