|
|
| (18 intermediate revisions by the same user not shown) |
| Line 1: |
Line 1: |
| =Overview= | | =Overview= |
| Block Playback
| | Videos which autoplay in the background will now have their load deferred until the tab is visible for the first time -- this avoids autoplay during session restore and premature playback. This means no more "Where's that sound coming from?" moments when an ad for instance decides to autoplay in a tab you've specifically opened in the background. |
| Video/Audio will be blocked if a new tab been opened but never bring to front.
| | Resources will still be preloaded if indicated but Firefox will delay the start of playback until you actually visit the tab. Once a tab / RenderFrame has ever played media before, it's allowed to continue to autoplay/autoload indefinitely; this is to support playlist type applications. This feature prevents obviously user annoyance but also conserves power as Firefox will only consume power once the tab is foregrounded. |
|
| |
|
| | | ==Issue== |
| ===Meta Bug===
| | This mechanism will also block notification sounds from websites such as Facebook or Gmail if a user open the tab but haven’t visited it yet. There are also users who want to open new tab for music without needing to visit the tab. The mechanism will force users to visit the tab for the music to start playing. |
| * [https://bugzilla.mozilla.org/show_bug.cgi?id=1262053 Bug 1262053]
| |
|
| |
|
| ===Other Resources=== | | ===Other Resources=== |
| * [https://goo.gl/0dylCY Meeting Minute] | | * [https://goo.gl/0dylCY Meeting Minute] |
|
| |
|
| | =='''MVP Scope'''== |
| | Meta Bug: [https://bugzilla.mozilla.org/show_bug.cgi?id=1308154 1308154 - (block-autoplay-media) [meta<nowiki>]</nowiki> Block autoplay media until the tab is visible at first time] |
|
| |
|
| =Known Issues=
| |
|
| |
|
| |
|
| |
| ''Related telemetries'':
| |
| * [https://telemetry.mozilla.org/new-pipeline/dist.html#!cumulative=0&end_date=2016-08-28&keys=All!AV%252C241-480!V%252C241-480!AV%252C481-720&max_channel_version=nightly%252F51&measure=VIDEO_SUSPEND_RECOVERY_TIME_MS&min_channel_version=null&product=Firefox&sanitize=1&sort_keys=submissions&start_date=2016-08-18&table=0&trim=1&use_submission_date=0 VIDEO_SUSPENDED_RECOVERY_TIME_MS]
| |
|
| |
| ''Related bugs'':
| |
| * [https://bugzil.la/1272919 Bug 1272919 - Video shows the frame that was on screen when I switched from youtube to a different tab for a fraction of a second]
| |
|
| |
| ===Video element as content source===
| |
| The situation is about the video element itself is used as a content source and its decoder might be suspended at the same time. This issue could be split into two cases:
| |
| * Case 1: The decoder is suspended '''AFTER''' the {cpatureStream(), drawImage(), createImageBitmap()} is called.
| |
| * Case 2: The decoder is suspended '''BEFORE''' the {cpatureStream(), drawImage(), createImageBitmap()} is called.
| |
|
| |
| '''Case 1''' is relatively easy, we could mark the video element as being a content source while {cpatureStream(), drawImage(), createImageBitmap()} is invoked and then its decoder should never be suspended.
| |
|
| |
|
| '''Case 2''' is rather complicated. The critical issue is that resuming decoder is an async operation with latency. So, while {cpatureStream(), drawImage(), createImageBitmap()} is invoked on an already-suspended-video, there must be several "blank" frames been leaked, even though we resume the decoder immediately. To completely solve this problem, we must make "resuming decoder" a blocking operation, however, it might block the main thread; otherwise, we leak the blank frames.
| | <bugzilla> |
| | { |
| | "blocks":"1308154", |
| | "status":["NEW","REOPENED","UNCONFIRMED","ASSIGNED","RESOLVED","VERIFIED","CLOSED"], |
| | "include_fields": "id, summary, status, target_milestone, resolution, assigned_to, depends_on, blocks, whiteboard" |
| | } |
| | </bugzilla> |
|
| |
|
| | | = Schedule = |
| ''Related bugs'':
| | Targeting: '''Firefox 52''' |
| * [https://bugzilla.mozilla.org/show_bug.cgi?id=1284389 Bug 1284389 - Don't suspend video elements captured via mozCaptureStream()]
| |
| * [https://bugzilla.mozilla.org/show_bug.cgi?id=1295921 Bug 1295921 - Don't suspend video decoder for elements as sources to drawImage() and createImageBitmap()]
| |
| | |
| =Planning=
| |
| ==Goals==
| |
| * Help users to reduce device resource usage (CPU & Memory)
| |
| * Finish tasks across videos
| |
| * No impact on current user flow
| |
| | |
| ==Feature Plan==
| |
| * Phase 0: Shutdown decoders when video element is invisible
| |
| * Phase 1: Using a blank video decoder to replace video decoder instead of shutdown decoders directly. In this phase, the mechanism is applied to 1) Low-resolution video (480P) and 2) Video film without audio track
| |
| * Phase 2: We're going to enhance the mechanism and make sure it can apply to all video files.
| |
| | |
| ==Success==
| |
| * Effective cross-discipline teams solving problems across platforms
| |
| * Validation of key assumptions through telemetrics
| |
| * video decode suspend in the hands of all our users
| |
| | |
| ==Schedule==
| |
| | |
| ===Product Milestones===
| |
| | |
| | |
| = Engineering =
| |
| Block Playback [status: <span style="color:#0f0">Green</span>]
| |
| | |
| Current Goals:
| |
| * Ship disabling video decoders for ads
| |
| * Silent, looping videos where user doesn't really care the exact point
| |
| | |
| Next Milestone:
| |
| | |
| | |
| | |
| == Milestones ==
| |
| === Optimizations ===
| |
| * [https://bugzil.la/1282710 Bug 1282710 - Suspend and resume foreground video decoders according to visibility events]
| |
| ** done.
| |
| * [https://bugzil.la/1282012 Bug 1282012 - Seek to nearest keyframe when resuming videos with no audio]
| |
| ** [https://bugzil.la/1294657 Bug 1294657 - Seek to nearest keyframe when resuming videos with no audio - with audio track but muted]
| |
| *** pending, excluded from phase 1.
| |
| ** [https://bugzil.la/1294658 Bug 1294658 - Seek to nearest keyframe when resuming videos with no audio - with audio track but might be silent]
| |
| *** pending, excluded from phase 1.
| |
| ** [https://bugzil.la/1294656 Bug 1294656 - Seek to nearest keyframe when resuming videos with no audio - no audio track]
| |
| *** done.
| |
| * [https://bugzil.la/1274919 Bug 1274919 - Resume suspended video decoders on tab mouse hover.]
| |
| ** under review.
| |
| * [https://bugzil.la/1284389 Bug 1284389 - Don't suspend video elements captured via mozCaptureStream()]
| |
| ** WIP.
| |
| * [https://bugzil.la/1295921 Bug 1295921 - Don't suspend video decoder for elements as sources to drawImage() and createImageBitmap()]
| |
| ** WIP.
| |
| * Resume video when keyboard is used to change tags - Needs Bug
| |
| ** For example, if it's possible to detect the direction of cycling and videos are within, say, 5 tabs of the current tab, then start decoding again.
| |
| ** Need to hook into the tab navigation/change code and alert media elements, like bug 1274919
| |
| | |
| === Telemetry ===
| |
| * '''Amount of time hidden''' - measure of user value (Bucket results by resolution; i.e. are 720p videos hidden less often?)
| |
| ** [http://bugzil.la/1285419 Bug 1285419 - Telemetry to support background video decoder suspend: Hidden play time]
| |
| ** [https://mzl.la/2c0493i VIDEO_HIDDEN_PLAY_TIME_MS]
| |
| ** [http://bugzil.la/1287987 Bug 1287987 - Telemetry to support background video decoder suspend: Percentage hidden/total play time, keyed by audio presence and height ranges]
| |
| ** [https://mzl.la/2c04hQl VIDEO_HIDDEN_PLAY_TIME_PERCENTAGE]
| |
| ** [http://bugzil.la/1293145 Bug 1293145 - Telemetry to support background video decoder suspend: Percentage video-decode-suspended/total play time]
| |
| ** [https://mzl.la/2c07M9s VIDEO_INFERRED_DECODE_SUSPEND_PERCENTAGE]
| |
| | |
| * '''Recovery time''' - measure of user cost (separate for noisy vs silent videos) (Bucket results by resolution; i.e. do 720p videos take longer to recover?)
| |
| ** [https://bugzil.la/1294349 Bug 1294349 - Telemetry to support background video decoder suspend: Recovery time from video-decode-suspended]
| |
| ** [https://mzl.la/2c947Iv VIDEO_SUSPEND_RECOVERY_TIME_MS]
| |
| | |
| * '''Key frame spacing''' - distribution allows better tuning
| |
| ** [http://bugzil.la/1289668 Bug 1289668 - Telemetry to support background video decoder suspend: Inter-keyframe timings]
| |
| ** [https://mzl.la/2c04Utr VIDEO_INTER_KEYFRAME_AVERAGE_MS]
| |
| ** [https://mzl.la/2c956bN VIDEO_INTER_KEYFRAME_AVERAGE_PERCENTAGE]
| |
| | |
| === Mochitests ===
| |
| * [https://bugzil.la/1284177 Bug 1284177 - Add tests for video suspend in background]
| |
| ** done
| |
| * [https://bugzil.la/1294358 Bug 1294358 - Add test for suspended videos still fire 'ended' event]
| |
| ** done
| |
| * [https://bugzil.la/1295844 Bug 1295844 - Test suspended videos with webm files]
| |
| ** done
| |
|
| |
|
| =Team= | | =Team= |
|
| |
|
| Product owner:
| | Eng: Alastor, JW, Kaku |
| | |
| Eng: Alastor, Daniel, Gerald, JW, Kaku, | |
|
| |
|
| Program Management: Blake, Josh | | Program Management: Blake, Josh |
| Line 124: |
Line 32: |
| UX: Mark, Morpheus | | UX: Mark, Morpheus |
|
| |
|
| QA: SoftVision and William | | QA: SoftVision |
| | |
| =Communications=
| |
| | |
| IRC:
| |
| | |
| Email:
| |
| | |
| VidyoRoom:
| |