QA/Webm eme support widevine
WebM EME support for Widevine
This section describes the modifications that have been made to this wiki page. A new row has been completed each time the content of this document is updated (small corrections for typographical errors do not need to be recorded). The description of the modification contains the differences from the prior version, in terms of what sections were updated and to what extent.
|2016-08-08||1.0||Ciprian Georgiu||Created first draft|
- 1 Overview
- 2 Testing summary
- 3 Requirements for testing
- 4 Test Strategy
- 5 Status
- 6 References
- 7 Test cases
- 8 Bug Work
- 9 Sign off
Detail the purpose of this document. For example:
- The test scope, focus areas and objectives
- The test responsibilities
- The test strategy for the levels and types of test for this release
- The entry and exit criteria
- The basis of the test estimates
- Any risks, issues, assumptions and test dependencies
- The test schedule and major milestones
- The test deliverables
This wiki details the testing that will be performed by the project team for the WebM EME support for Widevine project. It defines the overall testing requirements and provides an integrated view of the project test activities. Its purpose is to document:
- what will be tested,
- how testing will be performed.
- Florin Mezei (:FlorinMezei) - Project Manager of the Release QA Team
- Ciprian Georgiu (:ciprian.georgiu) - leading QA efforts
- Iulia Cristescu (:IuliaC) - QA
Scope of Testing
Testing efforts will be focused on confirming that WebM EME is used by default for eligible video content that makes use of the Widevine DRM.
Out of Scope
Functionality of the Widevine Media Optimizer plugin with all available features is not included in the testing efforts. Only Firefox support for the WebM EME Widevine feature is tested.
Requirements for testing
Hollywood studios require companies that license movies for streaming use DRM between the streaming company and the end user. On the Web, this has traditionally been done by using the Microsoft PlayReady DRM component inside the Silverlight plug-in or the Adobe Access DRM component inside the Flash Player. As the Web platform gains more capabilities, general purpose plug-ins like Silverlight or Flash can be phased out.
The Widevine Media Optimizer is a browser plugin designed for the viewing of premium video content. The Widevine plugin is commonly used by premium content providers.
Full Testing will be performed on:
Mac OS X 10.6 or later
Exploratory/Smoke testing will be performed on:
Which streaming services use which DRM on which platforms?
|Service||Windows 7–10||Windows XP||Mac OS X||Linux|
|Amazon Video||Widevine CDM||Silverlight||Widevine CDM|| Not supported yet
(Widevine CDM on Chrome)
|Netflix||Adobe CDM||Silverlight||Silverlight|| Not supported yet
(Widevine CDM on Chrome)
|YouTube||Widevine CDM||Flash Access||Widevine CDM|| Not supported yet
(Widevine CDM on Chrome)
Other configuration/environmental setup needed:
- media.mediasource.mp4.enabled - set to false
- media.mediasource.webm.enabled - set to true
The feature is available on Firefox 50 starting with 2016-07-31.
Test Execution Schedule
The following table identifies the anticipated testing period available for test execution.
|Project phase||Start Date||End Date|
|Study documentation/specs received from developers||2016-08-08||2016-08-09|
|QA - Test plan creation||2016-08-08||2016-08-10|
|QA - Test cases/Env preparation||2016-08-08||2016-08-08|
|QA - Nightly Testing||N/A||N/A|
|QA - Aurora Testing||2016-08-09||2016-09-14|
|QA - Beta Testing||2016-10-25||2016-10-28|
Detail the tools to be used for testing, for example see the following table:
|Test plan creation||Mozilla wiki|
|Test case creation||TestRail|
|Test case execution||TestRail|
- Landed in Nightly 50 on 2016-07-31.
- Landed in Aurora 50 on 2016-08-03.
- Landed in Beta 50 on 2016-09-20.
- Shipped with Firefox 50 on Tbd.
- Identify the high-risk assumptions.
- Identify existing bugs on the feature with high risk.
- Identify if other areas are affected by the fix.
- List and links for specs
- https://en.wikipedia.org/wiki/Encrypted_Media_Extensions - Encrypted Media Extensions
- Meta bug: Bug 1279077
The support for the QA/Webm EME feature was added in Nightly with Bug 1279077  which landed in 2016-07-31. First available in this build.
Track the dates and build number where feature was merged to Aurora.
Track the dates and build number where feature was merged to Release/Beta.
|Interraction (scroll, zoom)||Yes|
|Usability and/or discoverability testing||Yes||Is this feature user friendly|
|Does this affect partner builds? Partner build testing||Yes||yes/no options, add comment with details about who will lead testing|
|Web Compatibility||depends on the feature|
|Testing against target sites||Yes|
|Survey of many sites for compatibility||Yes|
|Interoperability||depends on the feature|
|Common protocol/data format with other software: specification available. Interop testing with other common clients or servers.||Yes|
|Coordinated testing/interop across the Firefox products: Desktop, Android, iOS|
|Interaction of this feature with other browser features||Yes|
Webm eme support widevine - Mid-Aurora/Pre-Beta Sign off (Fx50) testsuite in  (need Testrail account)
Tracking bug – meta bug.
Bug fix verification
|1298027||[EME] Video playback fails when page with EME video is moved into new window||RESOLVED||FIXED|
|1298030||[EME] Video is stuck on Youtube when is moved into New window||RESOLVED||DUPLICATE|
|1298031||[W10] WebM is not being used via the 'Stats for nerds' on YouTube||RESOLVED||INVALID|
|1301423||[EME] Manually forwarding the video to it's end, gives an error which is thrown on top of the video||NEW|
4 Total; 1 Open (25%); 3 Resolved (75%); 0 Verified (0%);
- All test cases should be executed
- Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan
- All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)
List of OSes that will be covered by testing: Windows 7, Windows 10, Ubuntu 16.04 LTS, Mac OS 10.11.
List of OSes that will be covered by testing: Windows 7, Windows 10, Ubuntu 16.04 LTS, macOS 10.12 Sierra (Beta).
|Testing Prerequisites (specs, use cases)||[DONE]|
|Testing Infrastructure setup||[DONE]|
|Test Plan Creation||[DONE]|
|Test Cases Creation||[DONE]|
|Full Functional Tests Execution||[DONE]|
|All Defects Logged||[DONE]|
|Critical/Blockers Fixed and Verified||[DONE]|
|QA Signoff - Nightly Release||MISSED|
|QA Aurora - Full Testing||[DONE]||(Yellow) Sign off email sent on 26/08/2016|
|QA Signoff - Aurora Release||[DONE]||(Green) Sign off email sent on 14/09/2016|
|QA Beta - Full Testing||[DONE]|
|QA Signoff - Beta Release||[DONE]||(Green) Sign off email sent on 01/11/2016|