QA/Fennec/HardwareEncoderForWebRTC
From MozillaWiki
Overview
Purpose
The purpose of this wiki is to serve as a general test plan for verifying the progression and performance of the hardware encoder for WebRTC
Ownership
Developer contact: Munro Mengjue Chiang [:mchiang]
QA: Bogdan Surd [:bogdan]
Overall Status
[LANDED] Nightly 56: 16-06-2017
[NOT LANDED] Beta 55:
Testing Summary
Scope of testing
- The testing will be focus on
- Testing the performance of the app with the hardware encoder enabled
- No crashes are caused with the hardware encoder enabled in Firefox
Environments
Testing will be performed on following devices:
- Phones
- Android 4 -> Android 7
- Tablet
- Android 4 -> Android 7
Specifications
- In about:config set media.navigator.hardware.vp8_encode.acceleration_enabled & media.navigator.hardware.vp8_encode.acceleration_remote_enabled to true
- Trello Board: https://trello.com/c/7MeEQvrY/270-fennec-hw-encoder-for-webrtc-on-android
Acceptance Criteria
- Firefox does not crash
- Application performance is improved and device battery life is extended.
Testing details
Test Cases
Testing days
Date:2017-06-21
- Created Test Plan
- Partial Test Cases Created - WIP
Date:2017-06-23
Date:2017-06-26
- Website performance tests on various devices & retested video constraints
Date:2017-07-03
- Ran video constraints tests
- Testing Methods
- For Video Constraints:
- Connect phone via USB Debugging and set video constraints values
- In console use var senderStats = document.getElementById("senderStats"); to set values that will be returned when starting testing
- Start test by pressing Get Media > Select camera and microphone and press Share > press Connect > after pressing Connect use setTimeout(function(){console.log(senderStats.innerHTML);}, 59000); in the console window, get the required values returned in the console window.
- For secondary video tests
- Created a batch file to automatically take a screenshot after one minute using the command
- Set the required video resolution in the page and press run.
- Run batch file and wait for the screenshot > Copy values from the screenstot
- For Video Constraints:
Batch File |
@echo off
cls timeout /t 59 adb shell screencap -p "/sdcard/one.png" && adb pull "/sdcard/one.png" && adb shell rm "/sdcard/one.png" && one.png goto repeat
echo Repeat? pause goto begin
echo Start Timer? pause goto begin |
These testing methods were used in order to eliminate as much human error as possible.
Date:2017-07-05
Date:2017-07-28
- Ran WebRTC Samples tests
- Tested Popular websites
Date:2017-09-15
- Ran WebRTC Samples tests
- Tested Popular websites
Bug Work
Tracking bugs
- Meta bug
- bug 1265755 - [Firefox for Android] Utilize hardware encoders to save power consumption & improve performance
ID | Priority | Summary | Status | Assigned to |
---|---|---|---|---|
1257777 | P2 | Move Android audio/video decoder out of application process | RESOLVED | John Lin [:jhlin][:jolin] |
1354993 | P2 | [Firefox for Android] Add PictureID into VP8 CodecSpecificInfo | RESOLVED | |
1358030 | P1 | [Firefox for Android] add mochitest to test remote h/w encoder | RESOLVED | Munro Mengjue Chiang [:mchiang] Be Mozillian since 1/19/2018 |
1379926 | P1 | [HW Encoder] Media streaming not working on https://opentokrtc.com/ | VERIFIED | Munro Mengjue Chiang [:mchiang] Be Mozillian since 1/19/2018 |
4 Total; 0 Open (0%); 3 Resolved (75%); 1 Verified (25%);
Signoff
Criteria
- All test cases should be executed
- All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed
Results
Nightly testing
- N/A
Beta testing
- N/A