|
|
| (36 intermediate revisions by 2 users not shown) |
| Line 30: |
Line 30: |
| Run the [http://mozilla.github.com/webrtc-landing WebRTC-Landing tests] with no failures. | | Run the [http://mozilla.github.com/webrtc-landing WebRTC-Landing tests] with no failures. |
|
| |
|
| == Other Tasks == | | == Known Issues == |
| === FIXED Bugs ===
| | * {{bug|936034}} - Sound is distorted on one side on 1:1 call |
| The following tracks verification of WebRTC bug fixes. As you verify a bug fix please list it below.
| |
| * [http://mzl.la/196DkC0 549 more bugs] | |
| | |
| === UNCONFIRMED Bugs ===
| |
| The following tracks confirmation of WebRTC bugs. As you confirm a bug exists please list it below.
| |
| * [http://mzl.la/14a3qyW 17 more bugs]
| |
| | |
| === WebRTC Health Report ===
| |
| The following tries to estimate the relative "health" of WebRTC based on bugs in bugzilla. Update this table each week with the number of bugs reported, resolved, and reopened.
| |
| {| class="querytable"
| |
| | class="header" | Period
| |
| | class="header" | Bugs Reported
| |
| | class="header" | Bugs Resolved
| |
| | class="header" | Bugs Reopened
| |
| |-
| |
| | 2013.07.28 - 2013.08.03
| |
| | 9
| |
| | 32
| |
| | 4
| |
| |-
| |
| | 2013.07-21 - 2013-07-27
| |
| | 14
| |
| | 17
| |
| | 3
| |
| |-
| |
| | 2013.07.14 - 2013.07.20
| |
| | 12
| |
| | 11
| |
| | 2
| |
| |-
| |
| | 2013.07.07 - 2013.07.13
| |
| | 16
| |
| | 16
| |
| | 2
| |
| |-
| |
| | 2013.06.30 - 2013.07.06
| |
| | 11
| |
| | 17
| |
| | 3
| |
| |}
| |
| | |
| == Testing ==
| |
| === AppRTC ===
| |
| * Demo Site: [http://apprtc.webrtc.org/ apprtc.webrtc.org]
| |
| * Test Methodology:
| |
| ** Running the demo 1:1 with no other processes running
| |
| ** Running the demo 1:1 with another vanilla Firefox process running
| |
| ** Running the demo 1:1 with another heavy Firefox process running (Flash, videos, games, gmail, etc)
| |
| ** Restart the caller browser and repeat the above
| |
| ** Restart the callee browser and repeat the above
| |
| * Expected Result:
| |
| ** A call lasting 5 minutes or longer should be clear and accurate (no delays, no artifacts, no crashes or hangs, no degradation in audio or video quality from the previous revision)
| |
| * Platforms:
| |
| ** P1: Windows XP, Windows 7, Windows 8, Mac OSX 10.8, Ubuntu 12.04, B2G
| |
| ** P2: Windows Vista, Mac OSX 10.7, Ubuntu 13.04, Android
| |
| * Builds:
| |
| ** P1: Firefox Nightly, Chrome Stable, B2G Browser
| |
| ** P2: Firefox Aurora/Beta/Release, Chrome Canary, Firefox Android (all branches)
| |
| | |
| ==== Results: YYYY.MM.DD ====
| |
| {| class="querytable"
| |
| | class="header" style="width:30%" | Caller
| |
| | class="header" style="width:30%" | Callee
| |
| | class="header" style="width:40%" | Result
| |
| |-
| |
| | Windows XP w/Firefox Nightly YYYY.MM.DD
| |
| | Windows 7 w/Firefox Nightly YYYY.MM.DD
| |
| | '''RESULT''': detailed notes
| |
| |-
| |
| |}
| |
| | |
| === Nightly ===
| |
| * '''Firefox 27.0a1''' - see [https://wiki.mozilla.org/Releases/Firefox_27/Test_Plan#WebRTC testplan]
| |
| | |
| === Aurora ===
| |
| * '''Firefox 26.0a2''' - see [https://wiki.mozilla.org/Releases/Firefox_26/Test_Plan#WebRTC testplan]
| |
| | |
| === Beta ===
| |
| * '''Firefox 24.0b4''': Tested for parity with Firefox 22.0 via AppRTC - see [https://wiki.mozilla.org/Releases/Firefox_24/Test_Plan#Regression_Tests_3 testplan]
| |
| | |
| === Release ===
| |
| * '''Firefox 23.0.1''': Tested for parity with Firefox 22.0 via AppRTC - see [https://wiki.mozilla.org/Releases/Firefox_23/Test_Plan#WebRTC testplan]
| |
| | |
| == Activity Log ==
| |
| * 2013.08.20: Firefox 24.0b4 signed-off for parity with Firefox 22.0 [[https://wiki.mozilla.org/Releases/Firefox_24/Test_Plan#Regression_Tests_3 details]]
| |
| * 2013.08.15: Firefox 23.0.1 signed-off for parity with Firefox 22.0 [[https://wiki.mozilla.org/Releases/Firefox_23/Test_Plan#WebRTC details]]
| |
|
| |
|
| == Meetings == | | == Meetings == |
| * [https://etherpad.mozilla.org/qa-webrtc-meeting-20130814 August 14, 2013]: Kick-off meeting with WebRTC Devs | | * [https://etherpad.mozilla.org/qa-webrtc-meeting-20130814 August 14, 2013]: Kick-off meeting with WebRTC Devs |
See: https://wiki.mozilla.org/Media/WebRTC/Testing for background on automated testing.
Basic Topology
The basic topology should include two separate machines on separate networks,
each with their own NAT. We have some flexibility about which NAT, but
for concreteness, I suggest a pair of Apple Airport Express NATs. I.e., the
following topology:
The following test series should be run, with one machine on each network with Nightly.
Endpoints
- Firefox Nightly <-> Firefox Nightly
- Firefox Nightly <-> Chrome Stable
- Firefox Nightly <-> Chrome Canary
Platforms
- Desktop (Windows, Mac, Linux)
- B2G (Unagi, Peak)
- Android
AppRTC
Conduct a call for no less than 5 minutes with apprtc resulting in no lag, no freezing, no audio or video delays, no degradation in quality from the previous revision.
WebRTC.ME
Conduct a call for no less than 1 minute using webrtcme resulting in minimal lag and freezing.
WebRTC Landing
Run the WebRTC-Landing tests with no failures.
Known Issues
- bug 936034 - Sound is distorted on one side on 1:1 call
Meetings
