QA/Desktop Firefox/WebRTC

From MozillaWiki
Jump to navigation Jump to search

See: https://wiki.mozilla.org/Media/WebRTC/Testing for background on automated testing.

Basic Topology

The basic topology should include two separate machines on separate networks, each with their own NAT. We have some flexibility about which NAT, but for concreteness, I suggest a pair of Apple Airport Express NATs. I.e., the following topology:

Topoplogy.png

The following test series should be run, with one machine on each network with Nightly.

Endpoints

  • Firefox Nightly <-> Firefox Nightly
  • Firefox Nightly <-> Chrome Stable
  • Firefox Nightly <-> Chrome Canary

Platforms

  • Desktop (Windows, Mac, Linux)
  • B2G (Unagi, Peak)
  • Android

AppRTC

Conduct a call for no less than 5 minutes with apprtc resulting in no lag, no freezing, no audio or video delays, no degradation in quality from the previous revision.

WebRTC.ME

Conduct a call for no less than 1 minute using webrtcme resulting in minimal lag and freezing.

WebRTC Landing

Run the WebRTC-Landing tests with no failures.

Testing

Weekly Sign-off

  • Test: 10 minute 1:1 call with no other process running using apprtc.webrtc.org
  • End-points:
    • Windows -> Windows
    • Windows -> Linux
    • Windows -> Mac OSX
    • Mac OSX -> Windows
    • Mac OSX -> Mac OSX
    • Mac OSX -> Linux
    • Linux -> Windows
    • Linux -> Mac OSX
    • Linux -> Linux
  • Test Methodology:
    • Running the demo 1:1 with no other processes running
    • Running the demo 1:1 with another vanilla Firefox process running
    • Running the demo 1:1 with another heavy Firefox process running (Flash, videos, games, gmail, etc)
    • Restart the caller browser and repeat the above
    • Restart the callee browser and repeat the above
  • Expected Result:
    • A call lasting 5 minutes or longer should be clear and accurate (no delays, no artifacts, no crashes or hangs, no degradation in audio or video quality from the previous revision)
  • Platforms:
    • P1: Windows XP, Windows 7, Windows 8, Mac OSX 10.8, Ubuntu 12.04, B2G
    • P2: Windows Vista, Mac OSX 10.7, Ubuntu 13.04, Android
  • Builds:
    • P1: Firefox Nightly, Chrome Stable, B2G Browser
    • P2: Firefox Aurora/Beta/Release, Chrome Canary, Firefox Android (all branches)

Bug Triage

FIXED Bugs

The following tracks verification of WebRTC bug fixes. As you verify a bug fix please list it below.

Verified Fixes

UNCONFIRMED Bugs

The following tracks confirmation of WebRTC unconfirmed bugs. As you confirm a bug exists please list it below.

Confirmed Bugs

Meetings

Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Desktop_Firefox/WebRTC&oldid=752097"