QA/Desktop Firefox/WebRTC

See: https://wiki.mozilla.org/Media/WebRTC/Testing for background on automated testing.

Routine Manual Testing Tasks

The following tasks are essentially mechanical, but at least for now the need a person.

The basic topology should include two separate machines on separate networks, each with their own NAT. We have some flexibility about which NAT, but for concreteness, I suggest a pair of Apple Airport Express NATs. I.e., the following topology:

The following test series should be run, with one machine on each network with Nightly.

Full test series on AppRTC

• A call on apprtc between each pair of desktop endpoints
  • (9 calls in all)
• A call on apprtc between Firefox and Chrome (Stable + Canary) on any platform
  • (2 calls in all).
• Each call should last no less than 5 minutes with no major failures (failure to connect, sustained freezes, etc.)

Connect series on WebRTCMe

• A call on webrtcme between two Firefox endpoints (any platform)
• A call on webrtcme between Firefox and Chrome Stable (and platform)
• Each call should connect and work for at least one minute

B2G Testing

• A call on webrtcme between a B2G phone and any desktop OS for the following phones;
  • Unagi, Peak
• This test needs to be run on a home-built build with Steven's patch
• Each call should connect and work for at least one minute with no major glitches (some video lag and freezing is acceptable).

We need some Android tests, but I don't know what the status of Android is, so I'm reluctant to prescribe any.

• Additional tests:

 * WebRTC Landing tests  WebRTC-Landing on all platforms.

QA Tasks

• Daily verification of whatever fixes landed in that day's Nightly
• Weekly triage and attempted confirmation of unconfirmed/regression bugs
• Monthly re-evaluation of regression rate to inform adjustments to the above approach

Testing

• Manual
  • What must be done manually?
• Automated
  • What can be automated?