QA/Fennec/WebRTC/Top Site Testing
From MozillaWiki
Overview
This document provides an overview of how to do top site testing around WebRTC web applications.
Setup
- Install the Desktop Firefox build you intend to use for testing
- Install Chrome
- Install the Firefox for Android build you intend to use for testing
- Install Chrome Beta for Android
- Enable WebRTC in chrome://flags
- Ensure your testing machine has a webcam and a microphone
- Multiple distinct desktop and android devices are needed to complete the breadth of this testing
Top Site Tests
- Video/Audio Conferencing (1:1)
- Try to conduct a 1 to 1 call across two different machines and or devices on the same Wi-Fi network using the demos below on both device and machine. Then, try to talk to each other while you are in the call. How is the sound quality? How is the video quality?
- Try to conduct a 1 to 1 call across two different machines and or devices on different networks using the demos below on both device and machine. Then, try to talk to each other while you are in the call. How is the sound quality? How is the video quality?
- Try to conduct a 1 to 1 call across two different machines and or devices on different networks using the demos below on both device and machine. Then, kill the network connection on one of the machines. Does the call fail gracefully?
- Try to conduct a 1 to 1 call on the same machine on the same network using the demos below on both device and machine. Then, try to talk to each other while you are in the call. How is the sound quality? How is the video quality?8
- Try to conduct a 1 to 1 call on two different devices in locations far away from each other (e.g. different countries if possible) using the demos below. Then, try to talk to each other while you are in the call. How is the sound quality? How is the video quality?
- Try to conduct a 1 to 1 call on two different devices in locations far away from each other using the demos below. Then, let the call run for about five minutes. Next, try to talk to each other while you are in the call. How is the sound quality? How is the video quality?
- Try to conduct a 1 to 1 call on two different devices on the same network using the demos below. Then, end the call on one of the devices by closing the tab running the tokbox demo. Does the call fail gracefully?
- Try to do a 1 to 1 call on two different machines in locations far away from each other using the demo on Firefox Beta on one machine and Chrome on the second machine. Then, let the call run for about five minutes. next, try to talk to each other while you are in the call. How is the sound quality? How is the video quality?
- Try to do a 1 to 1 call across two different machines and or devices on different networks using the demos below on one device and or machine and Chrome on the other device and or machine. Then, kill the network connection on one of the devices. Does the call fail gracefully?
- Try to do a 1 to 1 call on the same device on the same network using the demos below a on one other device and Chrome on the device. Then, try to talk to each other while you are in the call. How is the sound quality? How is the video quality?
- Video/Audio Conferences (Multi-Person)
- Try to conduct a multi-person call with 3 people on three different devices and or machines on mixed networks using the demos below. Then, try to talk to each other while you are in the call. How is the sound quality? How is the video quality?
- Try to conduct a multi-person call with 4 people on four different devices and or machines on different networks using the demos below. Then, try to talk to each other while you are in the call. How is the sound quality? How is the video quality?
- Try to conduct a multi-person call with 4 people on four different devices and or machines on different networks using the demos below. Then, try to kill the network on two of the machines. Does the call continue for the two users who still have connectivity and end the call gracefully for the two users who left?
- Try to conduct a multi-person call with 4 people on four different devices and or machines on different networks using the demos below. Then, let the call run for five minutes. Next, try to talk to each other while you are in the call. How is the sound quality? How is the video quality?
- Try to conduct a multi-person call with 4 people on four different devices and or machines on different networks using the demos below on two machines and or devices and Chrome on two other machines and or devices. Then, let the call run for five minutes. Next, try to talk to each other while you are in the call. How is the sound quality? How is the video quality?
Note: WebRTC bugs found during running any of these tests should be filed here.
Top Sites Process
The goal for WebRTC top site testing is to find out if the site is a reliable WebRTC site to show off in Firefox or not using the target tests specified above. This means you need to analyze the following areas as part of running the above tests:
- What functionality works well in the above tests?
- What issues did you see while running the above tests?
- What bugs were filed when running the above tests?
The sites currently being tested as part of the top sites testing are the following:
