QA/Fennec/WebRTC/Dogfood: Difference between revisions

From MozillaWiki
< QA‎ | Fennec‎ | WebRTC
Jump to navigation Jump to search
 
(11 intermediate revisions by 3 users not shown)
Line 14: Line 14:


;Make the most of Mobility
;Make the most of Mobility
*We really need a variety of network configs. Conduct testing from your home office, your local coffee house, airport, pub, wherever. More is more.
*We are fairly confident about things working well with residential WiFi, we need a *variety* of network configs<br> **What happens in the SF office?
** What happens in the Portland office?
** What happens in the Toronto office?
**Hotel wifi networks
**Coffeehouse wifi networks
**Airport wifi networks
**Data plans (3G, 4G)


==Make the call!!==
==Make the call!!==
Line 26: Line 32:
* http://johntsai.me/chat/
* http://johntsai.me/chat/
* https://gittogether.com/
* https://gittogether.com/
==Capturing Logs==
Nightly
* adb shell am start -a android.activity.MAIN -n org.mozilla.fennec/.App --es env0 NSPR_LOG_MODULES=mediamanager:5,signaling:5,mediastreamgraph:6,mtransport:5
Beta
* adb shell am start -a android.activity.MAIN -n org.mozilla.firefox_beta/.App --es env0 NSPR_LOG_MODULES=mediamanager:5,signaling:5,mediastreamgraph:6,mtransport:5


==Test Results Log==
==Test Results Log==
Post your results here and file a bug with any issues http://mzl.la/157L9U8
;Post your results here and file a bug with any issues http://mzl.la/157L9U8
*Recording carrier-to-carrier combinations that work (or don't) is really helpful data
*If using WiFi, specifying your router model is also helpful (for both passing and failing cases)
*If you have a really crappy experience, try speedof.me
*If your connection completely fails, try desktop: you can use nightly, aurora or beta
*Considerations: How did the call establish? How was the video latency? Audio latency?
 


{| class="querytable"
{| class="querytable"
  | class="header" style="width:30%" | Caller
  | class="header" style="width:20%" | Caller Browser, Device
  | class="header" style="width:30%" | Callee
  | class="header" style="width:20%" | Callee Browser, Device
| class="header" style="width:10%" | Location Caller 1
| class="header" style="width:10%" | Location Caller 2
  | class="header" style="width:10%" | Call Length
  | class="header" style="width:10%" | Call Length
  | class="header" style="width:30%" | Notes  
  | class="header" style="width:30%" | Notes  
  |-
  |-
  | Firefox 24 (Android, Samsung Galaxy S4, Office WiFi)
  | Firefox 24 (Android, Samsung Galaxy S4)
  | Firefox 24 (Android, HTC One, Office WiFi)
  | Firefox 24 (Android, HTC One)
| Kind Grind, Santa Cruz
| Starbucks, San Diego
  | 3 min
  | 3 min
  | Caller: video was tiny on talky.io but and v were clear <br> Callee: Ditto, call was great until we tried to add a 3rd person
  | Caller: video was tiny on talky.io but and v were clear <br> Callee: Ditto, call was great until we tried to add a 3rd person
  |-
  |-
| Firefox 26 (Android, Galaxy Nexus)
| Firefox 26 (Android, Galaxy Nexus)
| SF office, SF
| SF office, SF
| 5 min
| On talky.io, experience was similar between caller and callee - video and audio were very delayed and fuzzy, essentially unusable. Perhaps worth noting that we were sharing the same WiFi network. Also, callee crashed (Bug 910964). Marginally better but still unusable experience on 24 (sans crash).
|-
|
|
|
  |
  |
  |  
  |  
| 
  | Caller: <br> Callee:  
  | Caller: <br> Callee:  
  |-
  |-
  |  
  |  
  |  
  |  
|
| Caller: <br> Callee:
|-
  |  
  |  
  |  
  |  
Line 59: Line 89:
  |  
  |  
  |
  |
|
|
  | Caller:  <br> Callee:  
  | Caller:  <br> Callee:  
  |-
  |-
|
|
  |  
  |  
  |  
  |  
Line 69: Line 103:
  |  
  |  
  |   
  |   
|
|
  | Caller:  <br> Callee:  
  | Caller:  <br> Callee:  
  |-
  |-
|
|
  |  
  |  
  |  
  |  
Line 79: Line 117:
  |  
  |  
  |   
  |   
|
|
  | Caller: <br> Callee:
  | Caller: <br> Callee:
  |-
  |-
|
|
  |  
  |  
  |  
  |  
Line 89: Line 131:
  |  
  |  
  |   
  |   
|
|
  | Caller:  <br> Callee:  
  | Caller:  <br> Callee:  
  |-
  |-
|
|
  |  
  |  
  |  
  |  
Line 96: Line 142:
  | Caller:  <br> Callee:  
  | Caller:  <br> Callee:  
  |-
  |-
|
|
  |  
  |  
  |  
  |  
Line 101: Line 149:
  | Caller:  <br> Callee:
  | Caller:  <br> Callee:
  |-
  |-
|
|
  |  
  |  
  |  
  |  
Line 106: Line 156:
  | Caller: <br> Callee:
  | Caller: <br> Callee:
  |-
  |-
|
|
  |  
  |  
  |  
  |  
Line 111: Line 163:
  | Caller: <br> Callee:
  | Caller: <br> Callee:
  |-
  |-
|
|
  |  
  |  
  |  
  |  
Line 116: Line 170:
  | Caller: <br> Callee:  
  | Caller: <br> Callee:  
  |-
  |-
|
|
  |  
  |  
  |  
  |  
Line 121: Line 177:
  | Caller: <br> Callee:  
  | Caller: <br> Callee:  
  |-
  |-
|
|
  |  
  |  
  |  
  |  
Line 126: Line 184:
  | Caller:  <br> Callee:
  | Caller:  <br> Callee:
  |-
  |-
|
|
  |  
  |  
  |  
  |  
Line 132: Line 192:
  |-
  |-
  |  
  |  
|
|
  |  
  |  
  |  
  |  
  | Caller:  <br> Callee:  
  | Caller:  <br> Callee:  
  |-
  |-
|
|
  |  
  |  
  |  
  |  
Line 141: Line 205:
  | Caller:  <br> Callee:  
  | Caller:  <br> Callee:  
  |-
  |-
|
|
  |  
  |  
  |  
  |  
Line 146: Line 212:
  | Caller: <br> Callee:  
  | Caller: <br> Callee:  
  |-
  |-
|
|
  |  
  |  
  |  
  |  
Line 152: Line 220:
  |-
  |-
  |  
  |  
|
|
  |  
  |  
  |  
  |  
Line 158: Line 228:
  |  
  |  
  |
  |
|
|
  |  
  |  
  | Caller: <br> Callee:
  | Caller: <br> Callee:
  |-
  |-
|
|
  |  
  |  
  |  
  |  

Latest revision as of 18:04, 30 August 2013

Intro

We plan to ship WebRTC support for Firefox for Android in version Fx24 which is currently in beta. Our target audience for this release are Developers and we want to make their experience with Firefox on Android is as stable and high fidelity as possible.

Set Up

The focus of dogfooding for FxA is Android to Android devices, we need to cover this matrix
  • FxA Beta 24 - FxA Beta 24
  • Chrome 29 - Chrome 29 <== why? because we really want to compare and contrast the experience with FxA
  • Chrome 29 - FxA Beta 24
Install the Firefox for Android build you intend to use for testing
Make the most of Mobility
  • We are fairly confident about things working well with residential WiFi, we need a *variety* of network configs
    **What happens in the SF office?
    • What happens in the Portland office?
    • What happens in the Toronto office?
    • Hotel wifi networks
    • Coffeehouse wifi networks
    • Airport wifi networks
    • Data plans (3G, 4G)

Make the call!!

Navigate to one of these pages with your browser to make Video/Audio Calls

Capturing Logs

Nightly

  • adb shell am start -a android.activity.MAIN -n org.mozilla.fennec/.App --es env0 NSPR_LOG_MODULES=mediamanager:5,signaling:5,mediastreamgraph:6,mtransport:5

Beta

  • adb shell am start -a android.activity.MAIN -n org.mozilla.firefox_beta/.App --es env0 NSPR_LOG_MODULES=mediamanager:5,signaling:5,mediastreamgraph:6,mtransport:5

Test Results Log

Post your results here and file a bug with any issues http://mzl.la/157L9U8
  • Recording carrier-to-carrier combinations that work (or don't) is really helpful data
  • If using WiFi, specifying your router model is also helpful (for both passing and failing cases)
  • If you have a really crappy experience, try speedof.me
  • If your connection completely fails, try desktop: you can use nightly, aurora or beta
  • Considerations: How did the call establish? How was the video latency? Audio latency?


Caller Browser, Device Callee Browser, Device Location Caller 1 Location Caller 2 Call Length Notes
Firefox 24 (Android, Samsung Galaxy S4) Firefox 24 (Android, HTC One) Kind Grind, Santa Cruz Starbucks, San Diego 3 min Caller: video was tiny on talky.io but and v were clear
Callee: Ditto, call was great until we tried to add a 3rd person
Firefox 26 (Android, Galaxy Nexus) Firefox 26 (Android, Galaxy Nexus) SF office, SF SF office, SF 5 min On talky.io, experience was similar between caller and callee - video and audio were very delayed and fuzzy, essentially unusable. Perhaps worth noting that we were sharing the same WiFi network. Also, callee crashed (Bug 910964). Marginally better but still unusable experience on 24 (sans crash).
Caller:
Callee:
Caller:
Callee:
Caller:
Callee:
Caller:
Caller:
Caller:
Callee:
Caller:
Callee:
Caller:
Callee:
Caller:
Callee:
Caller:
Callee:
Caller:
Callee:
Caller:
Callee:
Caller:
Callee:
Caller:
Callee:
Caller:
Callee:
Caller:
Callee:
Caller:
Callee:
Caller:
Callee:
Caller:
Callee:
Caller:
Callee:
Caller:
Callee:
Caller:
Callee:
Caller:
Callee:
Caller:
Callee:
Caller:
Callee:
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Fennec/WebRTC/Dogfood&oldid=701505"