Mobile/Testing/04 30 14

Previous Action Items

• gbrown to file a bug for the Android 2.3 R5 crashes - bug 1000538
• kmoir will check into ix capacity for Android 2.3 - yes, I think we have sufficient capacity

Status reports

A Team

• Android 2.2 failure rate: [3.06%]
  • last week 4.06%
• Android 2.3 failure rate: [0.95%]
  • last week 3.52%
• Android 4.0 failure rate: [17.98%]
  • last week 17.37%
  • excluding retries 2.46%

Android 2.3 emulator

• R5 crashes - bug 1000538 - may have self-healed

Autophone

• Work progressing on bug 990601 to use adb instead of the SUTAgent.
  • Issues identified
    • Previously with the SUTAgent, I used the logcat entry for SUTAgent's command to start fennec to identify the start time. Without the SUTAgent, there is no clear indicator in the log which I can use to detect the start time. I am using the first mention of Gecko or fennec in logcat after starting fennec to determine the start time. If you have a better approach, I would very much appreciate hearing about it.
    • Some issues with devicemanagerADB hung commands. Am working on a short term replacement for autophone before incorporating the changes into mozbase's devicemanagerADB.
    • We require the device to boot to an unlocked screen in order to run tests. On Nexus 4 and later I can set the device to not lock on boot but for older devices such as Nexus One and Nexus 2, I need to use something to automatically unlock the device.
  • Test results
    • Have moved the -2 nexus devices from production testing to development testing.
    • Ran a test run with the SUTAgent and the Watcher installed on all phones and another with only the Watcher installed on Nexus One and Nexus S phones
    • Most recent test results for the nexus devices using adb can be found at for April 27: http://phonedash-dev.allizom.org/#/org.mozilla.fennec/throbberstart/local-blank/norejected/2014-04-27/2014-04-27/notcached/noerrorbars/standarderror
    • The presence of the SUTAgent does have an affect on the measurements though it is not consistent across devices.
  • Next steps
    • Will finish the adb command replacements today.
    • Will submit patch for comment and review by tomorrow.
    • Would like to move all phones to the new adb based code for testing.
    • Need to finalize code in preparation for shipping the devices to Mountain View for installation in the QA lab during the week of May 12.

Eideticker

• Dashboard still down, save a brief testrun on Friday. :( Ports issue resolved (bug 988606) but devices keep on falling off network. Netops investigating in bug 1003507