Mobile/Testing/07 11 12

Previous Action Items

• joduinn & Asa to figure out when automated testing for metro will come online
• blassey to get assignees for bug 763166 and bug 749917
• blassey to test fennec on a beagleboard
• blassey and armen to test panda boards with the image ted just posted
• joel to file a bug for turning tcheck3 on and unhiding tcheck2
• armen to file the bugs needed to get panda boards into production

Status reports

IT

• Panda Chassis has been racked in SCL1 and is close being turned over for testing/staging bug 769429
• Panda boot loader has been scripted to pxe boot or fail to sdcard.
  • Next step is setting up linux kernel/initrd image and tftp server for re-imaging tools bug 764534
• IT is working on a more panda dense chassis revision

A Team

• Total failure rate is 33%.
  • This is mostly due to M1, M3, M8, RC, R2, R3.
  • It appears in most of these tests the tegras are rebooting a lot, working on making that workflow bubble up to tbpl and not turn the tests orange
  • also working on adding health-info to the log files so we can get more information about these reboots.

• Panda Boards
  • tsvg ran 10 times in a row
  • tcheckerboard (and other robocop based tests) are not running at all, needs further attention
  • reftest runs, but there are a lot of oddities
    • running r1 I get about 890 tests in and the test number counter resets and we skip to a different directory. Logging to the log file stops, but we continue to test (as per logcat). Then the panda hangs and adb nor sutagent are accessible. Power reset is the only option
    • running reftests in 10 chunks seem to allow runs to continue. This still encounters problems.
  • many times I do the intial connection to copy the profile over and the sutagent is dead before I even try to launch the browser. I can remedy via adb reboot most of the time

Eideticker (not idle ticker)

• Can now zoom in on regions of dashboard for easier comparisons
• Turned off XUL and stock android comparisons for LG G2X 2.2 on eideticker dashboard for now to save time generating results
  • QUESTION: Is there a benefit to continuing to get these? IMO (wlach) we have enough data on how we behave with these older browsers now.
• Now using a new method to synthesize events on Android called Orangutan, this will affect dashboard results: http://wrla.ch/blog/2012/07/the-evolution-of-simulating-events-in-eideticker-from-monkeys-to-orangutns/
  • QUESTION: Should we archive the old/historical results now?
• Current version of devicemanager + agent not quite eideticker compatible. Patches to fix that in flight, already applied on the dashboard server.
• Galaxy Nexus dashboard more-or-less ready to go, pending bug 759877
• Working on making dashboard somewhat more resilient to failure: bug 771375