Auto-tools/Goals/2011Q4

• [ON TRACK] Achieve and maintain a 5% or less intermittent orange rate for Android automation (talos and tests) to achieve parity with desktop intermittent orange rates.
  • [ON TRACK] Land modifications to make it easier for developers to run Android talos (to make it easier to debug failures)
  • [ON TRACK] Create and land specialpowers-core JS module to simplify test utilities and reduce bugs in "alive test", talos, reftest, and mochi* test families

• [ON TRACK] Expand and enhance our test and performance automation to anticipate and address needs for Fennec and B2G.
  • [ON TRACK] Deliver video capture data from an alive test (Ts/Tp) an a "driver" test (Tuser) from eideticker to developers for analysis. Stretch: have system ready for duplication by end of Quarter
  • [ON TRACK] Deploy Talos RSS metrics in Android production
  • [ON TRACK] Land marionette with ability to "runscript" in B2G context and enough action support to run Tuser eideticker test.
  • [ON TRACK] Land enhancements to special powers to entirely remove enablePrivilege from all testing frameworks

• [ON TRACK] Deploy user responsiveness automation into production
  • [ON TRACK] Deploy Talos User Responsiveness test on desktop and android automation
  • [ON TRACK] Deploy Regression User Responsiveness (peptest) on desktop production automation

Supporting Projects

Many of our projects are inter-related, and there are many supporting projects that we want to call out as they are important and cannot be entirely neglected in favor of these goals. This section calls these out.

Bughunter

• Q4 Outcomes
  • [ON TRACK] UI Deployed, UI taken through find/fix cycle
  • [ON TRACK] Conduct demo/training for QA, Dev, and Product Mgmt on tool
  • [ON TRACK] Create plan for soccoro integration

• Community Involvement
  • Targeted involvement with other areas of Mozilla (see training outcome)

• Interconnections
  • Increasing quality and security for the gecko platform by providing good data on crashes.
  • Visualization technologies will be repurposed
  • Future: Android integration, re-using technology for "classify the web" effort

Pandaboards

• Q4 Outcomes
  • [ON TRACK] Create repeatable, deployable build/flash system to get a functioning board running
  • [ON TRACK] Analyze results from unit and performance tests on board, but not in production this quarter

• Community Involvement
  • Working with linaro, having presence in pandaboard community, publicizing the created scripts

• Interconnections
  • Aids with the "expand mobile test and performance automation" Q4 goal.

Mozbase

• Q4 Outcomes