Auto-tools/Goals/2013Q2

Our Goal in Q2 is mostly ergonomics and stability. In other words:

• Goal: Improve Developer/QA ergonomics around running and debugging automation by:
  • [DEFER] Add debugger support to B2G testrunners
  • [DONE] Get Gaia unit and ui tests running in TBPL against B2G desktop builds
    • Note::Landed on Cedar, rolling out to other branches in Q3
  • [DONE] Improve patch review experience on bugzilla
    • A lot of little improvements to splinter. We are currently researching a full replacement of splinter and will start on

that next quarter.

• • [MISSED] Replace graph server with Datazilla for all desktop Talos tests
  • [DONE] Create a project branch view of B2G builds/tests that tracks commits per target, rather than per repo (all commits to any repo for unagi for instance) joint with releng

• Goal: Improve stability, reliability, and scalability of automation platforms by:
  • [DONE] Addressing all P1 B2G automation stability issues
  • [MISSED] Complete native REST interface for Bugzilla
  • [MISSED] Complete Mochitest to Mozbase port
  • [DROPPED] Create POC Microbenchmark suite to get feedback from developers for its usefulness - waiting on confirmation from perf team on this priority
    • [DONE] standup webpagetest.org on the stoneridge hardware to provide speed index metrics for the necko team. This replaces the microboenchmark goal as it is more important as per the perf team.
  • [DONE] Reduce orange factor to 5 or lower (for the last three weeks of the quarter) by driving awareness among developers, following up with assignees, and fixing intermittent automation failures.
  • [DONE] Update pulse so that its dependent (client side) libraries are current for its use in Tree Herder
  • [DONE] Create a prototype and a specification for a rig to power cycle real phones. Have a vendor selected to build the rig by end of quarter.

• Goal: Build toward our future by:
  • [DONE] Deploy a minimally viable TBPL replacement by end of quarter, aim to switch over mid-Q3
  • [DONE] Release a mozmill 2 and work with contributor community to drive toward a 2.1
  • [DONE] Automate a scrolling fps test using camera Eideticker measurements for Contacts, Email, Settings, and Message apps
  • [DONE] Create and Deploy BMO user profile data for community building and education
  • [DONE] Create a solid net and filter system to build a bugmaster community
  • [DONE] Identify 5 things we can do to increase our infrastructure capacity by 10% through more streamlined usage and bringing new capacity online. (joint with releng)
  • [DONE] Enable performance and test automation to run on Android x86 hardware, investigate Android 2.3.x support on Pandas.