Auto-tools/Goals/Roadmap2012

Last year, we put a ton of energy into creating extensibility into all our automation tools and streamlining our deployment processes for those tools. This year, we plan to put all that work to use to solve several new challenges as the Mozilla Project takes on Apps, Identity, WebRTC, B2G, and other frontiers.

Here's a very high level of how I expect the year to play out. I have less and less detail as we get farther out, but by the end of the year, I'd like to see that all the automation systems we use are accessible from one very high level dashboard. This will be aided by having a solid set of extensible utilities running our automation. I'd like to see:

• Marionette serving as the base level for all our gecko automation needs
• All our existing (and new) automation frameworks using mozbase/mozharness at their core
• Talos transformed into a statistically relevant tool that makes it easier to diagnose and track performance regressions
• Bughunter and assorted web-scale data collection mechanisms an integrated part of QA/Developer work-flow.
• Bugzilla becoming a stellar system that aids us in keeping in touch with all parts of the Project and continues to save developers and triagers and QA'ers time.
• A coherent visualization system to tie all these systems together at an extreme high-level, where you can drill into various relevant dashboards.
• That the A*Team continue to lead the way in development team contributor recruiting

2012 Q1

• Complete B2G automation and marionette
• Complete Native Fennec automation rollout
• Complete responsiveness automation rollout
• Work on Making Talos more statistically relevant
• Solidify base level items in automation - pulse, flying tanks, brasstacks, elastic search etc.

2012 Q2

• Make talos more statistically relevant (two quarter project)
• Expand mozbase to all test harnesses
• Expand marionette into other test harness drivers
• Expand into B2G hardware automation, drive emulator automation into production grade system
• Help introduce new platforms (mobile and desktop) into automation
• Tie bughunter into people's normal workflows
• Talos regression/visualization tool

2012 Q3

• All test harnesses implemented atop mozbase/mozharness.
• Complete rollout of production grade B2G automation
• Complete new platform introduction into automation
• Complete Talos regression/visualization tool

2012 Q4

• All automation systems - desktop/mobile/sync/apps/identity/qa/b2g accessible from one centralized dashboard.