Auto-tools/Goals/2013Q3

	THIS PAGE IS A WORKING DRAFT
	The page may be difficult to navigate, and some information on its subject might be incomplete and/or evolving rapidly. If you have any questions or ideas, please add them as a new topic on the discussion page.

Projects and Tasks

Bugzilla

Optimize bugzilla as a platform that we can build on top of.

• REST API expansion and optimization
• Stand up reviewboard as a semi-independent reviewing service
• Tracking flag migration

Performance

• Real World Benchmarking
  • Define and deliver a mocked up harness that uses the live internet for developers (non-automated)
• Webpagetest
  • deliver an internal version of this using recorded pages, stretch: ssl
  • push ssl support to networking team (they create record/replay for ssl)
• Stoneridge
  • add additional pages to stoneridge, make the numbers repeatable and reliable
• Signal from noise / talos / datazilla
  • Roll out statistics on datazilla for ALL test suites on m-c
  • Integrate kyle's alert system (calc the proper stats for this) with the data
• Games benchmark
  • Stabilize and help roll out a minimally viable automation for a small set of benchmarks
  • Help with creating their dashboard as well

Datazilla

• Automate datazilla database rollover and rollover talos data. What do we want to do with legacy data? (Stretch goal)
• Refactor datazilla to remove all portions of the old UI, unnecessary controllers, and legacy code not in production
• Provide datazilla landing page that displays data for all projects
• Replace graph server for talos on m-c,m-i

TreeHerder

• Deliver UI parity with existing TBPL
• Implement UI plugin architecture
• Set up publicly accessible instance
• Develop a logging strategy:
  • for the data adapters
  • for the restful api

MozBase

• unittests for all modules
• full mochitest and reftest integration with mozprofile/mozprocess/mozrunner
• continue to standardize + impove dev practices
• structured logging

Eideticker

• Get other people involved in writing tests for Eideticker (davehunt, gbrown?)
• Define, implement and run a set of B2G tests that helps B2G product team track progress on their 1.1/1.2 performance goals

Infrastructure Load

• parallelize xpcshel test framework
• analyze always passing tests and determine a strategy for turning tests off selectively/running intermittently

sheriff goals

TBD

• Drive treeherder UI functional specification

mozmill 2.0 released

• Release 2.0 (should be done Q2)
• SV leads metro testing using mozmill

Bugmaster

• TBD

Marionette

• Get a JS marionette client hooked up with our test runner (https://bugzilla.mozilla.org/show_bug.cgi?id=879976)
• Clean up oranges
• Better error reporting
• Add support for nested content processes
• Continue support for B2G testing / developer ergonomics (webqa, mach support) ... originally scheduled for Q2 but dropped for focus on gaia tests
• Support social API tests

B2G general

• Get gaia integration tests running in TBPL
• Finish getting gaia-ui-tests running in TBPL
• Stand up tests on new Android 4.2 emulator (this is potentially going to be very time consuming)
• Get b2g jsreftests working, so we can turn off no-ion builds/tests (stretch goal)
• Get tests running using full-stack emulator builds
• Implement heuristics to notify dev managers of "real" failures from B2G failures on Jenkins