Notices, Highlights, Roundtable

• Review old Q2 goals - went ahead and marked a few things that we expect to complete this week as "done" and things that we expect to miss or flow into the early part of next quarter as "miss".
• Review new Q3 goals draft - we are doing more proactive things this quarter, in particular those are:
  • Creation of services automation system in AWS, integrating with Treeherder
  • Completing treeherder's UI, allowing us to begin using it
  • Expanding the notion/reach of performance benchmarks (and re-using the same code to do it)
  • Allowing datazilla to take over from graphserver for talos testing.

Keep in mind that this is a full list. So anything not on this list, we should de-prioritize so that we can stay focused and drive these tasks to completion. Note that some of them (like the services automation piece) are going to need to be heavily scoped.

Goals

• Goal: Improve Developer/QA ergonomics around running and debugging automation by:
  • Add debugger support to B2G testrunners
  • Improve patch review experience on bugzilla/Make bugzilla more user-centric
  • Replace graph server with Datazilla for all desktop Talos tests
  • Create a project branch view of B2G builds/tests that tracks commits per target, rather than per repo (all commits to any repo for unagi for instance) joint with releng

• Goal: Improve stability, reliability, and scalability of automation platforms by:
  • Addressing all P1 B2G automation stability issues
  • Complete native REST interface for Bugzilla
  • Complete Mochitest to Mozbase port
  • Create POC Microbenchmark suite to get feedback from developers for its usefulness
  • Reduce orange factor to 5 or lower (for the last three weeks of the quarter) by driving awareness among developers, following up with assignees, and fixing intermittent automation failures.
  • Update pulse so that its dependent (client side) libraries are current for its use in Tree Herder
  • Create a prototype and a specification for a rig to power cycle real phones. Have a vendor selected to build the rig by end of quarter.

• Goal: Build toward our future by:
  • Deploy a minimally viable TBPL replacement by end of quarter, aim to switch over mid-Q3
  • Release a mozmill 2 and work with contributor community to drive toward a 2.1
  • Automate a scrolling fps test using camera Eideticker measurements for Contacts, Email, Settings, and Message apps
  • Create and Deploy BMO user profile data for community building and education
  • Create a solid net and filter system to build a bugmaster community
  • Identify 5 things we can do to increase our infrastructure capacity by 10% through more streamlined usage and bringing new capacity online. (joint with releng)
  • Enable performance and test automation to run on Android x86 hardware, investigate Android 2.3.x support on Pandas.

Deep Dives

Mobile (1st Monday)

AutoPhone/Noah's Ark

B2G Automation

Mobile Evangelism Site-Scraping Tool

Android Automation

Tools (2nd Monday)

Bugzilla

Bughunter

GoFaster

Pulse

MozTrap

Tinderboxpushlog

• [edmorley] New things in production: https://bugzil.la/883940
• [edmorley] Patches:
  • mcMerge has now been moved into TBPL's repo & made testable from the local filesystem (bug 875323). The temporary (and outside of our control) mcmerge.m.o instance can now be switched off \o/
  • WIP patch to add starring auto-selection & keyboard shortcuts.
  • Misc patches: https://bugzil.la/883940,884721

War on Orange

• OrangeFactor on trunk for last 7 days: 5.3

TBPL Version 2

Performance (3rd Monday)

New Graphserver (Datazilla)

Signal From Noise

SpeedTests

Eideticker

• [wlach] Added minimal travis-ci continuous integration.

Talos

General Framework Support (4th Monday)

MozBase

• We now have support for MetroFirefox via Mozrunner on Windows 8 (bug 879386, bug 880839)

In-tree Automation/Harnesses

• [edmorley] Patch to add timeouts to mozharness urllib2.urlopen() calls & make download_file() timeouts retry (bug 884115).

WebRTC Test Development

• New test framework for datachannels and a couple of basic tests have been just landed on inbound.
• Some smaller patches remain to be written before we can sign off from our assistance

Process (4th Monday)

Sheriffing & Trees

• [edmorley, Tomcat] More sheriffing training. Have started rough notes for a wiki guide so we can extend the process more easily to volunteer sheriffs.
• [edmorley] Opt mochitest-metro-chrome jobs are now shown by default on all trees (bug 885565).

Treestatus

Bugmasters

Holidays and Trips

• [edmorley] PTO Mon 1st to Fri 5th July.

Misc

• [ffledgling] - moznetwork/mozinstall - direction on specific unit tests