Details

July 02, 2013 @ 11:30am PDT / 2:30pm EDT

dial-in:

• vidyo: Jgriffin's vidyo room
  • guest link: https://v.mozilla.com/flex.html?roomdirect.html&key=vjLJS93feY9p

Backchannel #ateam on irc.mozilla.org

Take meeting notes here: https://etherpad.mozilla.org/great-automation-refactor Then copy and paste them below afterwards

Meeting Notes

NO MEETING DUE TO WORK WEEK

Work week notes

Mozbase Manifesto

"A mozilla-centered standard library to help us (and you!) build things faster"

• Make test harnesses and tools more maintainable
• Consolidate all wheels we use
• mozilla-centered standard library for python
• Useable in tree for both top tier test harnesses and supporting test harnesses (jetpack, services, etc)
• Useable outside of m-c
• Useful extensions to python?
• Python only? JS as well?
  • JS bits keep getting reinvented
  • Toolkit is supposed to be a consolidation point
  • A-team is fairly python-centric
• Make it easier to add features to existing harnesses
• In-tree code seems to be driving mozbase

Mozbase Roadmapping

goal: make it easy to roll out new harnesses (both desktop and remote)

• it's already partially fulfilling this goal
• bugs in mozbase can interfere with this; we're acquiring some amount of technical debt that no one has time to fix
• design flaws make mozbase components sometimes unsuitable (e.g., gps is creating an alternate process module?)

Gaps in mozbase:

• no fennec support in mozrunner/mozprocess
• moztest
• no integration layer, like a generic test runner that utilizes all the components; or a generic design pattern that people could use as an example when writing a new runner
  • annotated examples in mozbase docs?
  • an example runner?
• a mozdownload-like component that would be able to download a wider variety of builds
• a configuration/command-line component (like Jeff's) based on argparse, not optparse

Moztest

• leave it as a result container/outputter
• too hard to make a generic, flexible runner

Documentation

• missing mozprocess and other components

APIs to be improved

• mozprocess: what Ted wants (api to mirror subprocess?)
• mozprofile: stuff should be moved around a bit; it'd be nice if this started feel more collective
• mozlog: not completely flushed out; enter, chmanchester
  • problem of stiching together output of python harness code vs. javascript assertions in things like mochitest
  • Improvements to mozlog: structured output + mixin for easier integration
• mozhttpd: ted recommends mirroring
• manifestdestiny: missing APIs

Summary

• need better docs and examples of usage (example runner)
• missing API's: mozdownload and configuration
• targeted API improvements if the cost isn't too high, inlcuding fennec support for mozrunner
• more evangelism to e.g., mobile and B2G teams so they don't write similar tools on their own (e.g., adb wrappers)

Longer-term

• plan to move to Python 3?
• cloud-based services
• data output to datazilla and/or ES? let consumers store and retrieve data using their own data structures
• servo support?

Lower-level work

• move legacy runners on mozbase
• making use of packages more consistent (talos, mozharness, xpcshell, make targets ...)
• cleanup (see below)
• adding more unit tests
• improvements to mirroring scripts? (dealing with direct changes in m-c, github pull requests ...)

Q3 Goals

• implement m-c commit watcher
• finish work on refactoring desktop & B2G mochitest for mozbase
• docs: document missing packages, add some best practices and example testrunner class
• manifestdestiny API work (bug 860005)
• TODO:
  • each project should have a goal and what it would enable/unblock

Next steps: ManifestDestiny

TRACKING: 860005 next round of work for manifestdestiny 658671 Manifest format doesn't support nested includes 659898 tags in xpcshell.ini not respected (run-if,skip-if) 683660 xpccheck.py should be (mostly?) upstreamed to ManifestDestiny 860499 manifestparser should have a way of comparing manifests with directory structure

Next steps: mozprocess

Blocks: usage in mozbuild; :gps has hacked around mozprocess limitations (which will have to be unhacked) Subareas:

• IO
• tests
• windows o_O

--> TO FILE: coordinate with :gps (etc)

mozprocess I/O

794984 mozprocess should not always cross the streams (redirect stderr to stdout) 798300 mozprocess intermingles stdout/stderr poorly on Windows 875508 Consider buffering output from mozprocess processes

mozprocess test improvement

TRACKING BUG: 778267 mozprocess tests should be refactored 705864 mozprocess tests should use mozprocess.pid 796007 mozprocess should be tested with unicode and non-unicode output

mozrunner remote

697486 mozrunner should have a variant class for running fennec remotely

Consolidate -> Mozbase

TRACKING: 775756 Mozbase on Mozilla Central, round 1

Mozilla Tests on Universal Manifests

852418 Tracking - mozilla tests on universal manifests

mozfile.rmtree on windows

Currently, windows rmtree code is borked *and* duplicated several places. 861984 mozfile rmtree should retry on windows directory not empty

General Cleanup

General cleaup and refactoring. Enables faster mozbase development.

(Untriaged)

Bugs that can/should be tackled but are not yet triaged 758233 mozrunner now prevents keystrokes from reaching Firefox