Auto-tools/Meetings/goals meeting template2015Q1: Difference between revisions

= Goal Updates =

== Marionette ==

* details: implement modal dialog support, separation of Marionette client into a separate package, release of older version of Marionette client to support update tests, modification of harness to dynamically load appropriate version of Marionette client

* bugs: {{bug|906712}}, {{bug|1109183}}, {{bug|1107336}}

=== Support the conversion of targeted P1 mozmill tests to Marionette and get them running in CI [hskupin] ===

* stretch goal: get test results reported to Treeherder

* stretch goal: support the conversion of Search tests

* '''progress since last update''':

===''Supporting Tasks''===

== MozReview and Autoland (joint with RelEng) ==

=== Add support for autolanding from MozReview to try [mcote, dminor, mdoglio] ===

* details: Allow developers to trigger try runs from directly within MozReview

* '''progress since last update''':

* details: Make MozReview data in Bugzilla more useful by creating a Bugzilla field that contains dynamic information about a MozReview review request

===''Supporting Tasks''===

== Perfherder ==

* details: We want to use Treeherder to store and display Talos performance data, because Datazilla is being deprecated and Graphserver doesn't support the kind of performance analysis we'd like to perform on the data.

== Treeherder ==

* details: Tier 1 and Tier 2 jobs will have different sheriffing guidelines and different expectations.  Accordingly, we need to display them and allow users to interact with them differently.

* bug: {{bug|1113322}}

* '''progress since last update''':

* details: Develop a minimal structured log viewer which can be used to view the structured logs produced by the test harnesses.  For the initial implementation, the user should be able to toggle between the parsed log viewer and the structured log viewer, for those harnesses that produce both.

* bug: {{bug|1113873}}

* '''progress since last update''':

* details: develop a usable but minimal UI that sheriffs can use to quickly file new intermittent issues; the UI should automatically fill out some of the details that sheriffs normally have to find manually.  Future iterations can improve on this by auto-filling more fields.

* bug: {{bug|1117583}}

===''Supporting Tasks''===

* Continue to improve the performance and operational aspects of the system

===''Backlog''===

=== Implement an alternate bug view [glob] ===

* details: Implement an alternative view of bugs, to provide UX and responsiveness improvements, and as a foundation for task-/team-centric views.

* '''progress since last update''':

=== Implement versioning framework for REST API [dkl] ===

* '''progress since last update''':

=== GitHub authentication [dylan] ===

* '''progress since last update''':

== DevTools Harness ==

* details: Take the prototype that was developed in 2014 Q4 (https://github.com/luser/luciddream) and get it running in continuous integration and visible in Treeherder. It's TBD whether this will be run in buildbot or TaskCluster, but we should get it running somewhere per-commit this quarter against linux desktop Firefox and a B2G emulator.

* '''progress since last update''':

== Test Infrastructure ==

* details: Create a prototype of a command-line tool that can be used by sheriffs and others to automate retrigger-based bisection.  This could be used to help bisect new intermittent oranges, and to backfill jobs that have been skipped due to coalescing.  Integration with Treeherder or other service will be done later.

* details: Create a Proof of Concept “big data” project which will store information about every test file we run:  test status, error details, test machine and test duration to begin with.  We will use this project to develop schemas and queries that work with data this large, and we will use this data to normalize chunk sizes and provide details about which tests never fail.

===  Implement the ability to normalize chunk durations in mochitest [ahal] ===

* details: For mochitest variants on desktop and B2G, modify manifestparser and the test harnesses to be able to specify which tests are run in specific chunks.

* stretch goal: Implement the same feature for Android mochitest, which still uses old-style JSON manifests.

* '''progress since last update''':

** a prototype image

** instructions for re-creating the image

** tests running in continuous integration

** "greening" of tests

* details: This quarter, we’ll validate the data from SETA and provide some recommendations to Releng about which jobs/platforms we could schedule less often in order to reduce test load.  We’ll monitor the impact of these changes in terms of sheriffing burden and the number of retriggers this demands, and may adjust as needed.  In subsequent quarters, we’ll use additional data from the high-resolution testcase data project and OrangeFactor to provide more finely-tuned scheduling changes.

===''Supporting Tasks''===

* Provide alternate solutions for the last consumers of Datazilla and work to decommission it

* Work with devs to introduce more dynamic analyzers (like Ehsan’s setTimeout check) in test harnesses

== Performance Testing ==

* details: We want to expand the pool of people who can perform performance sheriffing to make it scale better, and to reduce the bus factor problem.

===''Supporting Tasks''===

* Continue sheriffing Talos performance regressions

* Add new benchmarks as needed to mozbench

* Improve e10s support for Talos tests and infrastructure

* Move Talos into the tree

* Make running Talos locally easier

* resources: jmaher, dminor (for mozbench primarily), and contributors

== Community ==

= Other Project Updates =