MozillaWiki

EngineeringProductivity/Projects/Treeherder

< EngineeringProductivity‎ | Projects
Revision as of 20:33, 2 April 2015 by Wlach (talk | contribs) (→‎Getting Started: minor tweaks)

Overview

Treeherder is the successor to TBPL.

The goal of the first version of TBPL was to correlate checkins with Buildbot results, so that developers and sheriffs could quickly see test results across multiple platforms for automatic builds associated with a developer's specific Firefox changesets. As more test types, platforms, products and features were added to TBPL, a variety of architectural bottle necks became apparent.

The purpose of Treeherder is to support longer term use-cases for managing and visualizing data and rewrite the existing application from scratch, so that it can scale appropriately in the future.

Contributing

Getting Started

To make UI changes, in many cases you only need to perform a very simple setup running a local webserver pointing at the production instance, described here. If you wish to hack on the backend, or the UI and backend together, you will need to install Virtualbox and Vagrant and set up both the treeherder-ui and treeherder-service Github repos as a pair, described here. The links and other information are also available off the ReadMe's in their corresponding repos.

Roadmap

Milestone M1: Functional Prototype

  • Deliver a functional prototype that sheriffs can use for E2E testing.
  • Status: [DONE] delivered 2014 Q1.

Milestone M2: Production Deployment

  • Complete a security review and deploy into production with IT.
  • Status: [DONE] delivered 2014 Q2.

Milestone M3: Sheriff Transition from TBPL

  • Work with the sheriffs to address bugs and workflow issues so that Treeherder is a viable alternate for TBPL for their workflow, and transition the sheriffs to using Treeherder for their day-to-day work. Bugs: see dependents of bug 1030636.
  • Status: [DONE] delivered 2014 Q3 (newsgroup post).

Milestone M4: Make TBPL redundant for all users

  • Primary goal:
  • Secondary goals (may be deferred to later milestones):
    • M4.2: Improve support for submitting correctness data from non-buildbot sources such as TaskCluster and Autophone. Bugs: see #Data_submission_bugs below.
    • M4.3: General cleanup to make Treeherder development more accessible to others. Bugs: see #Hacking_UX_bugs below.
  • Status: [ON TRACK] for delivery 2015 Q1.

Milestone M5: Performance Data

  • We want to obsolete Graphserver and Datazilla, and make Treeherder the dashboard that is used to visualize performance data generated by Talos and other sources. This will be comprised of at least the following:
    • A performance panel in pushlog view that can be used to see details about the performance data submitted for a particular push, plus recent antecedents.
    • A performance summary tab that gives a high-level overview by branch, platform, and suite.
  • Status: [PROPOSED] detailed planning to take place in late 2014 Q4, with delivery in 2015 Q1.

Possible Future Milestones

  • Annotation of backouts, tree closures
  • Better performance graphs
  • Integration of performance alerts
  • Integration of OrangeFactor, GoFaster dashboards
  • Create a display for code coverage data
  • Additional metrics and visualizations
  • Tier 2 jobs support

Bug Tables

We're still refining our use of the priority flag, but this is the rough rule of thumb used when triaging:

  • P1: Bugs that should take absolute priority and mean other work is put on hold. Ideal time frame for work beginning: This week.
  • P2: Important "next" bugs. Time-frame: Within a month.
  • P3: Time frame: Ideally in the next few months.
  • P4: Time frame: Next quarter or two.
  • P5: Time frame: Unknown / future.

Once one queue (eg P1) is nearing depletion, a selection of the bugs from the group below will be promoted to that priority level. Once the substantial lists of #Developer_transition_blockers and infrastructure bugs have been reduced, the guide above may change.

Assigned

A summary of Treeherder bugs that are being worked on at the moment.

Bugzilla query error

Array ( [type] => error [message] => http-bad-status [params] => Array ( [0] => 429 [1] => Unknown Error ) ) 1

Unassigned P1

All open P1 Treeherder bugs that do not have an assignee. Ideally these are the bugs that should be worked on next, once the currently assigned bugs are fixed.

Bugzilla query error

Array ( [type] => error [message] => http-bad-status [params] => Array ( [0] => 429 [1] => Unknown Error ) ) 1

Needing Triage

All bugs that do not have a priority set and are unassigned (excluding Perfherder). These are ideally triaged weekly.

Bugzilla query error

Array ( [type] => error [message] => http-bad-status [params] => Array ( [0] => 429 [1] => Unknown Error ) ) 1


Developer transition blockers

Treeherder bugs blocking remaining users of TBPL agreeing to make the transition to Treeherder, as well as fixing use-cases for which regular Treeherder users still have to fall back to TBPL from time to time.

Bugzilla query error

Array ( [type] => error [message] => http-bad-status [params] => Array ( [0] => 429 [1] => Unknown Error ) ) 1

All open bugs

Bugzilla query error

Array ( [type] => error [message] => http-bad-status [params] => Array ( [0] => 429 [1] => Unknown Error ) ) 1


Recently fixed

Bugs changed to "Resolved FIXED" in the last 14 days.

Bugzilla query error

Array ( [type] => error [message] => http-bad-status [params] => Array ( [0] => 429 [1] => Unknown Error ) ) 1


Archive

Retrieved from "https://wiki.mozilla.org/index.php?title=EngineeringProductivity/Projects/Treeherder&oldid=1066202"