Canmove, confirm
2,850
edits
Changes
no edit summary
Treeherder is the dashboard used by sheriffs and the rest of Mozilla to see build and test results. When you first open [https://treeherder.mozilla.org Treeherder], there will be a lot of information thrown at you all at once. This document should help explain what to do with it all.
[[File:Treeherder_intermittent.png|thumbnail|right|Treeherder interface showing an intermittent failure]]
= Common Terms =
= The Treeherder UI =
== Header ==
At the top of the site's header section are some menus and buttons that can be useful. The 'Infra' menu holds links to a few other Mozilla tools/websites. The 'Repos' menu provides links to view other Mozilla source code repositories within Treeherder. The 'Filters' menu has tools to filter down the jobs shown within Treeherder on various criteria. The '?' menu has various help pages for Treeherder. The 'Login/Register' button lets you log in to Treeherder, allowing you to make changes to/with Treeherder's tools.
Below the site header is the main part of Treeherder. This shows the recent commits to the currently selected repository grouped by the [http://mozilla-version-control-tools.readthedocs.io/en/latest/hgmo/pushlog.html pushes] they were landed in.
The left side of each push shows the commit data for each commit in the push (eg, the commit hash, the commit author, and the commit message). This can be just a single commit or the push could contain up to hundreds of commits. (Treeherder caps the view at 20 commits for performance reasons. Click the "... and more" links to view the entire push's contents on hg.mozilla.org.)
The right side of each push shows the build and test results for all of the jobs running with this push's commits. This is where most sheriffing work happens, as it is where job failures are reported, and the job failures need to be classified (also referred to as 'starred' in various places).
== Bottom Panel ==
If you select one of the job symbols for a push, a panel at the bottom of the page will open. <<BOTTOM PANEL IMAGE>> The panel has a few sections with information about the job.
On the left side of the panel, there are a few buttons that can be clicked to open the log file for the job as well as submit a request for the job to be re-run. Below those buttons is a list of information about the job, like the name of the job, when the job was requested by the build system, when the build system actually started working on the job, and when the job finished being run.
The 'Job Details' tab lists more detailed information from the job, including all of the files created and uploaded as part of the job running (eg, log files, screenshots of certain failures, etc).
The 'Failure Summary' and 'Failure Classification' tabs show information on a job's reported failures. They both list the same information, but are used slightly differently (. 'Failure Summary' handles all failures in a job as a whole, 'Failure Classification' can handle each individual failure within a job on its own, and is part of the Autoclassify system.) These views will list each failure line, as well as any bugs from Bugzilla that match (or are at least similar in some way) the reported failure. If you're signed in, you'll also see extra controls for classifying the failures.
The 'Annotations' tab shows any classifications that have already been performed for the job. If you're signed in, you'll also see UI for removing classifications, in the case that something was classified incorrectly and you need to correct it.
=== Pinboard ===
The pinboard is an area where you can add one or more jobs and pair those with related bugs or comments/reasons, before you submit the classifications to the database. Typically you'd use this on a single job at a time, but if multiple jobs are broken for the same reason (, e.g: * a patch broke a test completely, and you need to classify the failed jobs against the push that broke/fixed it, * something in the build infrastructure is broken, causing lots of jobs to fail at once, etc), it It can save time to pin them all at once and apply a single classification to them all with a single click. <<pinboard image here>>
== Job Colors ==
== What does it do? ==
# Submits the failure and classification to [https://brasstacks.mozilla.com/orangefactor/ OrangeFactor]. This dashboard helps track trends in classified failures, so failures that happen frequently (or suddenly start failing frequently) can be prioritized.
# Stores the classification comment and any associated bugs in Treeherders's database, so everyone viewing that tree/push can see that the failure has been starred (ie annotated with a comment and the star icon added next to the job symbol), so they know it has been dealt with.
The list of jobs and classifications updates on its own periodically as pushes and job information are reported to Treeherder.
== How are things starred/classified? ==
You first need to sign in to Treeherder to classify failures and have the classifications recorded. Click the "Login/Register" button in the header and follow the login process.
* "Check the total items count - Got 17, expected 16" is the error that occurred.
Below the failure line can be one or more bugs from Bugzilla that match up with part or all of the failure line, with the matching portions between the two highlighted in '''bold''' text. In this example, the entire bug summary (except for the added word "Intermittent") is in bold, meaning it matches up exactly with the failure line. (Careful, this only matches each individual word from the failure line to each word from the bug summary, so the order of words doesn't matter for highlighting purposes. You'll need to check yourself whether the two actually match for real.)
Next to the bug summary is a button with a 'pin' icon. If you click it, the job is added to the pinboard, and the bug number is added to the 'Bugs' section of the pinboard. Once you're confident that the classification is correct, you can click the pinboard's "Save" button to submit the classification to Treeherder's database. If this job isn't already classified, the job symbol will get a star icon added next to it.
