EngineeringProductivity/Projects/Treeherder/Current TBPL Feature List

From MozillaWiki
Jump to: navigation, search

Menus

Help menu

  • Job result colour legend
  • Job letter & description list (description same as tooltip), generated from regex/mappings used to display them.
  • Keyboard shortcuts list
  • Link to TBPL wiki page

Tree info menu

  • Sheriff for current tree -> IRC link in the case of most trees. Used to be pulled from a calendar feed but rarely used any more, to now just hardcoded in Config.js for each tree (with fallback). Need to consider if going to use the new Sheriff duty site/API or if redundant in the employed permasheriff new-world.
  • Link to buildduty google calendar (build duty is across all trees; used to be pulled using Google Calendar API but unreliable, increases external asset load count, and also some devs didn't like google tie-in).
  • Link to open hidden builders admin panel
  • Link to tree status app (open/close the tree; treestatus.mozilla.org)
  • Link to hgweb pushlog for current tree
  • Link to clobberer for current tree (makes build slaves delete build object directories; requires LDAP auth).

Infrastructure menu

  • Links to both table and graph forms of pending and running job pages
  • Regularly refreshed (2 mins) summary of pending/running jobs per tree, plus overall count

Filters

  • Filter by person who pushed
  • Filter by job name (buildername). Supports regex. Very useful for sheriffing.
  • Toggle "show only unstarred failing jobs mode"

Menu bar misc

  • Count of unstarred failing jobs shown in current view (accounts for filtering)
  • Timezone toggle (between local and server time)
  • Tree selector (does MRU to chose which three trees to list outside of the dropdown, plus the order inside the dropdown)
  • Tree state (open/closed/approval required, tree MOTD, tree closed reason (using treestatus.mozilla.org API)
  • Message panel (only appears when messages present). eg: error loading pushlog, state of retriggers, page load/refresh percentage, number of intermittent-failure comments left to submit to bugzilla.

Pushlog

  • Shows by default last 10 pushes (with some time limit cutoff, e.g. <24hrs). Arrow to load additional batches of 10. &rev=changeset or &fromchange=changeset1&tochange=changeset2 can be used to specify ranges.
  • Updates every N minutes (**TODO**)
  • User who pushed -> links to filtered view of just their pushes (more relevant for Try)
  • Data/time of push (adjusted for local clock) -> links to view showing just that push
  • Changesets in the push
    • Changeset hash -> links to hgweb for that change set
    • Changeset author
    • Commit message
  • Checkboxes for using talos-compare
  • Link to m-cMerge (on hover). Ideally would host m-cMerge on a Mozilla property (or even in TBPL).
  • Link to BuildAPI (on hover). BuildAPI requires LDAP.
  • Cancel all pending/running jobs on push (on hover)
  • List changeset URLs (on hover)
  • Build/Test results:
    • Each OS/type of build on own row. Not every push may build everything (or at all)
    • Currently matches buildbot "buildername" against a tangled mess of regex in the UI to determine which jobs belong on which row, what letter represents which job type & which job types should be grouped in parenthesis).
    • Jobs can be pending (light grey), running (dark grey) or completed (colour depends on result).
    • Completed job results map to the standard buildbot result states (success, warnings, failure, exception, retry) [note soon to be 'cancelled' too, once buildbot updated from upstream]).
    • If a completed and != success job has had a comment added, is marked with an asterisk ("starring" failures).
    • Hovering over each job shows job description tooltip, ETA for running, runtime for completed.
    • Left clicking a job (or using keyboard shortcuts to cycle though unstirred jobs) makes it the selected job and displays the details for it in the lower TBPL UI
    • Middle clicking on a completed job opens the brief log in a new tab (saves selecting job, moving to lower UI panel and using link there; very useful)
    • Multiple jobs can be selected at once (using mouse or keyboard) in order to add a comment to all (e.g. multiple failures due to infrastructure issues). Bug suggestion UX for this suboptimal, since still shows suggestions, but just for the active job, not all selected jobs (mostly not used for bug suggestions, but has cause confusion for people in the past; may be best to disable suggestions if multiple jobs selected).

Highlighted job panel (lower part of UI)

  • Machine that performed the job -> links to buildapi recent job history for machine (to manually check if machine is bad)
  • Full job name -> links to filtering the current view by that job name
  • Job state/result
  • Job start/end time + duration for completed jobs or ETA for running jobs
  • Link to the uploaded build directory on ftp.mozilla.org (directory also contains raw logs)
  • Link to "brief" (i.e. excerpt that matches failure strings, with context either side) or full (but still parsed/formatted) log.
  • Auto-star button
  • Add a comment
  • Retrigger job (uses buildAPI)
  • Cancel running job (uses buildAPI)
  • Link to reftest analyser for completed reftests
  • TinderboxPrint output (prefixed strings in log for any additional info that should display in lower middle/right of TBPL UI). Eg: test pass/fail counts, links to Talos graph.m.o/datazilla results, gaia repo revisions for B2G etc
  • Annotated summary:
    • Only shown for completed jobs that != success (though this does mean harder to miss bad cases where there were matches but job is green).
    • For log lines that match specific failure regex specified server side.
    • Displays bug suggestions based on test name, full failure string and now also crash top frame if found (using bugs with keyword intermittent-failure). Doesn't yet handle the mozharness date/time log line prefix well. The way these are displayed currently is suboptimal (open and closed bugs shown, no sensible ordering based on likelihood of match; ideally we'd show top N and have rest under "shown more" etc).
    • Bug suggestions link to bugzilla.
    • Bug suggestions are accepting by toggling the star icon & then submitting using the auto-star/add a comment links lower left of the TBPL UI. UX could do with some improvements (e.g. 95% of the time if only one bug suggested, two clicks is unnecessary; also the second click is often missed by those new to TBPL)
    • Submitted comments are sent to a selection of {bugzilla,TBPL DB, orangefactor} & shown with timestamp above the annotated summary.
    • If a bug suggestion was added to a failure, the annotated summary shows a link to its orange factor page (shows trends for that intermittent-failure)
    • For leaks, no bug suggestions are currently shown, but a leak analyser link is displayed which opens a primitive analysis page in TBPL itself.

Leak analyser

  • Linked from the annotated summary for leaks
  • Uses the ++DOMWINDOW/--DOMWINDOW and TEST-START etc entries in the log to work out during which test the leak occurred. Only semi-reliable.

Reftest analyser

  • Linked from the left hand side of the lower UI
  • Supoptimal |data:$foo| implementation (can't open in new tab due to XSS issues etc)
  • Uses reftest analyser pulled from mozilla-central hgweb page

Parsed log views

  • **TODO**

Hidden builders (jobs) admin

  • Displays all buildernames for current tree (that have been imported into the DB, ie completed jobs only unfortunately) along with visible/hidden state
  • Currently sorted by buildername, no way to sort by visibility
  • Can filter by buildername (& also by visibility by typing "hidden", but I think this is a bug in its current form)
  • Is annoying to keep multiple trees in sync (have to adjust each individually)
  • Uses config.php password to control access (same password for all users, currently listed in a hidden bugzilla bug, to which people are CCed as they are given access. Hasn't changed for multiple years lol)
  • Submitted visibility changes are stored in a history table, but only accessible by hand crafting the history URL & human interpreting the returned json.

Keyboard shortcuts

  • Several keyboard shortcuts - but need more. Very heavily used by sheriffs - but often their existence is unknown to others, even though listed in help menu.
    • Open help
    • Cycle forward/back through unstarred failing jobs in current view
    • Select/deselect current highlighted job (for multiple selection)
    • Open "add a comment" panel
    • Submit the add a comment panel
    • Close open panel
    • Toggle "display only unstarred failing jobs" view

URL Parmeters

  • Tree name
  • Revision or from/to revisions
  • Job name filter
  • If unstarred failing jobs only view
  • If "hidden" jobs (ie hidden in buildername admin) should be shown (*no UI for this currently, devs have to remember parameter)
  • Pusher name filter
  • Embed mode (hides menu bar - for use by the Bugzilla Tweaks Firefox addon, which displays people's TryServer results inline in their bug).

Misc

  • Page title contains number of unstarred failing jobs, therefore when there are new failures that need investigating/starring, the tab title changes and thus turns the pinned tab blue.