|
|
| Line 1: |
Line 1: |
| = About =
| | #REDIRECT [[EngineeringProductivity/Projects/Perfherder]] |
| | |
| Perfherder is a dashboard intended to allow monitoring and analysis of performance tests run against Mozilla products (currently Firefox and Firefox for Android).
| |
| | |
| Perfherder is part of the [[Auto-tools/Projects/Treeherder|Treeherder]] project.
| |
| The production version of it lives at https://treeherder.mozilla.org/perf.html
| |
| | |
| = Team =
| |
| | |
| * Talos Sheriffs (:jmaher, :vaibhav1994)
| |
| * Developers (:wlach, :jmaher, :MikeLing)
| |
| * Others (:avih)
| |
| | |
| = Problem =
| |
| | |
| Mozilla generates a lot of data from performance tests and other metrics collected during the process of development. Most of this data is useful to see on a graph and look for trends over time. It becomes difficult to understand the data when there is a lot and we need different tools for different levels of data.
| |
| | |
| In some cases we want a graphical display, but understanding all the data points for a given change to the codebase might be better represented in a table summary. Most of the time these tools are useful for investigating specific issues, which means that we need tools to analyze the data and report regressions automatically.
| |
| | |
| = Goals =
| |
| | |
| Develop tools and integration of them to assist sheriffs to find and report regressions to developers, and to assist developers in understanding the impact of their changes and potential fixes.
| |
| | |
| This includes:
| |
| * graphical display of data over time (eventually replace the existing graph server)
| |
| * finding specific data and related data for the graphs
| |
| * an easy way to compare all tests for a given revision to the previous revision (or specified one) (replace compare-talos)
| |
| * A basic alerting system (replace the existing one in graph server)
| |
| * integration with normal developer work flows for using the Try server and landing code.
| |
| * an optimized workflow for sheriffing new alerts (replace existing alert manager)
| |
| | |
| = Non Goals =
| |
| | |
| * fixing tests
| |
| * fixing harnesses
| |
| * adding new tests
| |
| * bisection of a range of commits to find the root cause
| |
| | |
| = Dependencies / Who will use this =
| |
| | |
| We have a dependency upon tools to generate data. Currently this is Talos, but the design is to allow all other types of tools to post here as well.
| |
| | |
| The other big dependency we have is TreeHerder. Right now we are tightly integrated into the project and use a lot of common data (branches, revisions, history, pushlog). We fully expect this to remain the case and will do our best to integrate as much as possible if it helps the workflow of the users.
| |
| | |
| The primary users will be:
| |
| * sheriffs
| |
| * developers
| |
| | |
| Secondary users will be:
| |
| * manager (project, release)
| |
| * qa
| |
| | |
| = Contribution =
| |
| | |
| We welcome contributions to Perfherder! We track issues inside the [https://bugzilla.mozilla.org/buglist.cgi?resolution=---&component=Perfherder&product=Tree%20Management Perfherder component on bugzilla], see below for a list of open bugs.
| |
| | |
| The [[Auto-tools/Projects/Treeherder#Contributing|contribution section]] of the Treeherder page has some handy resources for those just getting started. In general, if you have a development version of treeherder running, you're all set to hack on Perfherder! Just jump to:
| |
| | |
| * http://local.treeherder.mozilla.org/perf.html (full copy inside Vagrant)
| |
| * http://localhost:8000/perf.html (if just running the UI using the node server)
| |
| | |
| If you have a question, jump on [irc://irc.mozilla.org/treeherder #treeherder] on irc.mozilla.org and look for wlach or jmaher (mostly online in EST timezone).
| |
| | |
| == Source and docs ==
| |
| | |
| See the [[Auto-tools/Projects/Treeherder#Source_and_Docs|Treeherder]] page.
| |
| | |
| == Bug Table ==
| |
| <bugzilla>
| |
| {
| |
| "component": "Perfherder", "component_type": "contains",
| |
| "resolution": "---",
| |
| "include_fields": "id,priority,component,summary,assigned_to",
| |
| "order": "priority,assigned_to"
| |
| }
| |
| </bugzilla>
| |