342
edits
TestEngineering/Performance/Sheriffing/Alert FAQ: Difference between revisions
TestEngineering/Performance/Sheriffing/Alert FAQ (view source)
Revision as of 10:13, 23 January 2020
, 23 January 2020Redirected page to TestEngineering/Performance/FAQ
(link platform_microbench) |
Alexandrui (talk | contribs) (Redirected page to TestEngineering/Performance/FAQ) |
||
| (10 intermediate revisions by 3 users not shown) | |||
| Line 1: | Line 1: | ||
#REDIRECT [[TestEngineering/Performance/FAQ]] | |||
= FAQ = | = FAQ = | ||
= What alerts are displayed in Alert Manager = | = What alerts are displayed in Alert Manager = | ||
[https://treeherder.mozilla.org/perf.html#/alerts Perfherder Alerts] defaults to [ | [https://treeherder.mozilla.org/perf.html#/alerts Perfherder Alerts] defaults to [[TestEngineering/Performance/Sheriffing/Alerts|multiple types of alerts]] that are untriaged. It is a goal to keep these lists empty! You can view alerts that are improvements or in any other state (i.e. investigating, fixed, etc.) by using the drop down at the top of the page. | ||
= Do we care about all alerts/tests = | = Do we care about all alerts/tests = | ||
| Line 10: | Line 11: | ||
Here are some platforms/tests which are exceptions about what we run: | Here are some platforms/tests which are exceptions about what we run: | ||
* Linux 64bit - the only platform which we run dromaeo_dom | * Linux 64bit - the only platform which we run dromaeo_dom | ||
* Linux 32/64bit - the only platform in which no [ | * Linux 32/64bit - the only platform in which no [[TestEngineering/Performance/Sheriffing/Alerts#platform_microbench|platform_microbench]] test runs, due to high noise levels | ||
* Windows 7 - the only platform that supports xperf (toolchain is only installed there) | * Windows 7 - the only platform that supports xperf (toolchain is only installed there) | ||
* Windows 7/10 - heavy profiles don't run here, because they take too long | * Windows 7/10 - heavy profiles don't run here, because they take too long while cloning the big profiles; these are tp6 tests that use heavy user profiles | ||
Lastly, we should prioritize alerts on the Mozilla-Beta branch since those are affecting more people. | Lastly, we should prioritize alerts on the Mozilla-Beta branch since those are affecting more people. | ||
| Line 24: | Line 25: | ||
We have some tests which measure internal metrics. A few of those are actually reported where a higher score is better. This is confusing, but we refer to these as reverse tests. The list of tests which are reverse are: | We have some tests which measure internal metrics. A few of those are actually reported where a higher score is better. This is confusing, but we refer to these as reverse tests. The list of tests which are reverse are: | ||
* canvasmark | |||
* dromaeo_css | * dromaeo_css | ||
* dromaeo_dom | * dromaeo_dom | ||
* rasterflood_gradient | |||
* speedometer | |||
* tcanvasmark | |||
* v8 version 7 | * v8 version 7 | ||
Here is a view of a reverse regression: | Here is a view of a reverse regression: | ||
| Line 37: | Line 41: | ||
NOTE: for the reverse tests we take that into account, so the bar graph will know to look in the correct direction. | NOTE: for the reverse tests we take that into account, so the bar graph will know to look in the correct direction. | ||