342
edits
TestEngineering/Performance/RegressionBugsHandling: Difference between revisions
TestEngineering/Performance/RegressionBugsHandling (view source)
Revision as of 12:55, 12 May 2020
, 12 May 2020no edit summary
Alexandrui (talk | contribs) (Created page with "= Handling of Performance Regression Bugs = We use different policies for handling large and small regressions. = Policy #1: Backouts within 48-hours for large regressions =...") |
Alexandrui (talk | contribs) No edit summary |
||
| Line 3: | Line 3: | ||
We use different policies for handling large and small regressions. | We use different policies for handling large and small regressions. | ||
= Policy #1: Backouts within 48-hours for large regressions = | == Policy #1: Backouts within 48-hours for large regressions == | ||
The perf team and the A-Team are testing out a new policy: backing out patches that cause significant performance regressions within 48 hours. | The perf team and the A-Team are testing out a new policy: backing out patches that cause significant performance regressions within 48 hours. | ||
| Line 20: | Line 20: | ||
The A-Team has been working hard on improving the tools for understanding performance regressions (e.g. Perfherder), and we think debugging a performance regression is a much less painful process these days. For example, there is now a highly usable view to visualize the comparison between a proposed patch against a baseline revision at https://treeherder.mozilla.org/perf.html#/comparechooser. | The A-Team has been working hard on improving the tools for understanding performance regressions (e.g. Perfherder), and we think debugging a performance regression is a much less painful process these days. For example, there is now a highly usable view to visualize the comparison between a proposed patch against a baseline revision at https://treeherder.mozilla.org/perf.html#/comparechooser. | ||
= Policy #2: Policy for smaller regressions = | == Policy #2: Policy for smaller regressions == | ||
===== When a performance regression is detected and the patch which caused it is identified ===== | ===== When a performance regression is detected and the patch which caused it is identified ===== | ||
| Line 49: | Line 49: | ||
The bottom line is that we want a visible decision on each case where the cause for the regression is known. | The bottom line is that we want a visible decision on each case where the cause for the regression is known. | ||
====See also==== | = Running performance tests = | ||
If, for some reason, the graph that generated the regression alert is not conclusive enough, there's a [[TestEngineering/Performance/RunningTests| Guide to running performance tests]] that can help you so additional pushes to try to find out what's missing. | |||
==== See also ==== | |||
* [https://wiki.mozilla.org/Buildbot/Talos/Sheriffing Regression sheriffing] - how are regressions detected and associated with patches. | * [https://wiki.mozilla.org/Buildbot/Talos/Sheriffing Regression sheriffing] - how are regressions detected and associated with patches. | ||