342
edits
TestEngineering/Performance/Sheriffing/Workflow: Difference between revisions
TestEngineering/Performance/Sheriffing/Workflow (view source)
Revision as of 08:32, 17 March 2022
, 17 March 2022→Handling improvements
Alexandrui (talk | contribs) |
Alexandrui (talk | contribs) |
||
| Line 230: | Line 230: | ||
Depending on the test, a high magnitude improvement (> 80%) should be treated more carefully. While a 100% improvement for a pageload test is impossible (the site never loads in an instant), over 80% is very rare and might imply that the test isn't loading what it should (an error page which is likely to contain much less code than the actual website). | Depending on the test, a high magnitude improvement (> 80%) should be treated more carefully. While a 100% improvement for a pageload test is impossible (the site never loads in an instant), over 80% is very rare and might imply that the test isn't loading what it should (an error page which is likely to contain much less code than the actual website). | ||
=== Invalid improvements === | |||
Here you can apply the same logic as for [https://wiki.mozilla.org/TestEngineering/Performance/Sheriffing/Workflow#Invalid_regressions invalid regressions], the difference is that the unstable graph evolution triggered an alert while the value changed in the sense of an improvement. | |||
== Invalid improvements == | == Invalid improvements == | ||