342
edits
TestEngineering/Performance/Sheriffing/Workflow: Difference between revisions
TestEngineering/Performance/Sheriffing/Workflow (view source)
Revision as of 15:06, 26 September 2023
, 26 September 2023→Finding the culprit
m (→Sheriffing Firefox-Android Alerts: GeckoView) |
Alexandrui (talk | contribs) |
||
| Line 86: | Line 86: | ||
<br /> | <br /> | ||
There are cases when the difference is much less noticeable and the data of the test is more unstable, and some retriggers are necessary in order to determine the interval for the test data and compare it between several adjacent tests:<br /> | There are cases when the difference is much less noticeable and the data of the test is more unstable, and some retriggers are necessary in order to determine the interval for the test data and compare it between several adjacent tests:<br /> | ||
[[File:Unstable improvement graph.png|Unstable improvement graph]]<br /> | [[File:Unstable improvement graph.png|Unstable improvement graph]] | ||
<br /> | |||
There are cases when there's a revision(s) gap between the data points in the graph, and a little investigation might save us manual bisection: | |||
https://mozilla.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=6f5cd956-d656-488c-bc91-b085009703c1 | |||
<br /> | |||
A less fortunate situation is when the test is unstable, there are gaps in graph when the tests didn’t run for various reasons and the regression/improvement is almost impossible to be determined. If the investigation takes more than 5 business days it’s recommended to ask for help, if you haven’t already: | A less fortunate situation is when the test is unstable, there are gaps in graph when the tests didn’t run for various reasons and the regression/improvement is almost impossible to be determined. If the investigation takes more than 5 business days it’s recommended to ask for help, if you haven’t already: | ||