342
edits
TestEngineering/Performance/Sheriffing/Workflow: Difference between revisions
TestEngineering/Performance/Sheriffing/Workflow (view source)
Revision as of 07:29, 10 November 2020
, 10 November 2020→Follow-up on regressions
(Reworded slightly to be more concise) |
Alexandrui (talk | contribs) |
||
| Line 212: | Line 212: | ||
= Follow-up on regressions = | = Follow-up on regressions = | ||
== Questioned regressions == | |||
The regression identified from the graph can be inaccurate for several reasons. When the author of the culprit patch/bug doesn't agree that their code caused the regression, we have to update the regression bug with the following whiteboard entry:<br/> | |||
* [perf:alert:?] | |||
[[File:Regression followup, regression questioned.png|regression followup, regression questioned]] | |||
<br/> | |||
<br/> | |||
When that happens, we usually take another look over the alert. What can the sheriff do is the following (but not necessary limited to):<br/> | |||
* retrigger/backfill the jobs around the regression, especially when the graph is noisy and the regression is not very clear. The sheriff is used to read the graph and can see something clear while the author of the patch doesn't have that skill.<br/> | |||
* it is possible that the alert contains 2 very close or neighbor regressions. If the alert contains different tests in terms of naming, it is possible that they are caused by different revisions and need to be confirmed/infirmed by some retriggers/backfills<br/ | |||
* sometimes, despite the graph is clear for the sheriff, the patch can contain "static" code (comments, documentation updates). In this case, other cause might be infra change. But be careful, the developer doesn't have to know what the patch does, so sometimes those are caught only if the developer questions the regression<br/> | |||
<br/> | |||
If the patch author's argument was valid, we have to update the regression bug's whiteboard entry with: | |||
* [perf:alert:0] | |||
[[File:Regression followup, regression questioned, deveoper argument valid.png|deveoper argument valid]]<br/> | |||
otherwise, we have to update the whiteboard entry with: | |||
* [perf:alert:1] | |||
[[File:Regression followup, regression questioned, deveoper argument invalid.png|developer argument invalid]]<br/> | |||
== Regressions with no activity == | |||
Regression bugs with no activity for three days should be: | Regression bugs with no activity for three days should be: | ||
* responding to open questions to sheriffs, or | * responding to open questions to sheriffs, or | ||