Confirmed users
513
edits
EngineeringProductivity/Projects/Stockwell/Meetings/2017-04-25: Difference between revisions
EngineeringProductivity/Projects/Stockwell/Meetings/2017-04-25 (view source)
Revision as of 20:34, 25 April 2017
, 25 April 2017add intermittent counts
(Created page with "= Previous Action Items = = Status = = Discussion Topics = * * = New Ideas to investigate = * = Action Items = * Discussion topics for next time ** ** *") |
Klahnakoski (talk | contribs) (add intermittent counts) |
||
| (7 intermediate revisions by 2 users not shown) | |||
| Line 2: | Line 2: | ||
= Status = | = Status = | ||
* OF: | |||
** last week: 6.64 | |||
** 4 weeks ago: 10.08 | |||
* high frequency intermittents | |||
** last week 31 | |||
** 4 weeks ago 55 | |||
* if all high frequencies were fixed, OF would be...? | |||
** 4.75 | |||
** 5.13 | |||
* year to date: 198 fixed, 110 disabled (disabled is 35.7% vs 34.9% 1 month ago) | |||
= Discussion Topics = | = Discussion Topics = | ||
* | * [gbrown] Starting to look at "new/modified test verification" - {{bug|1357513}} | ||
* | ** overall great idea, web-platform does this on upstream new tests, 10x per test | ||
** keep it small, don't do dependent files, or run randomization/high resource programs in the background | |||
** how to track the effectiveness of this: | |||
*** whiteboard tags for intermittents related to test changes | |||
*** build a dashboard for recent changes which are 5+ intermittents in a week | |||
* [gbrown] Trying to enable eslint on more tests - {{bug|1357557}} | |||
** will spend a few days hacking on this to see how easy this is to make a big sweep through the tree | |||
** might be tough to whitelist existing failures or only run on new changes. | |||
* [jmaher] Lifecycle of managing intermittents today | |||
** 3 sources: | |||
*** [https://brasstacks.mozilla.com/orangefactor/index.html overall OF] (looking for missing stockwell tags) | |||
*** [https://people.mozilla.org/~klahnakoski/NeglectedOranges/index.html neglected] | |||
*** [https://bugzilla.mozilla.org/buglist.cgi?resolution=---&resolution=FIXED&resolution=INVALID&resolution=WONTFIX&resolution=DUPLICATE&resolution=WORKSFORME&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=%5Bstockwell%20needswork%5D&list_id=13546073 needswork] | |||
*** [gbrown] reduced OF to the last day or two to find most recent troublemakers | |||
** is there value to retriggering to identify the root cause | |||
* [jmaher] Roadmap for autoclassification in Q2/Q3 ? (how can we track what is auto classified vs not) | |||
** query the treeherder db for what is autoclassified- should experiment with this and propose an api | |||
** Show all intermittents, with auto-classify count, for past week: https://activedata.allizom.org/tools/query.html#query_id=QaxkaHye | |||
{ | |||
"from":"treeherder", | |||
"select":[ | |||
{"aggregate":"count"}, | |||
{ | |||
"name":"is_auto", | |||
"value":{ | |||
"when":{"in":[ | |||
{"literal":"autoclassified intermittent"}, | |||
"failure.notes.failure_classification" | |||
]}, | |||
"then":1, | |||
"else":0 | |||
}, | |||
"aggregate":"sum" | |||
} | |||
], | |||
"edges":"bugs.bug_id", | |||
"where":{"and":[ | |||
{"eq":{"failure.notes.failure_classification":"intermittent"}}, | |||
{"gt":{"last_modified":{"date":"today-week"}}}, | |||
{"ne":{"repo.branch.name":"try"}} | |||
]}, | |||
"limit":10000 | |||
} | |||
= New Ideas to investigate = | = New Ideas to investigate = | ||