Confirmed users
3,376
edits
EngineeringProductivity/Projects/Stockwell/Meetings/2016-10-25: Difference between revisions
Jump to navigation
Jump to search
m
EngineeringProductivity/Projects/Stockwell/Meetings/2016-10-25 (view source)
Revision as of 14:34, 25 October 2016
, 25 October 2016→Discussion Topics: - purgatory
Klahnakoski (talk | contribs) (add ekyle updates) |
m (→Discussion Topics: - purgatory) |
||
| Line 11: | Line 11: | ||
* Classification Tasks (aka Quarantine) ideas: | * Classification Tasks (aka Quarantine) ideas: | ||
** Purgatory (frequent intermittent tests) - what is the point of running these if the job will always be orange? could we detect if a regression was introduced? Would we disable tests if they stayed here too long (say 30 days)? | ** Purgatory (frequent intermittent tests) - what is the point of running these if the job will always be orange? could we detect if a regression was introduced? Would we disable tests if they stayed here too long (say 30 days)? | ||
*** example [[https://treeherder.mozilla.org/#/jobs?repo=try&revision=9666ec40dfced1237aa6d5c269c9cfdb2f901ac9 try push]] | |||
** Quarantine (new/edited tests) - All tests which are edited/added would run for 2 weeks to ensure it isn't intermittent. If intermittent, would we move to Purgatory, backout original patch? | ** Quarantine (new/edited tests) - All tests which are edited/added would run for 2 weeks to ensure it isn't intermittent. If intermittent, would we move to Purgatory, backout original patch? | ||
** Sanity (quick, very safe - 100% green) vs longer more complex/slightly intermittent. How would we define these safe tests? would we backout if any test here was intermittent? criteria for moving to main test suites (<10 intermittents total)? | ** Sanity (quick, very safe - 100% green) vs longer more complex/slightly intermittent. How would we define these safe tests? would we backout if any test here was intermittent? criteria for moving to main test suites (<10 intermittents total)? | ||
** Making these changes will help keep our test jobs very green, without a purpose and workflow, these will add more confusion. | ** Making these changes will help keep our test jobs very green, without a purpose and workflow, these will add more confusion. | ||