Confirmed users
1,905
edits
Dev-sanity: Difference between revisions
Jump to navigation
Jump to search
→Intermittent Oranges: Fill in this section
(→Intermittent Oranges: Fill in this section) |
|||
| Line 5: | Line 5: | ||
= Intermittent Oranges = | = Intermittent Oranges = | ||
During the second half of 2015, we will focus on reducing the noise and improving the signal in the way we handle intermittents. This will result in notifications being meaningful and actionable, and the overall drag on developer efficiency will be reduced. | |||
The various pieces of this are: | |||
# Turn off bugmail for intermittents, and instead, periodically post a summary of bug stats. The period will probably vary depending on the frequency of failure. | |||
# Treeherder will start tracking the rate of intermittent test failures, and bugs will only be filed for intermittents that reach a certain threshold. Lower-incidence intermittents will never make it into Bugzilla, but will still be viewable in e.g., OrangeFactor. | |||
# Failing jobs on try runs will be automatically retriggered, within certain thresholds. This will allow developers to bypass the current time-consuming manual retrigger-and-wait cycles they currently often use to help determine if failed jobs are intermittent or not. | |||
# We will implement auto starring of oranges in Treeherder. This will initially likely apply to simple job failures (those that don't cause a cascade of other failures), with edge cases being handled later. This will reduce the sheriff workload and make interpreting try runs much easier. It will also enable other cool things, like smarter automatic retriggering on try runs and trunk branches, and provide data that can be used to seed automatic bisection bots. | |||
# Experiment with bots to vet new tests and changes to tests (to verify that these don't introduce intermittents) and to bisect new, frequent intermittents. | |||
Link to Whistler presentation: http://slides.com/jgriffin/deck#/ | |||
= Tree Closures = | = Tree Closures = | ||