Previous Action Items

Status

[gbrown] - Orange Factor review

• • last week's OF: 7.75
  • last month's OF: 10.37

Discussion Topics

Etherpad

• Autoclassification:
  • how much we should depend on it
  • what features are currently missing from it
• Finding failures that are the same root cause (i.e. a bunch of timeouts or crashes in different tests, but the same overall)
  • how to find them
  • how to ensure we don't disable a bunch of tests over time and ignore the big picture

New Ideas to investigate

• per-push vs. per-run for a test, for determining whether something should be backed out (i.e. *actually* how severe something is, even if it's only run infrequently), for developers to understand the severity/behaviour of an orange they're investigating
• Kyle's dashboards for tracking intermittent tests
  • https://people-mozilla.org/~klahnakoski/testfailures/test.html#search=accessible/tests/browser/e10s/browser_treeupdate_imagemap.js&sampleMax=2016-12-20&sampleMin=2016-12-07
  • https://people-mozilla.org/~klahnakoski/testfailures

Action Items

• Discussion topics for next time