Auto-tools/Projects/Signal From Noise/Meetings/2013-01-16

From MozillaWiki
Jump to: navigation, search

Datazilla

  • Updated https://datazilla.readthedocs.org/en/latest/
  • Added in-application help
  • Fixed a cross browser UI layout bug
  • Added the threshold revision for each metric datum. In the UI this is referred to as the "compared to" field which allows a user to immediately see what revision's replicates were used in the t-test. It also helps explain the pattern of the "No Fit" visualization of the trend line.
  • In process of testing all of these changes in staging, should be in production today https://bugzilla.mozilla.org/show_bug.cgi?id=831515.

Talos

  • lots of updates to crash reporting
  • talos goes orange when we crash- now we can easily go orange when a test fails!!
  • talos now generates new-style TBPL output: bug 816634; this still has be be parsed from TBPL and possibly fine-tuned
  • will probably need (at least one) new release soon (what else is new?)

Round Table

  • BenWa has proposed adding thresholds per test to the tree, everything outside the threshold would be orange and require a perf sherriff to adjust the value.

Action Items

  • [all] track the hanging ingestion job
  • [all, future] come up with a solution for retriggers
  • [jeads] outline possible corner cases where we need a solution (i.e. thresholds)
    • [jmaher, all] work with perf team to solve this, ideally per page
  • [jeads] hack into datazilla: if (osx) branchname += "-Non-PGO"
  • [jmaher] post new talos with datazilla friendlier posting and missing 'talos ' in test name
  • [all] verify tp5n is accurate
  • [all, future] start verifying other data, tsvg*
    • future data to verify would be ts* and counters
  • [jhammel] add datazillla links to TBPL
  • [all] turn jobs orange for tp5n