Auto-tools/Projects/Signal From Noise/Meetings/2012-08-30

From MozillaWiki
Jump to: navigation, search

Previous Action Items

  • [j*] resolve test options and names and how it works with the metrics calculations
  • [talos] get live data in production database
    • DONE

Datazilla

  • [jeads] carljm finished reviewing the metrics-schema branch.
  • [jeads] We determined how we're going to implement exponential smoothing for the t-test data and we're working on implementing it now.
  • [cdawson] Working on unit tests for new web service methods

Regression/Improvement Auto Detection

Talos (code, deployment)

Tests

  • bug 731159 - tresize needs to land as aurora/mozilla-central only (not on older branches)
  • bug 770317 - new xperf tests (threadIO) coming this month
    • final integration stage, should be up for review by tomorrow.
    • will need to deploy talos and make buildbot changes after r+
  • tp5n for windows with accessibility turned on
  • possibly more updates to a11y tests this quarter
  • bug 770718 - possibly websockets tests (connections/sec, MB/sec, packets/second) this quarter, including using stoneridge. Estimate a week of work to do this, P2 for now
  • tdhml needs an overhaul, low priority

Round Table

  • [Is talos useful] ** Big old regressions in tp5 that came under the spotlight when merge day happened. Lots of discussion.
    • [tp5] regression over time
  • regular maintenance, human intervention
    • eyeball graphs once a month
    • ensure we are collecting all data types and values
  • when do we turn off graph server?

Action Items

  • [jmaher/jhammel] - https://bugzilla.mozilla.org/show_bug.cgi?id=787143
  • jmaher - follow up on dev.platform post
  • jmaher/christina - discuss other tests/data and applying stats to similar tests
  • all - figure out a blog posting schedule and who will be posting what/when
  • christina - blog and brown bag about all the science behind it title 'she blinded me with science'
  • mozregression tool for developers to bisect when a regression happened (as per taras talos meeting)