Auto-tools/Projects/Signal From Noise/Meetings/2012-05-10: Difference between revisions
< Auto-tools | Projects | Signal From Noise | Meetings
Jump to navigation
Jump to search
(→Action Items: staging tbpl) |
|||
| (6 intermediate revisions by 2 users not shown) | |||
| Line 4: | Line 4: | ||
* [christina] - analyze reduced pageset for SfN metric | * [christina] - analyze reduced pageset for SfN metric | ||
= Metric Calculations = | = Metric Calculations = | ||
* 70 pages still are not fitting in the model :( | |||
** will provide top offenders to a*team | |||
= Datazilla = | = Datazilla = | ||
| Line 20: | Line 22: | ||
** if a page doesn't fit a model then lets not run it and file a bug to investigate the page (80/20 rule here) | ** if a page doesn't fit a model then lets not run it and file a bug to investigate the page (80/20 rule here) | ||
** consider a maybe flag (probably orange) which doesn't match our model, but we cannot determine programatically that it is a regressions or measurable improvement. | ** consider a maybe flag (probably orange) which doesn't match our model, but we cannot determine programatically that it is a regressions or measurable improvement. | ||
* new color for tbpl? (You might have regressed?) | |||
* killing pages for network access: https://bugzilla.mozilla.org/show_bug.cgi?id=720852 (could just use datazilla) | |||
= Action Items = | = Action Items = | ||
* find what pages we can consider reliable | |||
** define what "reliable" is | |||
* triangulate | |||
* [christina] - give jeads/ctalbert/jmaher a set of pages that are just too noisy and operating systems | |||
* local TBPL (contact: edmorley) | |||
Latest revision as of 18:49, 10 May 2012
Previous Action Items
- [ctalbert,carljm,jhammel,jmaher] - look at datazilla VM
- [jmaher] - identify tp5 tests that hit the internet, give christina set of numbers with those pages missing
- [christina] - analyze reduced pageset for SfN metric
Metric Calculations
- 70 pages still are not fitting in the model :(
- will provide top offenders to a*team
Datazilla
Page Specific Views
Compare Talos Functionality
Round Table
- proposal for moving forward
- focus on tdhtml and tsvg as page centric
- find a model for pages that can detect regressions with a single new data point
- display data on graphs for investigation purposes
- determine a metric to report for pass fail purposes and tracking as a page set (mochitests have pass/fail/todo, and if 1 test fails, we turn the job orange, maybe we do something similar here)
- if a page doesn't fit a model then lets not run it and file a bug to investigate the page (80/20 rule here)
- consider a maybe flag (probably orange) which doesn't match our model, but we cannot determine programatically that it is a regressions or measurable improvement.
- focus on tdhtml and tsvg as page centric
- new color for tbpl? (You might have regressed?)
- killing pages for network access: https://bugzilla.mozilla.org/show_bug.cgi?id=720852 (could just use datazilla)
Action Items
- find what pages we can consider reliable
- define what "reliable" is
- triangulate
- [christina] - give jeads/ctalbert/jmaher a set of pages that are just too noisy and operating systems
- local TBPL (contact: edmorley)