947
edits
Auto-tools/Projects/Signal From Noise/Execution2012: Difference between revisions
Jump to navigation
Jump to search
m
Auto-tools/Projects/Signal From Noise/Execution2012 (view source)
Revision as of 23:40, 28 December 2012
, 28 December 2012→Problems We Aimed to Solve with Datazilla
(→Execution of Signal from Noise: remove line that is now hanging without a reference to it) |
|||
| Line 21: | Line 21: | ||
=== Problems We Aimed to Solve with Datazilla === | === Problems We Aimed to Solve with Datazilla === | ||
* Preserve and capture raw performance numbers | |||
* Preserve and capture raw performance numbers: The Talos test framework is a bad place to do statistics, because if you do any averaging before uploading the results then the ability to retrieve the original data is forever lost. Instead, datazilla should take in all raw values from talos and provide a central platform for regression/improvement detection and statistical study | |||
* Reduce the granularity of Talos from a page set to a single page: http://k0s.org/mozilla/blog/20120425093346 ; statistics and regressions should be dealt with on a per-page basis, as pages may have wildly different performance values; see also https://wiki.mozilla.org/Metrics/Talos_Investigation#Unrolling_Talos | * Reduce the granularity of Talos from a page set to a single page: http://k0s.org/mozilla/blog/20120425093346 ; statistics and regressions should be dealt with on a per-page basis, as pages may have wildly different performance values; see also https://wiki.mozilla.org/Metrics/Talos_Investigation#Unrolling_Talos | ||
* Establish a full, extensible RESTful interface to the data: Datazilla's data and statistical methods should be accessible by all developers and the tools they wish to create and modify to use the data. | * Establish a full, extensible RESTful interface to the data: Datazilla's data and statistical methods should be accessible by all developers and the tools they wish to create and modify to use the data. | ||