947
edits
Auto-tools/Projects/Signal From Noise/Execution2012: Difference between revisions
Jump to navigation
Jump to search
m
Auto-tools/Projects/Signal From Noise/Execution2012 (view source)
Revision as of 23:24, 28 December 2012
, 28 December 2012→Execution of Signal from Noise
m (→Execution of Signal from Noise: extraneous) |
|||
| Line 14: | Line 14: | ||
== Execution of Signal from Noise == | == Execution of Signal from Noise == | ||
In general, it was initially expected that some back-of-the-envelope analysis of existing Talos numbers and some back-of-the-envelope engineering implementations of statistics that at least appeared (though non-rigorously) less noisy would satisfy the goals of the Signal from Noise project for the time being, with | In general, it was initially expected that some back-of-the-envelope analysis of existing Talos numbers and some back-of-the-envelope engineering implementations of statistics that at least appeared (though non-rigorously) less noisy would satisfy the goals of the Signal from Noise project for the time being, with ongoing effort optimistically being invested in analysis of performance data. The initial SfN effort was scheduled for a single quarter. This proved rather optimistic. (See also: http://k0s.org/mozilla/blog/20120829151007 .) | ||
In practice, because of the way averaging was split between talos and graphserver, it was effectively impossible to utilize the current system to develop more robust statistics. It quickly became apparent that we needed a system that preserved the raw measurements in order to allow us to use and compare the fidelity of different statistical models on the Talos data. The Talos test harness itself should not be crunching numbers: its role is measuring and reporting. | In practice, because of the way averaging was split between talos and graphserver, it was effectively impossible to utilize the current system to develop more robust statistics. It quickly became apparent that we needed a system that preserved the raw measurements in order to allow us to use and compare the fidelity of different statistical models on the Talos data. The Talos test harness itself should not be crunching numbers: its role is measuring and reporting. | ||