97
edits
Necko/MobileCache/MicroBenchmarks: Difference between revisions
Jump to navigation
Jump to search
Necko/MobileCache/MicroBenchmarks (view source)
Revision as of 21:21, 28 September 2011
, 28 September 2011→Purpose
| Line 4: | Line 4: | ||
== Purpose == | == Purpose == | ||
The purpose of the microbenchmarks is to be tools for programmers evaluate the effect of code-changes on cache-performance. I.e. you run benchmarks, record results, apply code-changes, then run the benchmarks again. The goal is that comparing two results should give a reasonable idea how your changes affects cache-performance. | The primary purpose of the microbenchmarks is to be tools for programmers evaluate the effect of code-changes on cache-performance. I.e. you run benchmarks, record results, apply code-changes, then run the benchmarks again. The goal is that comparing two results should give a reasonable idea how your changes affects cache-performance. | ||
Implications of the above include | Implications of the above include | ||
| Line 12: | Line 12: | ||
** at the very least we must quickly be able to see variance/deviation in a dataset | ** at the very least we must quickly be able to see variance/deviation in a dataset | ||
* what we measure (e.g. which units we use) is not really important - we just need something comparable | * what we measure (e.g. which units we use) is not really important - we just need something comparable | ||
A secondary purpose of these microbenchmarks is to uncover performance-problems on new platforms. For example, enabling the disk-cache is rumored to hurt performance on mobile devices, based on running TP4 on one or two devices with and without disk cache enabled. With a reasonable set of benchmarks we can pinpoint more precisely what the problem is, which devices it affects, and do something about it. | |||
Implications of the above include | |||
* we must somehow be able to identify a bad test-result (i,e, we cannot just report reproducible, abstract numbers - the results must be somehow understandable) | |||
* we need many measurements | |||
** search time vs cache sizes vs full/empty cache | |||
** read-time vs cache-type vs large/small data | |||
** write-time vs cache-type vs large/small data (writing is off main-thread, but will probably still impact performance on devices?) | |||
** creating, clearing, adjusting size, dooming entries | |||
=== Telemetry vs microbenchmarks === | === Telemetry vs microbenchmarks === | ||