Necko/MobileCache/MicroBenchmarks: Difference between revisions

There has been some discussion about xpcshell-based microbenchmarks vs using telemetry to measure cache-performance. Although IMO these approaches are complementary, there '''may''' be a point to use telemetry-code to capture data from xpcshell-tests (as opposed to using JS Time-functions from the test itself).

The overall and major benefit of this is to get real-life verification by telemetry '''after''' using synthetic, isolated and focused benchmarks in the lab. I.e. we use synthetic test-patterns implemented by xpcshell-tests in the lab to identify and qualify code-changes, then after landing code-changes we should be able to verify the effect of these changes on usage-patterns by real users.

Below is a pro/con list for each approach - feel free to add and comment. '''Note that lab-experiments in both approaches will use synthetic test-patterns - the difference is in the way we capture data.'''