Performance/Fenix/Performance reviews: Difference between revisions

Performance/Fenix/Performance reviews (view source)

Revision as of 01:31, 5 November 2021

670 bytes added , 5 November 2021

→‎Benchmark locally: Complete header

Mcomella

Confirmed users

975

edits

@@ Line 11: / Line 11: @@
 A benchmark is an automated test that measures performance, usually the duration from point A to point B. Automated benchmarks have similar trade-offs to automated functionality tests when compared to one-off manual testing: they can continuously catch regressions and minimize human error. For manual benchmarks in particular, it can be tricky to be consistent about how we aggregate each test run into the results. However, automated benchmarks are time consuming and difficult to write so sometimes it's better to perform manual tests.
-TODO
+'''To benchmark, do the following:'''
+# Select a benchmark that measures your change or write a new one yourself
+# Run the benchmark before your change
+# Run the benchmark after your change
+# Compare the result: generally, this means '''comparing the median'''
-=== Testing Start Up code ===
+We currently support the following benchmarks:
+* Cold start up with our custom system in [https://github.com/mozilla-mobile/perf-tools/blob/main/measure_start_up.py in <code>perf-tools/measure_start_up.py</code>]
+* Non-start up use cases with Jetpack Benchmark. See [https://github.com/mozilla-mobile/fenix/tree/main/app/src/androidTest/java/org/mozilla/fenix/perf here for existing benchmarks.]
-To test start up code, the approach is usually simple:
+=== Measuring start up duration ===
+To measure the start up duration, the approach is usually simple:
 # From [https://github.com/mozilla-mobile/perf-tools the <code>mozilla-mobile/perf-tools</code> repository], use <code>measure_start_up.py</code>.<br>The arguments for start-up should include your target (<code>Fenix</code> or <code>Focus</code>).