Confirm
975
edits
Changes
→How to measure what users experience: add section & content
** Avoid increasing the number of inflations
* Use lint to avoid multiple ConstraintLayouts in the same file ([https://searchfox.org/mozilla-mobile/rev/3af703be7790ff00f78d15465f3b8bb0fde0dccc/fenix/mozilla-lint-rules/src/main/java/org/mozilla/fenix/lintrules/perf/ConstraintLayoutPerfDetector.kt code])
== How to measure what users experience ==
When analyzing performance, it's critical to measure the app as users experience it. This section describes how to do that and avoid pitfalls. Note: our automated measurement tools, such as the [https://github.com/mozilla-mobile/perf-tools/blob/main/measure_start_up.py <code>measure_start_up.py</code> script], will always use our most up-to-date techniques while this page may get outdated. Prefer to use automated systems if practical and read the source if you have questions!
When measuring performance manually, you might follow a pattern like the following (see the footnotes for explanations):
* Configure your device and build. Use:
** a low-end device<sup>1</sup> (a Samsung Galaxy A51 is preferred)
** a <code>debuggable=false</code> build such as Nightly<sup>2</sup>
** enable any compile-time options that are enabled in the production app (e.g. Sentry, Nimbus, Adjust, etc.)<sup>3</sup>
* Warm-up run:
**Start the app, especially after an installation<sup>4</sup>
**Wait at least 60 seconds<sup>5</sup>. To be extra safe, wait 2 minutes.
**Set the state of the app as you want to test it (e.g. clear onboarding)
**Force-stop the app (to make sure you're measuring at least the 2nd run after installation)
* Measure or test:
**Start the app and measure what you want to measure
**If you force-stop the app, wait a few seconds before starting the app to let the device settle
**If you're testing page loads and need to force-stop the app before measuring, make sure to 1) load a page and 2) wait 15 seconds before force-stopping the app<sup>6</sup>
Footnotes:
* 1: high-end devices may be fast enough to hide performance problems. For context, a Pixel 2 is still relatively high-end in our user base
* 2: `debuggable=true` builds (e.g. Debug builds) have performance characteristics that don't represent what users experience. See https://www.youtube.com/watch?v=ZffMCJdA5Qc&feature=youtu.be&t=625 for details
* 3: if these SDKs are disabled, you may miss performance issues introduced by them or their absence will change the timing of our operations, possibly hiding performance issues. Note: the performance team would prefer for all SDKs to be enabled by default so developers can error-free build an APK similar to production APKs
* 4: we've observed the first run after installation is always slower than subsequent runs for an unknown reason
* 5: on first run, we populate certain caches, e.g. we'll fetch Pocket data and start a Gecko cache. 60 seconds will address most of these
* 6: the ScriptPreloader will generate a new cache on each app start up. If you don't let the cache fill (i.e. by loading a page and waiting until it caches ([https://searchfox.org/mozilla-central/rev/fc4d4a8d01b0e50d20c238acbb1739ccab317ebc/js/xpconnect/loader/ScriptPreloader.cpp#769 source]), the cache will be empty and you won't page load it as most users experience it
=={{#if:Glossary|<span id="Glossary"></span>|<span class="error">Error in {{tl|anchor}}: no anchor name has been specified.</span>}}<!--
