Confirm
975
edits
Changes
→Performance testing: Add list of performance tests
== Performance testing ==
Performance tests can have a goal of preventing regressions, measuring absolute performance as experienced by users, or measuring performance vs. a baseline (e.g. comparing fenix to fennec). It can be difficult to write tests that manage all of these. We tend to focus on preventing regressions.
=== List of tests running in fenix ===
The perftest team is working to dynamically generate the list of tests that run on the fenix application. Some progress can be seen in [https://sql.telemetry.mozilla.org/queries/77734/source this query]. Until then, we manually list the tests below.
As of Feb. 23, 2021, we run at least the following performance tests on fenix:
* Page load duration: see the query above for a list of sites (sometimes run in automation, sometimes run manually; todo: details)
* media playback tests (TODO: details; in the query above, they are prefixed with ytp)
* Start up duration (see [[#Terminology|Terminology]] for start up type definitions)
** COLD VIEW tests on mach perftest. Runs per master merge to fenix on unreleased Nightly builds so we can identify the commit that caused a regression
** COLD MAIN & VIEW tests on FNPRMS. Runs Nightly on production Nightly builds
* Speedometer: JS responsiveness tests (todo: details)
* tier 3 unity webGL tests (todo: details)
There are other tests that run on desktop that will cover other parts of the platform.
== Offense vs. defense ==
