Confirm, emeritus
998
edits
Changes
→Test types: Add Real-world WebExtension section.
* [[#Heavy]]: Run tests with the heavy user profile instead of a blank one
* [[#Web extension]]: Run tests with a web extension to see the perf impact extension have
* [[#Real-world WebExtensions]]: Run tests with a set of 5 popular real-world WebExtensions installed and enabled.
Some tests measure different things:
== Web extension ==
Web Extensions are what Firefox has switched to and there are different code paths and APIs used vs addons. Historically we don't test with addons (other than our test addons) and are missing out on common slowdowns. In 2017 we started running some startup and basic pageload tests with a web extension in the profile ({{bug|1398974}}). We have updated the Extension to be more real world and will continue to do that.
== Real-world WebExtensions ==
We've added a variation on our test suite that automatically downloads, installs and enables 5 popular WebExtensions. This is used to measure things like the impact of real-world WebExtensions on start-up time.
Currently, the following extensions are installed:
* Adblock Plus (3.5.2)
* Cisco Webex Extension (1.4.0)
* Easy Screenshot (3.67)
* NoScript (10.6.3)
* Video DownloadHelper (7.3.6)
Note that these add-ons and versions are "pinned" by being held in a compressed file that's hosted in an archive by our test infrastructure and downloaded at test runtime. To update the add-ons in this set, one must provide a new ZIP file to someone on the test automation team. See [[https://bugzilla.mozilla.org/show_bug.cgi?id=1575089#c3 this comment in Bugzilla]].
== Paint ==
