TestEngineering/Performance/Raptor: Difference between revisions

To see which Raptor performance tests are currently available on all platforms, use the 'print-tests' option, e.g.:

That will output all available tests on each supported app, as well as each subtest available in each suite (i.e. all the pages in a specific page-load tp6* suite).

To run Raptor locally, just build Firefox and then run:

For example, to run the raptor-tp6 pageload test locally, just use:

You can run individual subtests too (i.e. a single page in one of the tp6* suites). For example, to run the amazon page-load test on Firefox:

Raptor test results will be found locally in <your-repo>/testing/mozharness/build/raptor.json.

First, ensure your local host machine has the Android SDK/Tools (i.e. ADB) installed. Check if it is already installed by attaching your Android device to USB and running:

If your device serial number is listed, then you're all set. If ADB is not found, you can install it by running (in your local mozilla-development repo):

Then, in bootstrap, select the option for "Firefox for Android Artifact Mode," which will install the required tools (no need to do an actual build).

* The geckoview example app is already installed on the device (from ./mach bootstrap, above). Download the geckoview_example.apk from the appropriate [https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&searchStr=android%2Cbuild android build on treeherder], then install it on your device, i.e.:

The '-g' flag will automatically set all application permissions ON, which is required.

When updating the geckoview example app, you MUST uninstall the existing one first, i.e.:

Once your Android device is ready, and attached to local USB, from within your local mozilla repo use the following command line to run speedometer:

Note: Speedometer on Android GeckoView is currently running on two devices in production - the Google Pixel 2 and the Moto G5 - therefore it is not guaranteed that it will run successfully on all/other untested android devices. There is an intermittent failure on the Moto G5 where speedometer just stalls ([https://bugzilla.mozilla.org/show_bug.cgi?id=1492222 Bug 1492222]).

To run a Raptor page-load test (i.e. tp6m-1) on the GeckoView Example app, use this command line:

A couple notes about debugging:

* Raptor browser-extension console messages *do* appear in adb logcat via the GeckoConsole - so this is handy:

* You can also debug Raptor on Android using the Firefox WebIDE; click on the Android device listed under "USB Devices" and then "Main Process" or the 'localhost: Speedometer.." tab process

To run Raptor locally on Google Chrome, make sure you already have a local version of Google Chrome installed, and then from within your mozilla-repo run:

For example, to run the raptor-speedometer benchmark on Google Chrome use:

Raptor test results will be found locally in <your-repo>/testing/mozharness/build/raptor.json.

The easiest way to run Raptor tests on try is to use mach try fuzzy:

Then type 'raptor' and select which Raptor tests (and on what platforms) you wish to run.

To tell Raptor to create Gecko profiles during a performance test, just add the '--gecko-profile' flag to the command line, i.e.:

When the Raptor test is finished, you will be able to find the resulting gecko profiles (ZIP) located locally in:

To turn on Gecko profiling for Raptor test jobs on try pushes, just add the '--gecko-profile' flag to your try push i.e.:

Then select the Raptor test jobs that you wish to run. The Raptor jobs will be run on try with profiling included. While profiling is turned on, Raptor will automatically reduce the number of pagecycles to 2.