TestEngineering/Performance/Raptor/Browsertime

Browsertime is a harness for running performance tests, similar to Mozilla's Raptor testing framework. Browsertime is written in Node.js and uses Selenium WebDriver to drive multiple browsers including Chrome, Chrome for Android, Firefox, and (pending the resolution of Bug 1525126 and similar tickets) Firefox for Android and GeckoView-based vehicles.

Source code:

Our current (vendored) Browsertime uses the Mozilla fork of the canonical repo
In-tree: https://searchfox.org/mozilla-central/source/tools/browsertime and https://searchfox.org/mozilla-central/source/taskcluster/scripts/misc/browsertime.sh

Page-load tests

Running Locally

Prerequisites

A local mozilla repository clone with a successful Firefox build completed
Run ./mach browsertime --setup
If you plan on running Browsertime on Android, your Android device must already be set up (see more below in the Android section)

Running on Firefox Desktop

./mach browsertime https://www.sitespeed.io --firefox.binaryPath '/Applications/Firefox.app/Contents/MacOS/firefox-bin'

Running on Google Chrome

./mach browsertime https://www.sitespeed.io -b chrome

Running on the Android GeckoView Example app

./mach browsertime --android --browser firefox https://www.sitespeed.io

Running Browsertime on Try

Pending fix in bug 1566174

Bugs

Full Query

ID	Summary	Priority	Status
1563074	Lift Android device performance tuning out of `testing/raptor/raptor.py`	P2	RESOLVED
1564256	Expose browsertime to generic-worker based test tasks	--	RESOLVED
1564282	Refactor raptor/raptor.py in preparation for browsertime	--	RESOLVED
1565316	Process browsertime results and dump out for perfherder	P1	RESOLVED
1566166	Add automation to install browsertime on Win 10 production performance hardware	P1	RESOLVED
1566171	Expose browsertime-* paths to Raptor test harness	P1	RESOLVED
1566174	Taskcluster config for raptor browsertime win10 page-load test	P2	VERIFIED
1568048	Lift mozproxy management out of `Raptor` and into `Perftest`	P1	RESOLVED
1572804	Begin to build browsertime support into Raptor framework	P1	RESOLVED
1581580	Add limited set of --browsertime Raptor tests to make it easier to smoke-test changes	P1	RESOLVED

10 Total; 0 Open (0%); 9 Resolved (90%); 1 Verified (10%);