Electrolysis/Test Coverage

02/14/2016

Test Suite	Windows						OSX				Linux
	XP [1]		7		8 [1]		10.6		10.10		32-bit		64-bit		asan
	Dbg	Opt	Dbg	Opt	Dbg	Opt	Dbg	Opt	Dbg	Opt	Dbg	Opt	Dbg	Opt	Opt
R(J) js-ref tests			100	100						100	100	100	100	100	100
R(C) crash tests			99	99						99	99	99	99	99	99
R(R) ref tests			99	99						99	99	99	99	99	[0]
Cpp	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
X xpc shell tests	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
Mn marionette			99	99					99	99	99	99	99	99	99
JIT jit tests	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
Wr web platform reftest			99	99					99(ab)	99(ab)	99	99(ab)	99	99(ab)
M(oth-a11y) [2]
M(oth-chrome) [3]
M(bc)				trunk: 84 aurora: 82 beta: 75		trunk: 84 aurora: 82 beta: 75		trunk: 84 aurora: 82 beta: 75	trunk: 84 aurora: 0 beta: 0	trunk: 84 aurora: 82 beta: 75	trunk: 83 aurora: 81 beta: 77	trunk: 84 aurora: 82 beta: 76	trunk: 83 aurora: 81 beta: 77	trunk: 84 aurora: 82 beta: 76	trunk: 84 aurora: 82 beta: 76
M(dt)			trunk: 75 aurora: 0 beta: 0	trunk: 89 aurora: 0 beta: 0						trunk: 89 aurora: 0 beta: 0		trunk: 90 aurora: 88 beta: 87		trunk: 90 aurora: 88 beta: 87
M(gl)			99	99						99	99	99	99	99	99
M(JP) jetpack [bug 1246453]
M(1-5)				trunk: 82 aurora: 0 beta: 0						trunk: 82 aurora: 0 beta: 0	trunk: 81 aurora: 81 beta: 79	trunk: 82 aurora: 82 beta: 80	trunk: 81 aurora: 81 beta: 79	trunk: 82 aurora: 82 beta: 80	trunk: 82 aurora: 82 beta: 80
M(p) push			100	100						100	100	100	100	100	100
W(1-8) web platform tests			99	99					99(ab)	99(ab)	99(ab)	99(ab)	99(ab)	99(ab)

Base values indicate coverage on mozilla-central. annotations (a, b, r) indicate support for other releases. Current coverage here reflects 47 on mc, 46 on ma. Tests suite coverage is rolling out with the trains.

Related Bugs

• bug 1194533 - Schedule e10s tests on OSX 10.10
• bug 1215233 - Schedule e10s tests on Windows 7
• bug 1253849 - Get e10s tests passing on all Windows versions in CI
  • Due to capacity limitations, non-Win7 versions will run on a reduced interval

Individual Test Issues

• browser and mochitest plain spreadsheet

Mochitest a11y

a11y is known to not work under e10s, and the tests in this suite largely fail, so there are no short-term plans to enable running this suite in continuous integration.

Mochitest chrome

These tests are thought to be largely non-e10s specific, so it doesn't make sense to run them in the content process in e10s mode. There is work underway to identify tests which are relevant to e10s to and to figure out what to do with them. There are no short-term plans to enable this suite in continuous integration.

Windows Coverage

Because of constraints on our Windows hardware test pools, we're not planning to enable any e10s suites on Windows XP or Windows 8. Instead, Windows 7 is the focus of all our e10s test coverage.