Electrolysis/Multi Release Criteria: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
Line 141: Line 141:
{| class="wikitable"
{| class="wikitable"
! Responsible
! Responsible
| e10s - erahm<br> e10s-multi - '''<span style="color:#9932cc"> TBD</span>'''
| Blake
|-
|-
! Accountable
! Accountable
| e10s - blassey<br> e10s-multi - '''<span style="color:#9932cc"> TBD</span>'''
| erahm
|-
! Supporting
| e10s jmaher (Talos)<br> e10s-multi - '''<span style="color:#9932cc"> TBD</span>'''
|-
! Consulted
| e10s - Barbara<br> e10s-multi - '''<span style="color:#9932cc"> TBD</span>'''
|-
! Informed
| e10s - cpeterson, elan, release management<br> e10s-multi - '''<span style="color:#9932cc"> TBD</span>'''
|}
|}


Line 164: Line 155:
! Criteria Met?
! Criteria Met?
! Criteria Met Signed-off By
! Criteria Met Signed-off By
|-
| tp5o Main_RSS
| style="background-color: Yellow;" | [https://treeherder.mozilla.org/perf.html#/e10s?filter=Main_RSS Perfherder] '''<span style="color:#9932cc"> Need updated link</span>'''
| style="background-color: PaleGreen;" | 331418381.64 (OS X)
| style="background-color: #b0e0e6;" | '''<span style="color:#9932cc"> TBD</span>'''
| style="background-color: #b0e0e6;" | '''<span style="color:#9932cc"> TBD</span>'''
| style="background-color: #b0e0e6;" | '''<span style="color:#9932cc"> TBD</span>'''
|-
|-
| MEMORY_TOTAL ±25% (browser + content process)
| MEMORY_TOTAL ±25% (browser + content process)

Revision as of 17:41, 30 March 2017

Page to collect and track Electrolysis Multi Process release criteria.

Rollout Criteria

We plan to ship e10s-multi in Firefox 54/55 using a staged rollout.
See elan's e10s-multi release plan.

Release Criteria

Stability

Responsible jimm
Accountable chutten
Description Metric e10s Analysis Analysis By e10s value e10s-multi value Criteria Met? Criteria Met Signed-off By
chrome + content crash rate (chrome + (content - content shutdown termination crash reports) ) / 1000 use hours "M+(c-s)" on 48 Beta Update Link here TBD 5.20 crashes/khour TBD TBD TBD

Crash rates exclude XP and deprecated OS X operating systems

Jank

Responsible Gabor
Accountable chutten
Description Metric e10s Analysis e10s-multi Analysis Analysis By Criteria Met? Criteria Met Signed-off By
GC pauses GC_MAX_PAUSE_MS (parent, children) beta 46 experiment TBD TBD TBD TBD
Cycle Collector pauses CYCLE_COLLECTOR_MAX_PAUSE (parent, children) beta 46 experiment TBD TBD TBD TBD
Event loop lag INPUT_EVENT_RESPONSE_MS beta 46 experiment

Nightly 48 telemetry

TBD TBD TBD TBD
Browser responsiveness during page load Tab paint test N/A Need bug TBD TBD TBD

Page Load

Responsible jimm
Accountable TBD
Metric Analysis Analysis By e10s value e10s-multi value Criteria Met? Criteria Met Signed-off By
FX_PAGE_LOAD_MS ±5% beta 46 experiment (bug 1228020) Need updated link TBD e10s is much better. TBD TBD TBD

Memory Usage

Responsible Blake
Accountable erahm
Metric Analysis e10s value e10s-multi value Criteria Met? Criteria Met Signed-off By
MEMORY_TOTAL ±25% (browser + content process) beta 46 experiment and erahm's blog post Need updated link(s) TBD TBD TBD
VSIZE_MAX_CONTIGUOUS (browser + content process) beta 46 experiment Need updated link TBD TBD TBD
Additional Metric(s) Needed?
- process count 2, 4 and 8 data comparisons?
- AWSY?

UI Smoothness

Responsible e10s - mconley
e10s-multi - TBD
Accountable e10s - RyanVM
e10s-multi - TBD
Supporting e10s - jmaher, avih, gabor
e10s-multi - TBD
Consulted e10s - Barbara
e10s-multi - TBD
Informed e10s - cpeterson, elan, release management
e10s-multi - TBD

Performance numbers below are broken down for both e10s and enabling APZ. Negative numbers represent performance gains.

Metric Metric Owner Analysis e10s Data e10s-multi Data Criteria Met? Criteria Met Signed-off By
TART ±5% (Tab Animation Regression Test) TBD Perfherder Need updated link

e10s / APZ
xp = 4.4% combined
win7 = 1.8% / 9%
win8 = -0.6% / 10% (improvement)
osx = 6% / 3.5%
linux = 0.5% / 16%

TBD TBD TBD
tpaint ±5% (first paint for a new window) TBD Perfherder Need updated link

xp = 0%
win7 = 9.3%
win8 = 3.4%
osx = -11% (improvement)
linux = 7%

TBD TBD TBD
tresize ±5% (window resize responsiveness) TBD Perfherder Need updated link

e10s / APZ
xp = 1.0% combined
win7 = 0.0%
win8 = -22% (improvement overall)
osx = 1.0% combined
linux = 7% / 2.0%

TBD TBD TBD
tps ±5% (tab switch speed) TBD Perfherder Need updated link

xp = -27%
win7 = -28%
win8 = -34%
osx = -30%
linux = -19%

TBD TBD TBD
FX_TAB_SWITCH_TOTAL_MS TBD beta 46 experiment Need updated link 52.66 ms (median) < 82.61 ms (median) TBD TBD TBD
CART ±5% (Customize Animation Regression Test) TBD Perfherder Need updated link

e10s / APZ
xp = improvement
win7 = 4.0% / 2.5%
win8 = 2.0% / 2.5%
osx = 3.7% / 2.7%
linux = 6.8% / 6.8%

TBD TBD TBD

Graphics Performance

Is this needed for e10-multi?

Responsible Milan
Accountable RyanVM
Supporting gw280
Consulted Barbara, jmaher, graphics team
Informed cpeterson, elan, release management
Metric Metric Owner Analysis non-e10s value e10s value Criteria Met? Criteria Met Signed-off By
glterrain ±5% jgilbert Perfherder 18 19 Mixed results from 3% worse to 66% better Barbara (2016-05-06), Milan (2016-05-06)
tcanvasmark ±5% Perfherder 6444 6301 Mixed results from 2% worse to 3% better Barbara (2016-05-06), Milan (2016-05-06)

Slow Scripts

Responsible e10s - azhang
e10s-multi - TBD
Accountable e10s - bsmedberg
e10s-multi - TBD
Supporting e10s -
e10s-multi - TBD
Consulted e10s - Barbara, billm
e10s-multi - TBD
Informed e10s - cpeterson, elan, release management
e10s-multi - TBD
Description Metric Analysis Analysis By e10s value e10s-multi value Criteria Met? Criteria Met Signed-off By
Slow script warnings SLOW_SCRIPT_PAGE_COUNT beta 47 experiment Need updated link TBD 0.00158655016424 warnings per use hour (aka 630 hours/warning)

(bug 1260769)

TBD TBD TBD

Tests

All unit tests disabled for e10s-multi must be triaged and:

  • re-enabled (and fixed, if necessary)
  • or annotated that the test is irrelevant for e10s-multi
  • or deleted

We want each test directory to be signed-off by the feature area's owner.

Responsible e10s - Blake, Felipe
e10s-multi - TBD
Accountable e10s - jimm
e10s-multi - TBD
Supporting e10s - dolske (coordinating people fixing front-end tests)
e10s-multi - TBD
Consulted e10s - jgriffiths
e10s-multi - TBD
Informed e10s - cpeterson, elan, release management
e10s-multi - TBD
Task Owner Criteria Met? Criteria Met Signed-off By
Write test plan Tracy In progress: almost done (based on this doc triage) TBD
Execute test plan Tracy, SV In progress TBD
Re-enable disabled e10s tests Gabor, Blake In progress TBD

Release Blocking Bugs

All M9 bugs must be fixed before we release.

Responsible jimm
Accountable blassey
Supporting release management
Consulted jgriffiths
Informed cpeterson, elan

M9 bugs

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);


Release Criteria bugs

Need updated bug list

Bugzilla query error

Query options must be valid JSON.1


APZ Bugs

Need updated bug list

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);


Accessibility

  • Accessibility (a11y) does not block the initial release of e10s because e10s will only be enabled for users not using a11y.

Add-ons

  • Add-on support does not block the initial release of e10s because e10s will only be enabled for users without add-ons.

Service Workers

  • ???
Retrieved from "https://wiki.mozilla.org/index.php?title=Electrolysis/Multi_Release_Criteria&oldid=1167145"