E10s/Status/May20: Difference between revisions

From MozillaWiki
< E10s‎ | Status
Jump to navigation Jump to search
No edit summary
 
Line 1: Line 1:
[[File:Yellow-sm.jpg|thumbnail|left]]<br>
[[File:Green-sm.jpg|thumbnail|left]]<br>


=e10s Update: '''May 20'''=
=e10s Update: '''July 1'''=
==Executive Summary==
==Executive Summary==
* The biggest news is that the e10s [https://wiki.mozilla.org/Electrolysis/Experiments#Beta_47 Experiment for Firefox 47] Beta has been approved to run for the entire Beta cycle (vs. being disabled at Beta 6). This means that stability is trending in the right direction and that RelMan has been provided the [https://sql.telemetry.mozilla.org/dashboard/release-management-dashboard Telemetry-based stability dashboard] they need to easily decipher e10s vs. non-e10s crashes. The experiment will end when Firefox 47 goes to GA on June 7 and we will repeat a similar experiment for Firefox Beta 48 with an eye towards shipping to 1% of our Firefox 48 GA population so long as we meet our Release Criteria.  
* The biggest news is that the stability release criteria for Beta 48 has been signed off on [https://wiki.mozilla.org/Electrolysis/Release_Criteria#Stability]. Slow script count is also within release criteria target.
* We are also at [https://wiki.mozilla.org/E10s/Status/m9#Open_M9_bugs 0 M9's] (and have been) for a few weeks 
* As of this week, 569K users have been exposed to e10s in Beta 48.  
* Given that we have met all of our release criteria and we have not regressed since 47 beta, the project is now designated 'green' or 'on track' to ride the trains to GA and be gradually rolled out (1% at start) to eligible users (41% of total population) or about 800K users for the first 3 weeks.
* '''So what's the release plan?''' We are working on a day-by-day snapshot of the activities that have to occur once Firefox 48 goes to GA. This is basically a small 'throttle team' who will be at helm of throttling e10s up or down (or disabling). This is driven by RelMan but will be supported by e10s subject matter experts so that decisions can be made quickly. You can refer to these docs for more info and WIP details: http://bit.ly/29mIbXs and


* As of this week, 800,000 users have been exposed to e10s in Beta. Our two biggest risk areas remain stability and slow script count. Stability has benefited from continued focus on fixing top crashes and uplifting them. Through continued analysis, we've found landed on some interesting conclusions which lead us to be optimistic about progress, see information, [https://wiki.mozilla.org/E10s/Status/May20#Release_Criteria here].


==Next Steps==
==Next Steps==
Line 12: Line 15:
* 05/24: Release plan review
* 05/24: Release plan review


==Release Criteria==
;M9 Count
* We are down to [https://wiki.mozilla.org/E10s/Status/m9#Open_M9_bugs 3 M9's].
;Stability
Analyzing stability data is an ongoing effort, and we have found OS versions make a difference:
*XP Crash Rates are bringing our numbers up; excluding NT data indicates that e10s is more stable than non-e10s (see this [https://sql.telemetry.mozilla.org/queries/369#table query]).
*Additionally, we have found that stability is worse for Mac OS X 10.6-10.8. These OS versions will be no longer supported as of Firefox 49.
*Next Steps include excluding experiment participants who are using these OS versions and see how it impacts our crash stats as well as figuring out if graphics acceleration is part of this (also for non-e10s).
;[https://bugzilla.mozilla.org/show_bug.cgi?id=1260769 Slow Script Count]
*Slow script count increased up to 0.11 from week 2's 0.09
* We will be doing an analysis similar to the once cited above to help investigate


==Add-Ons + WebExtensions==
==Add-Ons + WebExtensions==
Line 31: Line 21:


==Automated tests==
==Automated tests==
* Everything we can have enabled in production is enabled, everything else is enabled on a separate branch called 'Ash'.
* We are at about 90% [https://wiki.mozilla.org/Electrolysis/Test_Coverage Test coverage for all channels. We have touched base with all module owners. Not all of them have signed off specifically but we are calling it within acceptable range for shipping at this point.
* We have all tests be owned
 
* We are still working on obtaining sign-offs
*We're tracking all the tests and who owns them, [https://docs.google.com/spreadsheets/d/10UeyRoiWV2HjkWwAU51HXyXAV7YLi4BjDm55mr5Xv6c/edit#gid=1777180571 here]
*We are tracking tests that are running and passing on some platforms, but are disabled on others.From a feature coverage point of view, it's in pretty great shape. There's very little that is disabled both on opt *and* debug on a same platform (meaning that that test is not running on this platform). Look for the "Fully disabled in a platform" entries in this [https://docs.google.com/spreadsheets/d/1oBnRct8FuTnAoVfbE08m10UHi1_5kOFaOPglJ6ittWc/edit#gid=0 spreadsheet].


==A11y==
==A11y==
* aklotz has published a demo of e10s working with A11y!
* It's looking like the initial API support will land in 50
* Next steps include shopping his build around to obtain feedback
* dbolter has engaged with add-ons team to determine existing add-on accessibility API usage


* We're also looking into Windows touchscreen/a11y compat, see more information as the plan is taking shape, [https://public.etherpad-mozilla.org/p/a11y+e10s here].
* We're also looking into Windows touchscreen/a11y compat, see more information as the plan is taking shape, [https://public.etherpad-mozilla.org/p/a11y+e10s here].

Latest revision as of 20:34, 1 July 2016

Green-sm.jpg


e10s Update: July 1

Executive Summary

  • The biggest news is that the stability release criteria for Beta 48 has been signed off on [1]. Slow script count is also within release criteria target.
  • We are also at 0 M9's (and have been) for a few weeks
  • As of this week, 569K users have been exposed to e10s in Beta 48.
  • Given that we have met all of our release criteria and we have not regressed since 47 beta, the project is now designated 'green' or 'on track' to ride the trains to GA and be gradually rolled out (1% at start) to eligible users (41% of total population) or about 800K users for the first 3 weeks.
  • So what's the release plan? We are working on a day-by-day snapshot of the activities that have to occur once Firefox 48 goes to GA. This is basically a small 'throttle team' who will be at helm of throttling e10s up or down (or disabling). This is driven by RelMan but will be supported by e10s subject matter experts so that decisions can be made quickly. You can refer to these docs for more info and WIP details: http://bit.ly/29mIbXs and


Next Steps

  • 05/20: Firefox 47 Beta 7 Ships
  • 05/25: Fourth derived data set available (4 weeks of data)
  • 05/24: Release plan review


Add-Ons + WebExtensions

  • Decision to not have an allow list has been made, we will take a more efficient approach. Only add-ons that have self identified as e10s compliant can be part of the e10s activation audience at first -in addition to all add-ons that are webextensions. See bug 1249060 for more details.
  • General roll-out plan for including extensions is here.

Automated tests

  • We are at about 90% [https://wiki.mozilla.org/Electrolysis/Test_Coverage Test coverage for all channels. We have touched base with all module owners. Not all of them have signed off specifically but we are calling it within acceptable range for shipping at this point.


A11y

  • It's looking like the initial API support will land in 50
  • We're also looking into Windows touchscreen/a11y compat, see more information as the plan is taking shape, here.

Milestones

We are at the point where we are setting milestones weekly; given the complexity of deliverables, we are using a Trackback Schedule to keep everything organized.

Release Schedule

Date Trunk Aurora Beta Release
3-07 48 default 47 default 46 A/B Tests 45 off
4-18 49 default 48 default 47 A/B Tests, Possible Rollout (add-ons = ally = no) 46 off
6-07 50 default 49 default 48 Possible Rollout (add-ons + ally = no) 47 off
8-02 51 default 50 default 49 Possible Rollout (add-ons + ally = no) 48 Possible Rollout (add-ons + ally = no)
9-13 52 default 51 default 50 Possible Rollout (add-ons + ally = no) 49 Possible Rollout (add-ons + ally = no)