E10s/Status/March25
e10s Weekly Update: March 25
Executive Summary
- Our current Plan of record is to continue to conduct A/B Experiments in our beta channel until we can lower our rate of content process and Plugin crashes. Our latest A/B experiment derived data set indicates we have work to do on stability before we're ready to ship to beta. We are, however, hitting our goals in many areas of our release criteria such as Page Load, Startup/Shutdown time, and Scrolling. We still have work to do in areas such as UI Smoothness, Plugin Jank, Memory Usage, and GFX performance. We have made excellent progress in our cadence of deploying A/B experiments and quickly ascertaining the data so we have hit a place where we are tightly coordinated and people are moving on issues swiftly. Another important thing to note is that we are currently using the System Add-On to deploy our A/B experiments so we are testing the mechanism we designed to ship e10s rather than using experiment code.
- The team has shifted to burning down M9's [1]. M9 tickets need to hit zero before we're ready to do our cohort release for GA. M9 is focused on fixing issues derived from our Telemetry A/B experiments such as top crashes, regressions, and backlog items deemed to be blockers by product. We have hit a stage where a portion of the M9's are the responsibility of teams outside of the e10s team so we will be rolling those tickets into central triage managed by RelMan to help with load balance.
- With the approach of Firefox 47 merging to Beta on April 18, market readiness remains our central focus.
Why yellow? e10s is designated 'yellow' or 'at risk' because:
Our release criteria indicates we have more work to do on the stability side. Given that we are at week 4 as of next week, our current call is that we will likely continue to do experiments in Beta 47 vs. releasing to a % of our cohort population.
Highlights and Accomplishments
We recently landed the following M8's in Fx47. Work is complete to deploy the second phase of the Telemetry A/B Experiments testing. We have some great analysis on incoming stability data from the Beta 45 experiment so far bug 1246180. Check out a summary of the engineering happenings this last week, here.
With the help of the quality team/SV, we have been conducting a backlog scrub of all the [2] issues being tracked for e10s. We've made quick progress to be sure there are not blockers or a combination small things that we would consider a negative trend and block to be found. We also wanted to be sure that we could close as many open items as possible. [3] See the report from SV for details.
Next Steps
As we continue to converge towards an eye for Firefox 46 beta, we'll need to start closely tracking M9's. As previously mentioned, focusing on our add-ons population and also being sure that we validate our measurements with real world proof is important. More details on how that will happen will be available soon. Solidifying that we have consensus and confidence in our roll-out plan is also an important next step and will be a major topic in the cross-functional kicking off next week.
Validation for Stability and Performance
- Phase 2 of the Beta 45 experiment is ready for deployment, see details, here http://pastebin.com/cNKAnW2d.
- We have solid data on stability and are taking action, see more [4] here.
- Jank results are positive thus far though we need to be sure to validate in the real world. See prelim details, [5] here.
- Information about add-ons which are causing jank is also incoming and we need to continue to ensure it's being validated. More on this, soon.
- Upcoming Milestones:
- Feb 12: Phase 2 starts
- Feb 25: Testing concludes for 45
Look here for more milestones tracking actions around incoming data/any adjustments we need to make
Memory Usage
- See report on Memory Usage which is mostly positive, here.
Add-Ons + WebExtensions
- Devs are looking for open bugs in webextensions API
- Working to converge SDK issues: https://docs.google.com/document/d/1v0tsxR2OS0eKY2mzDvadrkl-WTjNN-wr9qKU8rpJLa4/edit
- (Carry Over)We have a meeting next week with Product to review Use cases then UX/dev priorities/realities for roll-out selection capabilities for e10s with regards to add-ons (allow/deny lists).
- (Carry Over)We need to make progress on creating a white/black/grey list for incompatible add-ons bug 1233455. The first phase of the Telemetry A/B experiment launching on the 27th will be focused on curating data to help create this black list.
- Source of truth for add-ons for priority and status on compatibility can be found, here.
Automated tests
- Module owners need to be sure their tests are e10s compliant.
- Browser Chrome + and Mochi Test Plan (blocks). Need to get to 90%.
- Need to make sure that M-C and M-A are in lock step
- Goal to get all tests signed-off in M/C by Feb 29th
- More specific status available first week of Feb
- Here is a good test-by-test reference:
A11y
- Wiki to track work is, here.
- Currently focusing on test remoting of a11y calls from main process.
- Good progress is being made on a11y work in general; I will be tracking more closely once we're done with M8.
Release Schedule
Current Plan of Record is to Target 100% of the population who do not have add-ons installed or a11y features enabled for Firefox 46.
| Date | Trunk | Aurora | Beta | Release |
|---|---|---|---|---|
| 3-30 | 40 default (working on m5) | 39 off | 38 off | 37 off |
| 5-11 | 41 default (working on m6) | 40 prompt | 39 off | 38 off |
| 6-29 | 42 default (working on m7/m8) | 41 prompt | 40 off | 39 off |
| 8-10 | 43 default (working on m8) | 42 default | 41 off | 40 off |
| 9-21 | 44 default (release criteria driven) | 43 default | 42 off | 41 off |
| 11-02 | 45 default (release criteria driven) | 44 default | 43 off | 42 off |
| 12-14 | 46 default (release criteria driven) | 45 default | 44 A/B | 43 off |
| 1-25 | 47 default (release criteria driven) | 46 default | 45 A/B | 44 off |
| 3-07 | 48 default (release criteria driven) | 47 default | 46 A/B | 45 off |
| 4-18 | 49 default (release criteria driven | 48 default | 47 (addons & a11y = off) | 46 off |
Milestones
We are at the point where we are setting milestones weekly; given the complexity of deliverables, we are using a Trackback Schedule to keep everything organized.