e10s Weekly Update: Jan 15

Why yellow? e10s is designated 'yellow' or 'at risk' because:

1. Our plan of record is solid at this point in terms of release criteria. The Product Brief is still in flight but doesn't pose risk to the project; we don't anticipate any major characteristics in terms of value proposition to shift from current understanding. At this point, that doc serves as an informational reference for cross-functional teams as they communicate "the why". The release criteria is locked down and we will be flexible in order to put quality as #1.
2. Core team continues to working at a high velocity, we are down to 15 open M8's. See M8 Tracker to get granular details on what recent things have landed in Firefox 46, what's in review, and what we've chosen is safe enough to uplift for Firefox 45.

Decisions Made

• Over the holiday's the team met to go over risk vs. reward for 45 and we deemed Fx46 to be the next GA target for 50% of the population (who do not have add-ons or a11y installed). See the email announcing this, here.

POR Docs:

• Final Release Criteria
• Product Brief Draft
• Schedule

Validation for Stability and Performance

• Beta 44 A/B Telemetry Experiment ended on Jan 3. We found that the population included in the experiment included users with Add-Ons and people with a11y enabled. Both of which we originally intended to be omitted from the experiment. Here are some conclusions as posted to Dev-Planning: http://pastebin.com/SUT7WPkq
  • We need the BHR responsiveness measurement from the e10s content process in order to accurately compare e10s with non-e10s. bug 1228437.
  • We also have recently uplifted a patch to fix the fact that we're over counting crash reports for the content process bug 1227312.
  • At this point, we are working on the design for the Beta 45 experiment and being certain to curate the aforementioned uplifts (long with with others) that fix telemetry measures before we deploy to 45.
  • We are tracking various information, below:
  • Telemetry E10S Measurement Meta bug 1222849
  • Performance Tracker Meta bug 1198187
  • Plan for measuring e10s performance + stability, here.

• Beta 45 A/B Telemetry Experiment goal is to deploy as of Firefox 45 Beta 1 on January 27th
  • Jan 19: Experiment Design Review - send to Product for Approval
  • Jan 21: Hand off to QE for validation
  • Jan 26: QE Sign-Off
  • Jan 27: Deploy Experiment with Beta 1
  • Feb 1: Sanity check incoming data
  • Feb 5: Run existing data set

From there, we will identify a weekly milestone pertaining to results to ensure results are as visible and actionable as possible

Add-Ons + WebExtensions

• Add-Ons 'Office Hours' to help developers with real-time support is ongoing and was quiet over the holidays: http://atsay.github.io/e10s_office_hours/.
• We have a meeting next week with Product to review Use cases then UX/dev priorities/realities for roll-out selection capabilities for e10s with regards to add-ons (allow/deny lists).
• We need to make progress on creating a black list for incompatible add-ons bug 1233455. The first phase of the Telemetry A/B experiment launching on the 27th will be focused on curating data to help create this black list.
  • Source of truth for add-ons for priority and status on compatibility can be found, here.

Automated tests

• We're primarily missing 3 suites on Windows: mochitest-plain, reftest, and crashtest. Work needs to be done to green up each of these suites, so capacity issues are not currently the blocker.
• Making good progress on Win7 opt. He will update the tests status wiki page. Seeing a lot of issues on Win7 debug
• Next request is to get win7 tests running on aurora
• RyanVM will look at doing manual testing of the password manager since those tests are currently disabled and may be hard to get running.
• Here is a good test-by-test reference:

https://docs.google.com/spreadsheets/d/10UeyRoiWV2HjkWwAU51HXyXAV7YLi4BjDm55mr5Xv6c/edit#gid=1777180571

A11y

• Wiki to track work is, here.
• Currently focusing on test remoting of a11y calls from main process.
• Good progress is being made on a11y work in general; I will be tracking more closely once we're done with M8.

Release Schedule

Current Plan of Record is to Target 50% of the population who do not have add-ons installed or a11y features enabled for Firefox 46.

Milestones

Currently working on interim Milestones to track Telemetry Experiments