Firefox 41 Post-mortem

• (sylvestre) First release without Ryan
  • Sometime, hard to get proper sheriff coverage
  • (lmandel) autoland to inbound coming this quarter, next year autoland to branches

• (ritu) It took a lot longer to get Beta1 build ready. Post-merge there were several tests that were failing. Can these be found before merge?
  • (ritu) Most of these issues came due to addon signing was required.
  • there should be a try run before we land the change on aurora (during merge?) We could do this far in advance on a separate branch that is a mirror of beta

• (ritu) Add-on signing pref was changed one week before entering Beta and it led to 8 bugs.
  • Especially a lot of test related failures
    • (lmandel) for features we think have a chance of being backed out we can/should run tests with the pref on and off (like we do for e10s) - need to be careful about capacity issues though
  • We should try to land such changes earlier in Aurora cycle
    • We could explain the risks in timing more clearly & expectations for what happens during aurora
    • I'm not sure how to avoid this with future pref flips (for example, we have 2 upcoming end of aurora prefs to disable at least. We know it's happening ....
      • Remember to disable ICE TCP https://bugzilla.mozilla.org/show_bug.cgi?id=1206981
      • Make sure Push notifications api is disabled
    • Ritu makes the point it is different turning a feature ON when it hits beta. In that case we need more testing.

• (ritu) Patches approved but still miss timely landing in Aurora/Beta
  • https://bugzilla.mozilla.org/show_bug.cgi?id=1181907
  • (lmandel) better communication with sheriffs?
  • Need automation/reports (We could extend the relman nag email to do this - liz)
    • KaiRo mentioned that this automation already exists - not sent to sheriffs though

• (ritu) Fennec crashes may need closer monitoring as this was deemed a release blocker post Beta9 and was not mentioned in channel meetings before that
  • https://bugzilla.mozilla.org/show_bug.cgi?id=1194923#c37
  • Kevin says it may not have been a blocker, just that an engineer had a nexus 6. how do we decide which hardware platforms will be blockers (user impact)
  • What are our most popular fennec devices?

• (ritu) Disabling addons signing pref in last beta was risky
  • led to increase in startup crashes owing to two addons at least
  • https://bugzilla.mozilla.org/show_bug.cgi?id=1204324#c8
  • Add-ons that signing requirements blocked were suddenly enabled and caused or uncovered issues (some in our code that the add-ons just triggered)

• (lizzard) Last minute questions about blocking release based on regressions in bookmarks ui

• (KaiRo) Milan says we lost 2 days for gfx crash investigation post-release (for 41.0.1) because the gfx team was not directly contacted for those 2 bugs. We should ni? and/or directly email him right away for things that need action for a dot release.

• (KaiRo) The Flash 19 release very shortly before our release bit us this time as the freezing issues depended on the combination of 41 or higher with Flash 19 - usually this hasn't been an issue, but this time was different.
  • Want to suggest adding this to beta release checklist going forward.

• (KaiRo) We saw once again that we need better ways to deal with startup crashes.
  • in general or with malware?
  • dynamic updating of blocklists and silent updates getting rolled out would help prevent the dot releases.