From MozillaWiki
Jump to: navigation, search
« previous release | index | next release »

Firefox 40 Post-mortem

  • (lmandel) beta5 build failed and we failed to ship this beta
  • (lmandel) beta6 build was hung, cpearce noticed
  • (lmandel) beta4 mobile failed to update on the Play store, required poking Google for help
    • (ritu) This may happen from time to time.
      • (sylvestre) Do we know what happened? Our update gets queued in their review process and their standard turn around time is ~24 hrs.
    • (sylvestre) I will have some releng permissions to retrigger jobs
  • (lmandel) Difficult to manage the release as Sylvestre was on PTO and Liz was out sick. Should have better relman coverage once Ritu is fully up-to-speed.
    • (sylvestre) Thanks again! (lizzard) Yes thanks!!
  • (bhearsum) Consistency in release urgency. If we're burning the midnight oil to get releases built, we shouldn't be sitting on them.
    • Eg: we sat on 39.0.3/38.1.1 chemspills for 2-3h before shipping (waiting on advisories, I think)
      • (lizzard) Yes - that was partly that dan and al were traveling, partly that I overestimated how much time we would take to qualify the release -- we finished earlier than I predicted, and the sec advisories weren't ready.
   todo:   document how to publish the sec advisories. (pmac, jgmize, www channel)
      • Advisory could be managed by others
      • (hwine) might have a new request for moc notification of actual release
      • (sylvestre) Release management should communicate more when the situation is complex
    • Having a schedule before or right after a release starts may help set expectations more clearly
  • (bhearsum) How to know if we're doing a what's new page? We were told no in, then yes, but it turned out we were just doing a firstrun (no RelEng work needed)
    • Related, verification of WNP (what's new page). Verification for ESR didn't happen until Tuesday morning, and it delayed ship (we had an issue on Mac)


  • (bhearsum) should we hold off publishing security advisories until after we unthrottle? Not all users are offered updates when we first ship (because of the 25% rate), and if we have an immediate point release it can sometimes be a week before many users get an update even though we've posted advisories
  • (bhearsum) tiny reminder: "snippets" don't exist anymore!
  • (hwine - developer edition) some issues when 42a2 went live with download page links not updating. With increasing complexities of coordinating multiple artifacts, maybe time to add a "are we really shipped yet" checklist?
    • this specific instance was we pushed updaters live on a Friday Aug 14, in #releng I got flagged that the "install" links still showed 41a2 at -- iirc everyone checked the updates before leaving, no one checked the installer.
    • (lmandel) +1
  • (sylvestre) async plugin init hotfix
    • no clear owner
    • process is unclear

we have now updated some of the docs for this.

  • (sylvestre) 40.0.3 was smooth
  • (bhearsum) - notify of unusual releases (ie, point releases)
    • (sylvestre) we should cc them when doing a dot release.
  • (lizzard) socorro (crash-stats) didn't add 40.0.3 to its dropdown menu for tracking crash rates in the graphs (lars is investigating)
  • (robert) User feedback:

(kglazko) Survey results, better planning, more discussion about backporting refactoring/fixes, good collaboration between devs and testing this time, quick uplifts.