Firefox/Channels/Postmortem/29

From MozillaWiki
Jump to: navigation, search
« previous release | index | next release »

Firefox 29 Post-mortem

Talking Points

  • the numbers on the webwewant site — why so high? how can we get a breakdown of what they were based on?
  • Web We Want site data:
    • manual downloads (we were counting on restarted downloads? every request on download page?) -- querying zeus logs from d.m.o -- were they all unique? segmenting?
    • auto update pings
    • people who share the site

Timeline

29.0

  • Pushed to mirrors on the day before release so we could ship at 6am PST Tuesday April 29th
    • Caused some confusion, didn't get the message out to enough functional areas to prepare for this
    • People happened to manually trigger and see new version
    • There were 404 in product for early updaters
    • Should consider having something other than 404 in the future that is less confusing/scary to users
  • Went smoothly, we hit the target ship time within ~10 mins
  • Last minute fix for bug 988156 (fullscreen netflix) in 29
  • Took an unnecessary uplift for bug 991767 (too late for much testing) and resulted in bug 1003707 (pdf printing blank page)
    • Need to be more averse to last minute stability fixes that aren't verified on a previous beta
    • Look at crash fix bugs closer in the end days for areas that were impacted by the fix
    • In this case, no one used the STR which would have highlighted the issue -- on 30 we only watched for reduced crash data

29.0.1 Desktop

  • respin Desktop for bug 1003707 (pdf.js) - non urgent dot release
    • all desktop fixes: bug 1003707,bug 1001167,bug 907373,bug 1005958
  • 29.0.1 release with background updates enabled 100% on Friday May 9th

29.0.1 Mobile

  • respin for bug 1005074
   bug 1005074 Re-enable 'Send Tab to Devices' on Beta and Release with a string swap 
   bug 987867 JB & KK crash in java.util.ConcurrentModificationException: at java.util.LinkedList$LinkIterator.next(LinkedList.java)   
   bug 1003897  Character Encoding selection broken in Fennec 29 
  • Shipped to Google Play on Friday May 9th

Final Beta being RC

  • Worked well for getting stability feedback

QA

Desktop

Mobile

RelEng

Security

Marketing

  • April 30th saw the largest spike of manual downloads since Firefox 4 days.
  • The download/updates/share counter was removed from the webwewant.mozilla.org on May 12th after it passed 500,000,000.
  • Firstrun and whatsnew tours are looking great from a data perspective and got lots more data after 29.0.1 was released on Friday.
  • Mozilla.org is seeing +95,000 active users on the site every minute and it is mostly /whatsnew/ and /firstrun/ tours from users upgrading to 29.0 and 29.0.1.
  • Funnelcake test data is looking good and we are seeing a positive ADI impact for users who saw a firstrun tour.
    • noscript installed will not trigger tours.
      • Was this tested?
    • ESR whatsnew will need to trigger tours in Fx 31
      • Will mention this to habber/jbertsch

Other

  • Stability: We went out well, but the scrambling to get things fixed on beta took us right to the last minute. Hope that doesn't happen too often.

User Advocacy

  • Can we make sure to QA things like high contrast mode in windows in the future?
    • let's also bring in a11y team, UX team so it's early in the design of new UI
  • Also, figuring out what would be worth it as a "driver" for a patch. Broken sessions = dataloss but that wasn't a "driver". Unreadable tabs is also a bigger problem and we probably should have waved more flags. Maybe tighter tracking of what we're agreeing is /not/ getting fixed?
  • We should have caught a lot more during the beta period but Eichgate happened. So, we should be aware that the lack of a full beta leads to a lot of added risk.

Notes

  • qa testing pdf print - can we get a bug on this (if not one already?)
    • it's in the test plan, we could have (and in future) should
  • why did we uplift a patch that didn’t have crashes on 29 (pdf)?
    • multiple factors - different methods for verifying (based on lack of crashes vs. STR)
    • based on how inflated the numbers turned out to be, release drivers should never rely on data from marketing tools for our decision making on update enabling/disabling
  • what do people think about always pushing to mirrors on Monday?
    • manual checkers will always get update

Actions