Releases:Release Post Mortem:2015-08-11

From MozillaWiki
Jump to: navigation, search

Firefox 39.0.3

  • Bouncer submitter failed because relbranch didn't have correct code(?)
  • Compilation failures in build1, had to respin for it
  • win32 build failed because of bad symlink in gecko repo
    • worked around by skipping symlinks in archiver when extracting on Windows
  • two windows repack chunks failed with slave disconnects
    • potentially related to master lag increases, maybe time to restart the masters?
  • shipping updates & post release done by hand because next 40.0 build had already started (would've shipped that instead)

Firefox 38.1.1esr

  • Shared a lot of issues with 39.0.3build1 ( bouncer submitter, compilation failures, symlink issues, slave disconnects)
  • Shipping updates & post release also done by hand, because 38.2.0's next build had already started

Firefox 40.0

Build 1 covered last week, builds 2 through 5 had more issues:

  • Lots of race condition issues with updates builders, affected multiple builds. Nick has a patch in bug 1192674
  • Had to manually add 40.0 partials to beta* channels, affected multiple builds.
    • Still need to fix this. Might just be a change to the version regex in release config?
  • Beta final verify failed, possibly because of bug 1190779
  • Windows update verify timed out when purging builds. Should be fixed by bug 1191352, but if that isn't coming soon we should do something in the meantime
  • Build3 needed a manual patcher config update because of 39.0.3 chemspill
  • Linux repack chunk failed due to bug 1191353.
    • Fix is waiting on review
  • XULRunner win32 sdk had to be repacked by hand, bug 1193177, thanks Nick!

Firefox 38.2.0esr

Build 1 covered last week, build2 had more notable things:

  • Special patcher config tweak needed because of chemspills
  • What's new page was requested in bug 1191899. Set it up on esr-cdntest, nobody tried to verify until 20min before ship.
    • Planning to bring up communication issues around WNP in post mortem
  • Manual steps in Balrog needed to ship, because we enabled updates for esr31 and earlier.