Releases/Thunderbird 3.0rc1/Post Mortem

Post Mortem for the Thunderbird release of 3.0rc1 / 3.0rc2 / 3.0 (DialIn Info)

What went well

• Get Satisfaction
  • Really useful for finding the early problems.
  • Suspect this also diverted some bugs away from bugzilla.

What could have gone better, and lessons learned

String Freeze

• Two weeks after previous release date felt too little for final freeze.
  • Suggest three weeks minimum or to be more locked down before final beta.

Code Freeze

• Not having clear feature freeze date made some decisions on patches slightly harder.
• When branching need to think about flags a bit earlier.

Build

• l10n tagging failed on invalid revision for the 'ka' locale, we need a way to test these revisions before it's time to build
• Need to sign-off on hg.m.o/build/tools and make sure the signing box is updated
• Do we need some sort of quick confirmation that all locales have completed successfully in-between locale repacks & signing?
  • e.g. in build 2 Standard8 found that the first linux (el locale) repack had failed pulling from hg. Only easily noticed by counting directories (buildbot history isn't good enough).
• Single build wiki page gets cluttered/confusing when reaching a 3rd build, for instance.

QA

• Not enough migration coverage
  • We missed duplications of folders bug 505465.
  • Auth issues for some configs.

• Didn't follow builds closely enough myself relied on driver for that.

• Missed a few spots on update testing for the last build.

Actual Release

• release-drivers weren't notified of release.
  • We didn't know about it until reed told us.
  • Apparently some members of the list are Linux distributions.
    • We do have some of our own Linux and non-tier1 OS system contacts, possibly needs expanding.
  • Do we need an equivalent list for TB? (and split the general driving from the actual release driving?).
• 1-day notice was possibly a bit short
  • Surprised some people.
  • We didn't notify justdave early enough and he then forgot for a while, and only notified the mirrors a few hours before the actual release.
    • Do we need to have something better in place here, e.g. a mailing list that we can send to as well?
• mozilla.com needs updating with thunderbirdDetails as well (bug needs filing at release time).
• we needed to coordinate w/ pascal & webdev folks which surprised me

Updates

• TB 2.x updates weren't initially 100% fully throttled due to a bug in aus (now fixed)
• TB 2.x updates are being offered to non-supported platforms (bug 534180)
  • Needs checking before release.
• RC1 updates not initially available for at least a week after final released.
  • Can we get some sort of automated aus checker, so that we enter the build id for each release, and check that we're getting served the appropriate updates? (and going back across all alpha/beta releases?)

Support

• Could have done with SUMOMO articles being able to be written on top-reported problems, so we could just direct folks to them.