Call in details

• Mozilla Mountain View (The Bridge)
• 650-903-0800 or 650-215-1282 x92 Conf# 8605 (US/INTL)
• 1-800-707-2533 (pin 369) Conf# 8605 (US)
• irc.mozilla.org #planning for backchannel

Agenda

Key question: how can we do this even better next time?

How did Firefox 3.6 measure up to the goals for the release?

• faster development cycle (less than 1 yr for a meaningful product)
  • done...6.5 months
• deliver everything required so mobile team could ship their product the way they wanted
  • done
• improve startup time and other perceivable ways of tracking speed
  • definitely faster than 3.5, but didn't really hit goal on startup time. "Perception of performance" goals were a mixed bag.
• other comments:
  • define release criteria & goals more clearly up front
  • PR messaging around faster development cycle gets tricky - reporters start writing stories about a delayed release once it slips past initial expectations, so let's change the story to be more about why we're doing the faster releases (better for users, better for the web, etc)
  • nebulousness around ship date (may have) caused extra work/confusion for add-ons team...ex. Personas integration w/AMO site

What was good about the delivery schedule?

• props to PR efforts for rolling with the schedule changes and being ready at all times no matter what
  • and props in return from the PR team for really quick turnarounds & approvals on top priority issues
• rapid beta cycles worked very well - makes every part of the change management process easier
  • props to Build & QA teams for helping make this happen
• good work building up beta testing community very quickly (using tactics like Google snippets, placements on key mozilla.com pages, community marketing, etc)
• general consensus that having more betas & quick beta cycles was good, and we should consider changing the naming structure for next time ("beta version 1, 2, 3, etc" vs "beta 1, beta 2, beta 3, etc")
  • one potential drawback is that this could affect adoption among users (if they don't sense that betas are really changing)
• lack of changes on the front end made it easier to have a rapid beta cycle
• PR: held dev news/moz blog posts until the best time for worldwide impact rather than pushing them out right away (ex. waiting until Monday morning to announce rather than doing it late on a Friday night)
• AMO compatibility issues didn't cause major release problems

What caused delays in the delivery schedule?

• late-breaking file API changes slowed things down
• new heightened focus on Crash/Kill issues pushed back the release date
  • at the very least, it took a lot of engineering attention...results were good, but it impacted ability to ship on time
  • improvements made by the Crash/Kill process were worth the extra month of dev time before the release, though (next time let's start focusing on this at the beginning of the cycle rather than 3/4 in)
  • the increased number of blockers caused by Crash/Kill created a perception that the release was further away...also invited scope creep
• FFTs before we ship take a long time...amounts to something like 5% of the overall ship schedule
  • we should take a look at the last round of manual verifications and figure out how we can condense that
  • focus more on the incremental changes per beta
• adding features later in the process causes churn in QA - having to write new test cases, etc - and distracts them from the work of shipping betas, etc
• tegra project took away QA resources...should have asked for time estimate up front rather than just assigning the work without a full understanding of impact
• still feels awkward and surprising that we're still finding sites that are broken with our JS engine even at beta 5 or RC (a big cause of respins late in the game)
  • some of this was b/c of changes later in the process (ex. beta 5) that caused the problems...need to be aware that these changes may have other repercussions (although most of the time the changes have to be made)
• missed some Facebook compatibility issues, but a lot of that was on their side

What infrastructure improvements helped

• automated testing, branch mechanics have helped
• having dirty profiles was incredible...helped us catch a lot of things
• TS testing very helpful
• lots of of ways of understanding the effects of code changes we were making as we were making them
• kept Firebug compatible throughout all beta cycles

What infrastructure aspects could still be improved?

• automated testing for add-ons
• would be great if there was a suite of Firebug tests we could put into the tree so we could understand when we're breaking Firebug (Rob has a patch for this that we could build on)
• could we get add-ons developers to run our automated tests?
  • we have lots of tests that we could use for add-ons...would be great to know where the compatibility problems are
• in terms of l10n, it took a long time to get people's code from 1.9.1 to 1.9.2 - getting people to translate their stings on branch rather than trunk

Effectiveness of PR and marketing efforts, and things that could be improved

• full marketing postmortem details
• Beltzner got to sleep in his bed every night! (re: press briefings over the phone...lots of them, but travel not required))
  • accelerated dev cycle made this possible
  • new approach for briefings - getting in front of people who syndicate the most (good reaction to changing & consolidated media climate...things are different now than with 3.0 or 3.5)
  • having the "What's New in 3.6" video ready was good
• European PR - we briefed too early...need to time briefings to RCs (not betas)
• Air Mozilla on launch day was an effective community marketing tool - made people feel included
  • but, we could only get 200 people on at once
• were able to maintain community enthusiasm without a bunch of launch parties (especially good for a smaller numbered release)
  • active engagement with the community marketing list helped
• good work delivering mozilla.com content to l10n well in advance
• Personas uplift was tricky with l10n - lots of content changes, hard to get localizers to buy in
• big marketing/PR push for Personas...lots of adoption so far (ex. key site placements, Personas video, PR efforts, community marketing)
• need better communication with l10n over localized content (Mozilla Europe, Mozilla Japan, etc)...particularly wrt release timing

How things went on "ship day", and what could be improved

• people were a little caught off guard that we were shipping so soon once the decision was made
• overall, a much smoother and lower-stress release day than any previous ones (exp. vs Fx 2, 3 and 3.5)
• ability to announce release took longer than expected b/c Mozilla Europe was updating their server - infrastructure difference slowed us down there
• cut down on last-minute minor site changes
• communication over website delivery methods got bumpy - would be good to have a dedicated webdev/IT person watching over from the technical side so we'd have a centralized resource
• use different file names for key graphics (like wordmarks & logos) to make changes more clear and easier to track
• more details coming from Reed via email
• again, Air Mozilla was a very nice touch and added a lot
• we have a lot of channels - email lists, newsgroups, etc - for announcing a release. Can we consolidate?