Releases/Firefox 9/Ship options

There has been some previous and more recent discussions about Firefox 9. We need to decide on a release plan. I provide a couple of options here, some pros and cons, and some basic assumptions. Please feel free to correct what I have written, suggest other options, or provide additional insight.

Basic Assumptions

1. We want to release Firefox 9
   • This was established at a previous meeting. Firefox 9 has some good fixes and we would like to release it
     
2. We will have resources available for a chemspill December 20th-December 23rd
   • I have gotten explicit resource commitments from QA, IT, RelEng, product management, product marketing, JS (including Brian for TI), Press, release management, and Firefox front end, and platform

Below are some options and some of their pros and cons. The pros and cons are not exhaustive. Option #4 was generally discounted in a previous discussion.

Option 0 - "Standard release"

• Details
  • Normal release process

Option 1 - "Release audience testers"

• Details
  • 1 week before release when we have the final mozilla-beta build (and before the mozilla-release build) we push the final mozilla-beta build to some percentage of the release audience to get ~15 million Firefox 9 users for a week before the full / unthrolled / normal release on December 20th. We would *not* update the users on the final mozilla-beta to the mozilla-release build if no issues require a rebuild. We could choose to throttle / slow roll on December 20th as well.
• Official ship
  • December 20th
• Benefits
  • Hopefully will prevent a December 20th chemspill due to finding issues early
  • Still have time to rebuild and hit a December 20th date / no "slip"
  • Press team likes that we'd release on the 20th
• Downsides
  • Confusion if Firefox 9 is "released" for some percentage of the population
  • Using users as "testers" that have not opted into testing
    • Note this isn't a major concern as we think the final beta is done / of release quality
    • This is a horrible messaging proposition. If people come to us confused about why they got moved up, we say something along the lines of "we decided to make you a guinea pig without your permission". -- Cww
  • May be mechanically difficult as far as builds go (RelEng to comment)
  • May be mechanical considerations (crashes get grouped in a beta product, different crash ids in crash-stats are treated differently, increases QA's test matrix, etc)
  • Users on the mozilla-beta Firefox 9 will not be eligible for the Firefox 10 patch/partial update
  • Some fraction of users have almost nil 3rd party support. (We're not officially released so 3rd parties have no incentive to update their software for a week) -- Cww.

Option 2 - "Slow release ramp"

• Details
  • We ship Firefox 9 on December 20th as planned, though we throttle automatic updates to some small percentage to mitigate the impact of a 9.0.1 on the entire Firefox audience
• Official ship
  • December 20th
• Benefits
  • Limit the negative impact of a 9.0.1
  • No confusion about Firefox 9 being released or not
  • Hit a December 20th date / no "slip"
  • Press team likes that we'd release on the 20th
• Downsides
  • Using users as "testers" that have not opted into testing
    • Note this isn't a major concern as we think the Firefox 9 is of release quality
  • Increased risk of not finding or fixing issues in a 9.0.1 until the holidays, potentially impacting humans

Option 3 - "Delay / postpone"

• Details
  • We ship Firefox 9 in January, messaging why the "delay" as much as we can
• Official ship
  • Early January
• Benefits
  • No human impact for the holidays
  • More time to test / perhaps prevent a 9.0.1
    • Note that Beta would move to 10 or it would negatively impact 10 testing, so there will be minimal 9 testing at that point
  • Less/postponed update fatigue as there will be more time between Firefox 8 and 9
  • More time to potentially get add-on compatibility up
    • Don't expect much/any due to the end of year
• Downsides
  • We'll seem like we are slipping to some percentage of people, likely in the press as well
  • If we have to do a 9.0.1 potentially impacts 10 w.r.t update fatigue
  • No guarantee that issues causing a 9.0.1 would be prevented by waiting / may be no real benefit for breaking the "every 6 weeks" expectation

Option 4 - "Push the release up"

• Details
  • We ship Firefox 9 one week before December 20th
• Official ship
  • December 13th
• Benefits
  • Less chance of human impact for the holidays
  • More time to test / perhaps prevent a 9.0.1
• Downsides
  • Less testing on Beta
  • Less chance of 3rd parties being ready for release

Option 5 - "Delay the entire release train"

• Details
  • Delay the entire release train by a couple of weeks by a couple of weeks
• Official Ship
  • Early January (3rd or 10th)
• Benefits
  • All the benefits of option 3
  • We set a precedent
    • If a 6-week rapid release point falls within x distance of Christmas, then we'll shift it
    • In future years, we can plan this up front (e.g. if we don't change anything, then next years release & merge would be 1st January - I don't see that happening either).
  • Developers still get 6 weeks to land things in the normal cycle - (assuming they are actually on holiday for the Christmas period!)
    • i.e. the cycle after 20th Dec, we're going to loose about 1.5 weeks of dev time due to the holiday period. I don't know quite how much that will equate to, but I could see less being in the next release. There may be a little bit more get in, but I don't think that matters
  • If we've delayed the rapid release cycle, if we need a 9.0.1, then we're not going to be up against 10.0 and update fatigue
• Downsides
  • We will still look like we are slipping and/or it'll look like we're not keeping to the original premise of the train runs on time
  • If we say we're doing this regularly due to avoiding releases around the holidays, that's just common-sense, isn't it?
  • Less predictability of release & merge dates around year end
    • However, we could mitigate this by looking at the dates for the next two or three years now, and publishing a calendar. If we've a consistent rule, then it makes it easier.

Option 6 - "Manual only release"

• Details
  • We only release the manual updates on mozilla.org and the MU / Check for Updates. Automatic updates are disabled for desktop (obviously mobile would be fully released) until Jan. I think this is mechanically possible, not 100% positive.

• Official Ship
  • December 20th (note that anyone seeking out the update will get it, so we are officially shipped)

• Benefits
  • Hit our ship date
    
  • Slow ramp
  • 9.0.1 has less urgency / impact (we don't normally yank the builds from mozilla.org in chemspills)
  • The positive: We wanted to have a release but understand that a lot of our users may have a hard time updating during the holidays. This way we give you the choice entirely. -- Cww.
• Downsides
  • Fewer users see the benefits of Firefox 9 during late December
  • May not find issues that would require a 9.0.1 until automatic updates are enabled
  • What do we do about security announcements?
  • Possibly some noise about slow Firefox 9 uptake 
  • Most of the additional testing would be on the pave-over / new install path, not updates
    • There would still be some updates due to the manual check for updates, but not much

Preferences (in order)

• Christian: Option 1, 0, 6, 4
• Callek (behalf of SeaMonkey): Option 0, 6, 1, 2
• Cww: 6, 2, 0, 3, 5
• jprmc: 6, 1, 3, 5