Releases/Fennec1.0maemo/Post Mortem

Agenda

What went not so well?

  * Lack of communication between QA and other groups (aakash)
     * Starting with Beta 4, release dates weren't clearly communicated
     * bugs weren't filed, just talked about (john+)
        *  Not enough visibility into time remaining for bugs
     * overworked?
     * bugs weren't acted on when triaged (bugs only worked on when assigned)
     * bugs ignored when they should be acknowledged
     * no backup for stewart
     * not enough test cases for every patch
     * no more meta bugs (tracking)
  * Not enough communication between release and other teams, particularly around timelines
  * Finding regression windows- time at the end of the year was dedicated to finding a regression after it's happened
     * need to find regressions quicker
        * disk cache parameter got turned on and took forever to find
        * automated perf testing
     * hard to know where we are in solving regressions
  * lots of last minute time working on flash, risk should not be pushed towards the end of a release
  * lead times on release engineering are long, took one year to set up mobile linux builds, tri-server took 6 months
  * hard to ship 1.0 releases because you're doing everything at the same time- everything is priority
  * Launching multiple OS's at the same time sapped resources
  * Delayed early on due to unstable platform and hardware availability
     * N900's weren't easy to get until very late 2009/early 2010
     * Need to figure out how to get hardware faster if we move towards bundling
  * Spent too much time blocking ourselves on stuff
     * Need to make decisions to cut features
     * Need to be realistic regarding perf/launch features
     * Lots of debate over choices which could be deferred and could be prioritized
  * 1.0 was way too big
     * Prioritized features + quality over time
  * Scheduling between desktop + mobile for firefox and weave was tricky
  * Pressure to make last minute design changes as we approach 1.0 release
  * unit tests were ignored
  * Not sure which devices to support (n810 vs n900)
  * Firefox processes for mobile may not have been the best fit
  * L10n communication was abrupt and cantankerous, crankiness--, should have worked more closely with team to solve problems
  * Releng didn't have a good sense at 1.0 to understand expectations for launch (signing, crosschecking)
  * "Ship when ready" is hard on non-engineering teams 
     * not enough warning on releases
     * unclear to releng what was in each release
  * more communication between mobile and platform teams
     * tracemonkey drops would surprise mobile teams
     * winmo was affected for 2 weeks as a result- need automation
  * wrong people at thursday release meetings
  * Bugs get forgotten, and people don't necessarily know the ramifications of their changes
  * no automated testing, but anecdotal evidence wasn't taken seriously
  * didn't do a good enough job listening to users and support forum, particularly early adopters
  * good "getting started" documentation for n900 but not a lot of FAQ info (how to set default browser, manage plugins, etc)
  * Standards around good add-ons for mobile were fluid and hard for developers to understand