Releases/Fennec1.0maemo/Post Mortem
From MozillaWiki
< Releases
Contents
Agenda
What went not so well?
- Lack of communication between QA and other groups (aakash)
- Starting with Beta 4, release dates weren't clearly communicated
- bugs weren't filed, just talked about (john+)
- Not enough visibility into time remaining for bugs
- overworked?
- bugs weren't acted on when triaged (bugs only worked on when assigned)
- bugs ignored when they should be acknowledged
- no backup for stewart
- not enough test cases for every patch
- no more meta bugs (tracking)
- Not enough communication between release and other teams, particularly around timelines
- Finding regression windows- time at the end of the year was dedicated to finding a regression after it's happened
- need to find regressions quicker
- disk cache parameter got turned on and took forever to find
- automated perf testing
- hard to know where we are in solving regressions
- need to find regressions quicker
- lots of last minute time working on flash, risk should not be pushed towards the end of a release
- lead times on release engineering are long, took one year to set up mobile linux builds, tri-server took 6 months
- hard to ship 1.0 releases because you're doing everything at the same time- everything is priority
- Launching multiple OS's at the same time sapped resources
- Delayed early on due to unstable platform and hardware availability
- N900's weren't easy to get until very late 2009/early 2010
- Need to figure out how to get hardware faster if we move towards bundling
- Spent too much time blocking ourselves on stuff
- Need to make decisions to cut features
- Need to be realistic regarding perf/launch features
- Lots of debate over choices which could be deferred and could be prioritized
- 1.0 was way too big
- Prioritized features + quality over time
- Scheduling between desktop + mobile for firefox and weave was tricky
- Pressure to make last minute design changes as we approach 1.0 release
- unit tests were ignored
- Not sure which devices to support (n810 vs n900)
- Firefox processes for mobile may not have been the best fit
- L10n communication was abrupt and cantankerous, crankiness--, should have worked more closely with team to solve problems
- Releng didn't have a good sense at 1.0 to understand expectations for launch (signing, crosschecking)
- "Ship when ready" is hard on non-engineering teams
- not enough warning on releases
- unclear to releng what was in each release
- more communication between mobile and platform teams
- tracemonkey drops would surprise mobile teams
- winmo was affected for 2 weeks as a result- need automation
- wrong people at thursday release meetings
- Bugs get forgotten, and people don't necessarily know the ramifications of their changes
- no automated testing, but anecdotal evidence wasn't taken seriously
- didn't do a good enough job listening to users and support forum, particularly early adopters
- good "getting started" documentation for n900 but not a lot of FAQ info (how to set default browser, manage plugins, etc)
- Standards around good add-ons for mobile were fluid and hard for developers to understand
What went well?
- Firefox has never shipped a multilocale release, lots of cross pollination
- learned a lot about shipping firefox on different distros which applies towards desktop builds
- we shipped!
- better than expected!
- built for quality instead of to a timeline
- made tough decision to remove flash for 1.0
- turnaround was fast for youtube addon
- dramatic improvements in process and communication over the last month
- marketing videos were great for happiness, high leverage with community
- good press communication
- great attitude towards problem solving and dealing with crises
- dev group was entrepreneurial in the absence of releng, mktg, l10n etc in early days
- those groups were understanding when introduced to team
- testing was great, amazing that we had 1 QA person
- versatile engineering team- "can-do attitude"
- first successful mobile firefox implementation
- everyone internalized the key goals of the project and kept that vision in mind
How will we change in the future? (themes for improvement)
- need communication/product human
- get technical contact at nokia for questions - stewart
- better product scoping of features, more tactical decision making around key features before spending too much time on them
- small prototypes of things more often
- better bug filing and triaging, goal of keeping things from slipping through cracks
- automated testing
- find regressions faster, automated perf testing
- try to get more hardware as fast as possible or alter processes to mitigate for releng
- try to tune process to be more mobile specific
- cascade launch date decisions more effectively
- get add-ons quality improved, set UI standards and set expectations regarding good add-ons
- smaller meetings with the right people
- dealing with qualitative feedback
- no more major holiday cancellations