Thunderbird/Release Driving/Test Firedrill/Post Mortem
Post Mortem for the test firedrill: Thunderbird/Release_Driving/Test_Firedrill
Done via email/irc.
- We're looking for:
- How well did new processes introduced go?
- Any problem/pain points?
- Can we reduce time spent or speed up processes?
Schedule of Events
This is how we did on timescales (times are UTC):
|13:46||N/A||Original bug nominated (Standard8 was afk!)|
|15:45||0||Made final decision on a good bug and sent out go for firedrill|
|17:07||1h 22m||Bug fixed and landed. Go for build once tree had cycled green.|
|18:13||2h 28m||Tagging started|
|22:35||6h 50m||All unsigned builds completed|
|23:18||7h 33m||Signing started|
|01:19||9h 34m||Win32 signing complete|
Due to gozer needing to be afk for 3h, this is where firedrill ended.
Estimate for remainder of signing and update generation: 2h.
We'd have then been looking at approx 3h for final release (1-2 hours mirror uptake, 1h pushing website and announcements).
So probably could do a firedrill in 12h.
Leading up to the release (patch landing, code freeze etc)
- Picked the wrong bug for a drill! (no testcase/STR). So found a simpler one.
- Too many emails to tb-drivers, some of the needed detail got lost in the noise.
- Need steps to do items (build guide) and someone else in case gozer is afk or otherwise (2nd build engineer already looked for).
- Bouncer entries
- Could be an issue that if just TB is doing a firedrill at a weekend then may be difficult for getting bouncer entries added.
- Suggestion is that we submit new entries for the next release so that we have them there if we need them in a firedrill situation.
- Its possible, but we may need consider our own bouncer or access to MoCo's. Think this touches on existing bugs.
- May also be able to file blocker bug on MoCo IT.
- Standard8/gozer to consider.
- Did we do smoketests or just verify the bug? Does it matter if we're using the same revisions?
- No smoke test - I just made sure the bug was fixed and I could send emails, and That address book was unaffected (call that a ab smoketest - but that was done ad hoc).
- I would have done more testing if more than one bug would have been involved in the firedrill (and that would have been a complete smoke test)
- Do we have a guide/plan so that other people can do the job if Ludovic isn't about?
- NO but I can take an action item on writing what I did (send an email so I don't loose that fact)
Website (Release Notes, What's New, L10n issues etc)
- We didn't actually do this for real. However, based on 3.0.2, we could easily get pages updated in an hour or two whilst build is going on. We're getting well-versed in how to do this now.
- thunderbirdDetails.class.php could be the only sticking point as it needs review from MoCo webdevs.
- Worst case we could stop importing the product-details section and include it direct in our site. Then update the core version later.
- Alternately we could just land with review-pending and stick it in a bug.
Actual Release (Pushing out, updates, etc)
- We didn't do this.
- See the 3.0.2 post mortem page for a release we did the day after.
- Standard8 feels we need to get backups for everyone (who are familiar with the process) so that we can do hand-offs in the event of firedrill so that other folks can handle the process as well.