Thunderbird/Release Driving/Test Firedrill/Post Mortem

From MozillaWiki
Jump to: navigation, search

Post Mortem for the test firedrill: Thunderbird/Release_Driving/Test_Firedrill

Done via email/irc.

  • We're looking for:
    • How well did new processes introduced go?
    • Any problem/pain points?
    • Can we reduce time spent or speed up processes?

Schedule of Events

This is how we did on timescales (times are UTC):

Time Time Offset Description
13:46 N/A Original bug nominated (Standard8 was afk!)
15:45 0 Made final decision on a good bug and sent out go for firedrill
17:07 1h 22m Bug fixed and landed. Go for build once tree had cycled green.
18:13 2h 28m Tagging started
22:35 6h 50m All unsigned builds completed
23:18 7h 33m Signing started
01:19 9h 34m Win32 signing complete

Due to gozer needing to be afk for 3h, this is where firedrill ended.

Estimate for remainder of signing and update generation: 2h.

We'd have then been looking at approx 3h for final release (1-2 hours mirror uptake, 1h pushing website and announcements).

So probably could do a firedrill in 12h.

Leading up to the release (patch landing, code freeze etc)

  • Picked the wrong bug for a drill! (no testcase/STR). So found a simpler one.


  • Too many emails to tb-drivers, some of the needed detail got lost in the noise.
  • Need steps to do items (build guide) and someone else in case gozer is afk or otherwise (2nd build engineer already looked for).
  • Bouncer entries
    • Could be an issue that if just TB is doing a firedrill at a weekend then may be difficult for getting bouncer entries added.
    • Suggestion is that we submit new entries for the next release so that we have them there if we need them in a firedrill situation.
      • Its possible, but we may need consider our own bouncer or access to MoCo's. Think this touches on existing bugs.
      • May also be able to file blocker bug on MoCo IT.
      • Standard8/gozer to consider.


  • Did we do smoketests or just verify the bug? Does it matter if we're using the same revisions?
    • No smoke test - I just made sure the bug was fixed and I could send emails, and That address book was unaffected (call that a ab smoketest - but that was done ad hoc).
    • I would have done more testing if more than one bug would have been involved in the firedrill (and that would have been a complete smoke test)
  • Do we have a guide/plan so that other people can do the job if Ludovic isn't about?
    • NO but I can take an action item on writing what I did (send an email so I don't loose that fact)



Website (Release Notes, What's New, L10n issues etc)

  • We didn't actually do this for real. However, based on 3.0.2, we could easily get pages updated in an hour or two whilst build is going on. We're getting well-versed in how to do this now.
  • thunderbirdDetails.class.php could be the only sticking point as it needs review from MoCo webdevs.
    • Worst case we could stop importing the product-details section and include it direct in our site. Then update the core version later.
    • Alternately we could just land with review-pending and stick it in a bug.

Actual Release (Pushing out, updates, etc)

Anything else

  • Standard8 feels we need to get backups for everyone (who are familiar with the process) so that we can do hand-offs in the event of firedrill so that other folks can handle the process as well.