Releases/Thunderbird 2.0.0.21/BuildNotes

Build Engineers

nthomas
Version/config bump/tracking in bug 480864

Bonsai queries

Build data

Type	Build machine
[Windows installer/zip]	production-patrocles
[Mac compressed]	bm-xserve05
[Linux compressed]	production-crazyhorse

Notes

Build 1

Using RELEASE_AUTOMATION_M13, same as Fx3.0.7 but new for Tb. Leaving Tinderbox on RELEASE_AUTOMATION_M11 (don't need bug bug 413491).

Slave cleanup

production-prometheus-vm
- need ~10GB for tag, source, update generation and linux verify, stage; removed /builds/updates/thunderbird-2.0.0.19 to leave 14GB free
- 'ssh -i .ssh/ffxbld_dsa ffxbld@hg.mozilla.org' to accept key fingerprint (one off)

production-crazyhorse
- 21GB free on /builds
- update tinderbox to automation tag
  - cd /builds/tinderbox/mozilla/tools/tinderbox; cvs up -CPd -r RELEASE_AUTOMATION_M11

bm-xserve05
- need ~20GB for mac builds, l10n_verify, mac update verify (but 13GB already allocated for builds); removed /builds/verify/thunderbird-2.0.0.19/ to leave 15G free
- update tinderbox to automation tag
  - cd /builds/tinderbox/mozilla/tools/tinderbox; cvs up -CPd -r RELEASE_AUTOMATION_M11
- 'ssh -i .ssh/ffxbld_dsa ffxbld@hg.mozilla.org' to accept key fingerprint (one off)

production-patrocles
- 50GB free
- update tinderbox to automation tag
  - cd /cygdrive/e/builds/tinderbox/mozilla/tools/tinderbox; cvs up -CPd -r RELEASE_AUTOMATION_M11

production-pacifica-vm
- 30GB free, plenty for update verify
- 'ssh -i .ssh/ffxbld_dsa ffxbld@hg.mozilla.org' to accept key fingerprint (one off)

Land changes and retag tb-moz18-bootstrap.cfg
- cvs tag -d RELEASE_AUTOMATION_M13 tb-moz18-bootstrap.cfg # to delete the tag on the old version
- cvs tag RELEASE_AUTOMATION_M13 tb-moz18-bootstrap.cfg # to tag the new version

Setup master before starting:

update mozilla/tools/buildbot-configs/automation/production/tb-master.cfg for most recent milestone of bootstrap
as buildmaster@production-1.8-master, cd /builds/buildbot/Thunderbird-Automation/buildbot-configs. Check for local diff, revert any changes from last release, dry run a cvs up, then cvs up
master.cfg is "Locally Modified" to contain passwords; identical to repository version otherwise.
Ran 'buildbot reconfig /home/buildmaster/Thunderbird-Automation'
Buildbot waterfall is here (MPT-VPN); this is a separate instance from the Firefox master, with production-prometheus-vm and bm-xserve05 running a slave for each master
Stop nightly builds on bm-xserve05 (~cltbld $ buildbot stop /builds/slave/) and annotate Mozilla1.8 tree to explain its absence

Start automation run:

buildbot sendchange --username=nthomas --master=localhost:9990 -m"Thunderbird 2.0.0.21 build 1" release

Tag

config/milestone.txt was not bumped (we expect Firefox to have already done this). Manual fix (done between /cvsroot and /l10n tagging)

cvs up -r GECKO181_20090302_RELBRANCH milestone.txt
# edit and remove pre to have '1.8.1.21'
cvs ci -m "Bug 480864, update milestone.txt for Tb2.0.0.21 (on GECKO181_20090302_RELBRANCH)" milestone.txt
cvs tag -F THUNDERBIRD_2_0_0_21_BUILD1 milestone.txt
cvs tag -F THUNDERBIRD_2_0_0_21_RELEASE milestone.txt

bug 481079 to fix this up.

Source

Automated, no problems

Build/Repack

Automated, no problems

Sign

Signing doc

l10n verification

Failed with

Step Repack died: No such config variable: hgToolsTag at Bootstrap/Config.pm line 93.

Added 'hgToolsTag = default' to tb-moz18-bootstrap.cfg, and moved the RELEASE_AUTOMATION_M13 up to the new revision. No problems on the second attempt.

Generate updates

Automated, no problems

Publish updates to test channels (betatest & releasetest)

Automated, no problems

Update verify

Mac & Windows bombed out when pulling build/tools from hg - need to investigate. Stopped Linux because all the update verify configs are missing 2.0.0.16/17/18.

Coop fixed the configs, restarted all three platforms. Win32 bombed out with key problem pushing config change back to hg.m.o, fixed hgSshKey in tb-moz18-bootstrap.cfg and 3 other 1.8 configs and restarted win32.

All were green.

Push updates to beta channel

$ sudo su - cltbld
$ cd /opt/aus2/snippets/staging
$ time ~/bin/backupsnip 20090302-Thunderbird-2.0.0.21-beta
real    41m7.376s
user    0m37.186s
sys     1m5.541s

Waited for formal "go" to beta.

$ sudo su - cltbld
$ cd /opt/aus2/snippets/staging
$ time ~/bin/pushsnip 20090302-Thunderbird-2.0.0.21-beta
real    1m24.075s
user    0m0.099s
sys     0m5.007s

Sign installers

Done manually using these installer-signing-instructions here
complete stage-merged:

# on stage
cd /data/cltbld/thunderbird-2.0.0.21/
rsync -av batch1/mar/ stage-merged/
rsync -av batch1/stage-signed/ stage-merged/

Create MD5 and SHA1 checksum files

# on stage
cd /data/cltbld/thunderbird-2.0.0.21/stage-merged/
~/bin/checksum-files .

Fix permissions & ownership (on the two SUM files, and the detached sigs)

chown -R cltbld:thunderbird .
chmod 644 *SUMS

Update bouncer

Manually added entries to download.mozilla.org
Note for next release: Do not remove the Check Now bit on the Thunderbird-2.0.0.21 Products until well after the change to the rsync module (to prevent the likes of bug 464566)

Push to mirrors

push the stage-merged directory to the releases area:

# on stage (as cltbld)
rsync -av /data/cltbld/thunderbird-2.0.0.21/stage-merged/ /home/ftp/pub/thunderbird/releases/2.0.0.21/

Update rsync module

On stage.mozilla.org, add the new release to the smaller, faster, rsync module:

# on stage (as cltbld)
cd /pub/mozilla.org/zz/
vi rsyncd-mozilla-current.exclude

and replace the previous latest release (Tb2.0.0.19) with:

+ thunderbird/releases/2.0.0.21
+ thunderbird/releases/2.0.0.21/**

Final verification

Verify that releasetest points to valid bouncer links:

hg clone http://hg.mozilla.org/build/tools
cd tools/release/updates
cat moz18-thunderbird-*.cfg | grep -v major | sed 's/betatest/releasetest/' | grep -v 2.0a | grep -v 2.0b > update.cfg
./verify.sh -t update.cfg 2>&1 | tee quickVerify.log

Look for any HTTP error codes besides 200 ("OK") and 302 ("Found"):

grep HTTP quickVerify.log | grep -v 200 | grep -v 302

Found that www.mirrorservice.org was returning 404s. Turns out there are 6 machines in DNS for this and one of them was stale. Can't raise the admin on irc or email, disabled in bouncer.

Before pushing final updates,verify that "release" and "releasetest" channel match:

# on aus2-staging
cd /opt/aus2/snippets/staging/20090302-Thunderbird-2.0.0.21
find -type d -iregex '.*release.*' | perl -nle '$a = $_; $a =~ s/release/releasetest/; system("diff -r -u $_ ../20090302-Thunderbird-2.0.0.21-test/$a");'

Publish updates to release channel

In case you need to cancel/abort updates, the quickest way to do this is here
While waiting for QA to finish testing, do a backup:

# on aus2-staging verify we have correct tools
$ sudo su - cltbld
$ cd ~/bin
$ cvs update -dP
# backup snippets on release/live channel
$ time ~/bin/backupsnip 20090302-Thunderbird-2.0.0.21
real    43m45.645s
user    0m36.126s
sys     0m47.343s

After QA finished testing the releasetest channel and go from release driver, enable release channel:

# on aus2-staging
# put snippets on release/live channel
$ sudo su - cltbld
$ time ~/bin/pushsnip 20090302-Thunderbird-2.0.0.21
real    1m19.576s
user    0m0.082s
sys     0m3.777s

We disabled updates because of problematic redirects of first run pages, part of moving content to www.mozillamessaging.com, and then enabled them after that was fixed.

Update symlink

On stage.m.o, change symlink to point to new release:

$ cd /pub/mozilla.org/thunderbird/releases
# confirm where latest symlinks point to where before starting
$ ls -la latest*
lrwxrwxrwx  1 cltbld thunderbird 8 Dec 30 11:44 latest -> 2.0.0.19
lrwxrwxrwx  1 cltbld thunderbird 8 Dec 30 11:44 latest-2.0 -> 2.0.0.19
$ rm latest    # (note no trailing slash)
$ ln -s 2.0.0.21 latest
$ rm latest-2.0   # (note no trailing slash)
$ ln -s 2.0.0.21 latest-2.0
# confirm latest symlinks point to newest release
$ ls -la latest*
lrwxrwxrwx  1 cltbld thunderbird 8 Mar 18 13:54 latest -> 2.0.0.21
lrwxrwxrwx  1 cltbld thunderbird 8 Mar 18 13:54 latest-2.0 -> 2.0.0.21

Module	Parent Branch	Created Tag/Branch	Pull date
cvsroot/mozilla	MOZILLA_1_8_BRANCH	GECKO181_20090302_RELBRANCH	2009-03-01 16:14 PST
cvsroot/mozilla	GECKO181_20090302_RELBRANCH	THUNDERBIRD_2_0_0_21_RELEASE, THUNDERBIRD_2_0_0_21_BUILD1	2009-03-02 14:49 PST (but see Tag below)
l10n/l10n	MOZILLA_1_8_BRANCH	GECKO181_20090302_RELBRANCH	2009-03-01 16:14 PST
l10n/l10n	GECKO181_20090302_RELBRANCH	THUNDERBIRD_2_0_0_21_RELEASE, THUNDERBIRD_2_0_0_21_BUILD1	2009-03-02 15:18 PST

Releases/Thunderbird 2.0.0.21/BuildNotes

Contents

Build Engineers

Bonsai queries

Tags

Build data

Notes

Build 1

Tag

Source

Build/Repack

Sign

l10n verification

Generate updates

Publish updates to test channels (betatest & releasetest)

Update verify

Push updates to beta channel

Sign installers

Update bouncer

Push to mirrors

Update rsync module

Final verification

Publish updates to release channel

Update symlink

Navigation menu

Personal tools

Namespaces

Variants

Views

More

Search

Navigation

How to Contribute

MozillaWiki

Around Mozilla

Tools