From MozillaWiki
Jump to: navigation, search

« Back to Thunderbird 3 Alpha 1 Release Page

Build Engineers

rick, nthomas

Bug bug 411171 is no longer to be considered a blocker for the 3.0a1 release. Will try to build and upload the symbols manually.

Bonsai queries

Last checkins for alpha 1:

  • cvsroot -
  • l10n -


Tracking bug for Thunderbird 3.0a1 - bug 431788


Note: Thunderbird alpha tags to be reviewed. rick 2008-05-01

Module Tag Pull date
cvsroot/mozilla GECKO19_20080506_RELBRANCH HEAD @ 2008-05-01 03:00 PDT
THUNDERBIRD_3_0a1_BUILD1 GECKO19_20080506_RELBRANCH @ 2008-05-06 15:25 PDT
THUNDERBIRD_3_0a1_RELEASE GECKO19_20080506_RELBRANCH @ 2008-05-06 15:25 PDT
l10n/l10n not tagged, no locales for this release

Build data

Type Build ID SHA1 Push date Build machine
[Windows installer] en-US: tbnewref-win32-tbox
[Mac compressed] en-US: bm-xserve07
[Linux compressed] en-US: tb-linux-tbox

Build 1


Setup before starting:

  • create release branch of and mozconfig (for b2: compare nightly and release mozconfig/ files, merge any desired changes)
  • On bm-xserve07, tb-linux-tbox, tbnewref-win32-tbox, set up the Bootstrap code
cd /builds/   # /e/builds/ on windows
cvs -d co -d release -r RELEASE_AUTOMATION_M9 mozilla/tools/release
cd release/configs
cvs up -A     # in case we need to make changes later
cd ..
ln -s configs/tb-moz19-bootstrap.cfg bootstrap.cfg
mkdir -p old-logs/logs.tb30a1/build1
ln -s old-logs/logs.tb30a1/build1 logs

make test

Some machines don't have the perl module Config::General so comment out

use Config::General;

at the top of Bootstrap/Step/ This needed to be commented out on tb-linux-tbox which will be used for the Tag and Source steps.

  • On bm-xserve07, tb-linux-tbox, tbnewref-win32-tbox, stop running tinderbox and set up tinderbox directories (when ready to actually start the builds)
cd /builds/tinderbox/mozilla/tools/tinderbox
# on linux and mac (skip to windows otherwise)
cd /builds/tinderbox
mkdir Tb-Mozilla1.9-Release
cd Tb-Mozilla1.9-Release
# on windows
cd /e/
mkdir tb19rel
cd tb19rel
<copy in files from tinderbox dir>
# all platforms
rm tinderbox
cvs -d co -r release -d tinderbox-configs mozilla/tools/tinderbox-configs/thunderbird/linux # or macosx or win32
ln -s tinderbox-configs/mozconfig
ln -s tinderbox-configs/
  • on tb-linux-tbox (once off setup for Stage)
# as root
mkdir -p /data/cltbld
chown cltbld:cltbld /data/cltbld
ln -s /builds/release/bin /data/cltbld


On tb-linux-tbox in /builds/release:

./release -o Tag 2>&1 | tee logs/release-Tag-RC1.log


Died right off the bat. This was because the /builds/logs directory did not exist. This directory was created with 775 and owned by cltbld:cltbld.

The /builds/tags/THUNDERBIRD_3_0a1_BUILD1/ directory, which was created by the first failed run, was also removed. The Tag command was then re-run.

Failed a second time

The tagging itself went without incident. When it came time to bump the version.txt, the |cvs ci| failed with

The tbirdbld account is not permitted to check-in to this CVS repository.
If you think it should be allowed to do so, please contact
the system administrators at
cvs commit: Pre-commit check failed

Manual rescue is in progress by nthomas.

  • fixed - where LDAPCSDK_CO_TAG had been replaced incorrectly (recent regression from bug 426083)
  • set config/milestone.txt back to 1.9pre
  • checked into relbranch, logging to /builds/release/logs/release-Tag-RC1-manual.log
  • in /builds/tags/THUNDERBIRD_3_0a1_BUILD1/cvsroot/mozilla/
echo "Tagging THUNDERBIRD_3_0a1_BUILD1" >> /builds/release/logs/release-Tag-RC1-manual.log
cvs -d tag THUNDERBIRD_3_0a1_BUILD1 | tee -a /builds/release/logs/release-Tag-RC1-manual.log
echo >> /builds/release/logs/release-Tag-RC1-manual.log
echo "Tagging THUNDERBIRD_3_0a1_RELEASE" >> /builds/release/logs/release-Tag-RC1-manual.log
cvs -d tag THUNDERBIRD_3_0a1_RELEASE | tee -a /builds/release/logs/release-Tag-RC1-manual.log
  • skipped tagging l10n as not building any locales (need to add a shipped-locales on trunk though)
  • put back symlinks in ~/.ssh to tbirdbld


On tb-linux-tbox in /builds/release:

./release -o Source 2>&1 | tee logs/release-Source-RC1.log


1. Stop tinderbox

# on linux & mac
cd /builds/tinderbox
./tinderbox multi stop  # ps -ef | auxww to see check for tinderbox stopped and kill otherwise
# on windows
# ^C in the msys window where multi-tinderbox is running

2. Update tinderbox code to the latest release automation milestone

# on linux and mac
cd /builds/tinderbox/mozilla/tools/tinderbox
# on win32
cd /e/builds/tinderbox/mozilla/tools/tinderbox
# all

3. Symlink the ssh keys

cd ~/.ssh
rm id_dsa; ln -s cltbld_dsa id_dsa
rm; ln -s

4. Create the tinderbox build directories

# on linux and mac (skip to windows otherwise)
cd /builds/tinderbox
mkdir Tb-Mozilla1.9-Release
cd Tb-Mozilla1.9-Release
rm tinderbox
# on windows
cd /e/
mkdir tb19rel
cd tb19rel
cp /e/builds/tinderbox/mozilla/tools/tinderbox/{,,,,,,,} .
# all platforms
cvs -d co -r release -d tinderbox-configs mozilla/tools/tinderbox-configs/thunderbird/linux # or macosx or win32
ln -s tinderbox-configs/mozconfig
ln -s tinderbox-configs/

5. Checkout the release code and set up log dir, log and bootstrap links

cd /builds
cvs -d co -d release -r RELEASE_AUTOMATION_M9 mozilla/tools/release
cd release/configs
cvs up -A
cd ..
rm bootstrap.cfg logs
ln -s configs/tb-moz19-bootstrap.cfg bootstrap.cfg
mkdir -p old-logs/logs.tb30a1/rc1
ln -s old-logs/logs.tb30a1/rc1 logs
mkdir -p /builds/logs

make test

Then start the build, by changing directory to /builds/release (prefix with /e/ on Windows) and

./release -o TinderConfig 2>&1 | tee logs/release-TinderConfig.log
./release -o Build 2>&1 | tee logs/release-Build.log

After the build is complete, switch the keys back to tbirdbld, cvs up -A in /builds/tinderbox/mozilla/tools/tinderbox and restart tinderbox

TinderConfig Step Failures

All platforms had the error (or similar to):

Step TinderConfig died: Bootstrap::Config::Bump - Could not open /builds/config/thunderbird-3.0a1-build1/l10n_release/ for reading: No such file  or directory at Bootstrap/ line 334

since we are only building en-US for the alpha build. As nthomas notes,

14:53:35 <rick> and it's done.  again with same l10n_release/ issue
14:53:47 <nthomas> lets just make a note of that
14:54:12 <nthomas> We need some sort of createLocales flag in the bootstrap config  
                   to disable all this stuff

All generated by the TinderConfig step look good. Proceeding with the Build step itself.

Build Step

All Build steps are away and running. MozillaRelease

MacOSX: 2008-05-07 14:45 PDT
Linux: 2008-05-07 15:00 PDT
Windows: 2008-05-07 15:25 PDT

The builds succeeded on all platforms. However, since the Source step had been run with the tbirdbld sshUser, the directories on stage-old.m.o were created with the wrong permissions. This Build step ran with the cltbld sshUser and thus didn't have access to the folders on stage-old.m.o.

The ownership was fixed on stage-old.m.o and the Build push step was re-run on the linux and mac boxes. The windows box had no problem as the ownership issues were fixed before the windows build had gotten to the point of pushing the build out to stage-old.m.o.

This was done by running

./release -o Build -p 2>&1 | tee logs/release-Build-push.log

buildsymbols failure on mac and how to do them manually

The buildsymbols problem, bug 411171 did occur during the build. After the Build push above was done, vnc'd into the mac and ran:

make -C /builds/tinderbox/Tb-Mozilla1.9-Release/Darwin_8.8.4_Depend/mozilla/../build/universal/ppc buildsymbols

and that ran fine. Then ran,

make -C /builds/tinderbox/Tb-Mozilla1.9-Release/Darwin_8.8.4_Depend/mozilla/../build/universal/ppc  uploadsymbols

and that failed since a number of environment variables were not defined. (They are defined in the which is used by the script to run these two make commands). After poking about a bit, it looks like all that should need to be done to upload the symbols is:

SYMBOL_SERVER_PATH='/mnt/netapp/breakpad/symbols_tbrd' \
SYMBOL_SERVER_SSH_KEY='/Users/cltbld/.ssh/tbirdbld_dsa' \ 
make -C /builds/tinderbox/Tb-Mozilla1.9-Release/Darwin_8.8.4_Depend/build/universal/ppc uploadsymbols

I will verify this with nthomas before running it. My biggest concern at this point is that the uploadsymbols did run previously and I'm not sure whether something needs to be cleaned up on stage-old.m.o first before running this again.

Symbols have been uploaded. 2008-05-08 8:25 AM PDT

Bump version numbers

Bump the version number to 3.0a2pre in



Did this manually as we don't have a shipped-locales file for Thunderbird yet.

# on stage as cltbld
mkdir /data/cltbld/thunderbird-3.0a1
cd /data/cltbld/thunderbird-3.0a1

# borrow from a previous release
rsync -av /data/cltbld/thunderbird- stage-merged/

# get all the files and remove strays
mkdir -p batch1/stage-unsigned
cd batch1/stage-unsigned
rsync -av /pub/ .
rm -rf unsigned/ *.log *.txt

# Rename the installers/archives
mkdir -p source {mac,linux-i686,win32}/en-US
mv thunderbird-3.0a1-source.tar.bz2 source/
mv thunderbird-3.0a1.en-US.linux-i686.tar.bz2 linux-i686/en-US/thunderbird-3.0a1.tar.bz2
mv thunderbird-3.0a1.en-US.mac.dmg mac/en-US/Thunderbird\ 3.0\ Alpha\ 1.dmg
mv thunderbird-3.0a1.en-US.win32.installer.exe win32/en-US/Thunderbird\ Setup\ 3.0\ Alpha\ 1.exe

# rename the update files (not actually used)
cd ..
mkdir -p mar/update/{linux-i686,mac,win32}/en-US
mv stage-unsigned/*.mar mar/
mv mar/thunderbird-3.0a1.en-US.linux-i686.complete.mar mar/update/linux-i686/en-US/thunderbird-3.0a1.complete.mar
mv mar/thunderbird-3.0a1.en-US.mac.complete.mar mar/update/mac/en-US/thunderbird-3.0a1.complete.mar
mv mar/thunderbird-3.0a1.en-US.win32.complete.mar mar/update/win32/en-US/thunderbird-3.0a1.complete.mar

# fix up permissions 
find . -type f -exec chmod -v 644 {} \;
find . -type d -exec chmod -v 755 {} \;
chown -R cltbld:thunderbird *

rsync -av stage-unsigned/ stage-signed/

Sign Installers

  • pull stage:/data/cltbld/thunderbird-3.0a1/batch1/stage-signed/ to signing machine
  • follow installer-signing-instructions here
  • push signed bits back to same location on stage
  • complete stage-merged:
# on stage
cd /data/cltbld/thunderbird-3.0a1/
rsync -av batch1/mar/ stage-merged/
rsync -av batch1/stage-signed/ stage-merged/
  • Create MD5 and SHA1 checksum files
# on stage as cltbld
cd /data/cltbld/thunderbird-3.0a1/stage-merged/
~/bin/checksum-files .
  • Fix permissions & ownership (on the detached sigs and the two SUM files respectively)
chown -R cltbld:thunderbird .
chmod 644 *SUMS


  • get the last formal "go" from Thunderbird QA & release-drivers
  • push the stage-merged directory to the releases area :
# on stage
rsync -av /data/cltbld/thunderbird-3.0a1/stage-merged/ /home/ftp/pub/thunderbird/releases/3.0a1/
  • Update /etc/rsyncd-mozilla-current.exclude on surf and dm-stage02 - not critical that this happens immediately. Skipped for alpha release, we don't need the extra mirror coverage

Update Bouncer

  • Add entries to