From MozillaWiki
Jump to: navigation, search

Build Engineers

bhearsum got us started, then nthomas
Version/config bump bug

Bonsai queries

Build1: cvsroot


Build 1:

Module Parent Branch Created Tag/Branch Pull date
cvsroot/mozilla MOZILLA_1_8_BRANCH GECKO181_20081217_RELBRANCH 2008-12-02 20:33 PST
GECKO181_20081217_RELBRANCH FIREFOX_2_0_0_20_RELEASE, FIREFOX_2_0_0_20_BUILD1 2008-12-17 08:18 PST
l10n/l01n MOZILLA_1_8_BRANCH GECKO181_20081217_RELBRANCH 2008-12-02 20:33 PST
GECKO181_20081217_RELBRANCH FIREFOX_2_0_0_20_RELEASE, FIREFOX_2_0_0_20_BUILD1 2008-12-17 08:58 PST

Build data

Type Build machine
Windows production-pacifica-vm
Mac bm-xserve05
Linux production-prometheus-vm


Build 1

Slave cleanup:

  • production-prometheus-vm
    • /builds/verify/firefox-, thunderbird-
    • /builds/updates/firefox-
    • /build/source/firefox-, thunderbird-
    • /data/cltbld/firefox-
    • /builds/tags/FIREFOX_2_0_0_19_BUILD1, THUNDERBIRD_2_0_0_19_BUILD1
  • bm-xserve05
    • /builds/verify/firefox-, /builds/tinderbox/Fx-Mozilla1.8-FS/Depend*
  • production-pacifica-vm
    • /builds/verify/firefox-, /builds/symbols/firefox-
  • production-1.8-master
    • removed /builds/buildbot/Automation/twistd.log.???, twistd.log.??

Bootstrap tag: RELEASE_AUTOMATION_M11 (unchanged from Fx2.0.0.17 & 18 & 19)

Setup before starting:

  • Updated /builds/buildbot/Automation/buildbot-configs on production-1.8-master. (No change to deploy)
    • master.cfg is "Locally Modified" to contain passwords. Identical to repository version otherwise.
    • Turned off the nightly builders on production-1.8-master to make sure they don't interfere with the release. (We need to figure out how to avoid this in the future).
      • commented out "c['schedulers'].append(depend_scheduler)"
    • updated mozilla1.8 tinderbox to say "no nightlies right now"
  • land change to mozilla/tools/release/configs/fx-moz18-bootstrap.cfg, and move RELEASE_AUTOMATION_M11 tag to new revision (cvs tag -F RELEASE_AUTOMATION_M11 fx-moz18-bootstrap.cfg)
  • verify tinderbox checkout on automation tag
    • prometheus-vm, bm-xserve05: cd /builds/tinderbox/mozilla/tools/tinderbox; cvs up -CPd -r RELEASE_AUTOMATION_M11
    • production-pacifica-vm: cd /cygdrive/c/builds/tinderbox/mozilla/tools/tinderbox; cvs up -CPd -r RELEASE_AUTOMATION_M11
  • ensure that machines have enough resources
    • production-prometheus-vm - 30G in /
    • production-1.8-master - 30G in /builds, 19G in /data
    • production-pacifica-vm - ~31G in c:/
    • bm-xserve05 - 20G in /

Issued sendchange:

buildbot sendchange --username=bhearsum --master=localhost:9989 -m"Firefox build 1" release


  • No problems


  • No problems
  • Verified only version bump changes between Fx2.0.0.19 build2 and this build in the source tarballs


Automated, no problems. Checked that symbols were unpacked OK on talkback server.


Automated, no problems



Automated, no problems in log

Generate Updates

Automated, no problems

Publish Updates to Test Channels (betatest & releasetest)

Automated, no problems

Sign Installers

Done manually using these installer-signing-instructions here

  • complete stage-merged:
# on stage
cd /data/cltbld/firefox-
rsync -av batch1/mar/ stage-merged/
rsync -av batch1/stage-signed/ stage-merged/
  • Create MD5 and SHA1 checksum files
# on stage
cd /data/cltbld/firefox-
~/bin/checksum-files .
  • Fix permissions & ownership (on the two SUM files, and the detached sigs)
chown -R cltbld:firefox .
chmod 644 *SUMS

Update Bouncer

  • Manually added entries to
  • Note for next release: Do not remove the Check Now bit on the Firefox- Products until well after the change to the rsync module (to prevent the likes of bug 464566)

Push to mirrors

  • push the stage-merged directory to the releases area:
# on stage
rsync -av /data/cltbld/firefox- /home/ftp/pub/firefox/releases/
  • edit the exclude file /pub/ to add the new release ( and remove the previous release (

Final Verification

  • Verify that releasetest points to valid bouncer links:
# this can be run from anywhere
cvs co mozilla/testing/release
cd mozilla/testing/release/updates
cat moz18-firefox-*.cfg | grep -v major | sed 's/betatest/releasetest/' | grep -v 2.0a | grep -v 2.0b > update.cfg
./ -t update.cfg 2>&1 | tee quickVerify.log
  • Had some issues with download.m.o not responding quickly for a small proportion of requests (bug 470283, IT could not reproduce, marked WFM
  • Look for any HTTP error codes besides 200 ("OK") and 302 ("Found"):
grep HTTP quickVerify.log | grep -v 200 | grep -v 302
  • Before pushing final updates,verify that "release" and "releasetest" channel match:
# on aus2-staging
$ cd 20081217-Firefox-
time find -type d -iregex '.*release.*' | perl -nle '$a = $_; $a =~ s/release/releasetest/; system("diff -r -u $_ ../20081217-Firefox-$a");'

Publish Updates to Release Channel

# on aus2-staging as cltbld
$ time ~/bin/backupsnip 20081217-Firefox-
# took about 33 mins

$ time ~/bin/pushsnip 20081217-Firefox-
real    1m26.106s
user    0m0.132s
sys     0m6.370s


  • On stage.m.o, do the following:
    • Update "latest" symlinks
$ cd /pub/
$ rm latest-2.0 && ln -s latest-2.0

Push updates to beta channel

We wanted to get release out asap, and beta was done the next day.

# on aus2-staging as cltbld
$ time ~/bin/backupsnip 20081217-Firefox-
real    35m6.615s
user    0m26.607s
sys     0m29.729s
$ time ~/bin/pushsnip 20081217-Firefox-
real    1m47.408s
user    0m0.258s
sys     0m8.838s