Releases/Firefox 3.0.12/BuildNotes

From MozillaWiki
Jump to: navigation, search

Please be sure to use

  • UPDATE_PACKAGING_R8 in the bootstrap config, so that we start using SHA512 hashes for update snippets, bug 472790

- nthomas

  • Also please make sure that you use ffxbld for 'push to mirrors' and updating the latest-3.0 symlink , rather than cltbld (this is a change, but a desired one).

- bhearsum

Build Engineers

Tracking release bug

Bonsai queries


Build 1:

Module Branch Tag Pull date
cvsroot/mozilla HEAD GECKO190_20090706_RELBRANCH FIREFOX_3_0_12_BUILD1 FIREFOX_3_0_12_RELEASE


Build 1

Setup before starting:

  • On fx-win32-1.9-slave2:
    • Deleted /e/xr19rel/WINNT_5.2_Depend
    • Deleted /e/fx19l10nrel/WINNT_5.2_Depend
    • Deleted /e/fx19rel/WINNT_5.2_Depend
  • On fx-linux-1.9-slave2:
    • Deleted /builds/verify/firefox-3.0.11
    • Deleted /builds/updates/firefox-3.0.11
    • Deleted /builds/source/firefox-3.0.11
    • Deleted /builds/data/cltbld/firefox-3.0.11
    • Deleted /builds/tinderbox/Xr-Mozilla1.9-Release/Linux_2.6.18-53.1.13.el5_Depend
    • Deleted /builds/tags/FIREFOX_3_0_6_BUILD1
    • Deleted /builds/tags/FIREFOX_3_0_7_BUILD1
    • Deleted /builds/tags/FIREFOX_3_0_7_BUILD2
  • On fx-mac-1.9-slave2:
    • Deleted /builds/verify/firefox-3.0.11
  • Space free on slaves:
    • fx-mac-1.9-slave2: 26G free on /
    • fx-win32-1.9-slave2: 16.6G free on D, 14G free on E
    • fx-linux-1.9-slave2: 22G free on /builds
  • Kicked off automation:
 buildbot sendchange --username=lsblakk --master=localhost:9989 -m"Firefox 3.0.12build1 release" go


  • No problems


  • No problems

Build & Repack

  • No problems


No problems


  • No problems

Generate Updates

  • No problems

Publish Updates to Test Channels (betatest & releasetest)

  • No problems

Update Verify

Mac failed Update Verify because 3.0.11 throttling was still active from the major update.

Linux failed because pushing to hg would have created a new head.

Windows failed with "Step Updates died: shell call returned bad exit code: 1 at Bootstrap/ line 110."

After patching to include ?force=1 to the aus url, all platforms succeeded on a second build, though win32 second build failed with normal binary difference errors.


  • No Problems

Partner Repacks

  • No problems


Had to kick off manually, the dependent scheduler did not fire after Stage step.

Builds were fine, msg sent to release-drivers to announce candidates.

Push updates to beta channel

  # cltbld@aus2-staging.m.o
  # make sure scripts are up to date
  cd bin
  cvs update
  cd /opt/aus2/snippets/staging/
  ~/bin/backupsnip 20090707-Firefox-3.0.12-beta
  ~/bin/pushsnip 20090707-Firefox-3.0.12-beta

Sign Installers

Done manually using these installer-signing-instructions here -- Only up to the STOP HERE

On Stage complete stage-merged:

# on stage
cd /data/cltbld/firefox-3.0.12/
rsync -av batch1/mar/ stage-merged/
rsync -av batch1/stage-signed/ stage-merged/

Create MD5 and SHA1 checksum files

# on stage
cd /data/cltbld/firefox-3.0.12/stage-merged/
~/bin/checksum-files .

Fix permissions & ownership (on the two SUM files, and the detached sigs)

chown -R cltbld:firefox .
chmod 644 *SUMS

Update Bouncer

Done. Note for next release: Do not remove the Check Now bit on the Firefox-3.0.12 Products until well after the change to the rsync module (to prevent the likes of bug 464566)

Push to mirrors

Make sure that you use ffxbld for 'push to mirrors' and updating the latest-3.0 symlink , rather than cltbld (this is a change, but a desired one).

  • push the stage-merged directory to the releases area:
# on stage
rsync -av /data/cltbld/firefox-3.0.12/stage-merged/ /home/ftp/pub/firefox/releases/3.0.12/
  • edit the exclude file (as cltbld) /pub/ to add the new release (3.0.12) and remove the previous release (3.0.11).

Final Verification

hg clone
cd tools/release/updates
cat moz19-firefox-*.cfg | grep -v major | sed 's/betatest/releasetest/' > update.cfg
./ -t update.cfg 2>&1 | tee quickVerify.log
  • Look for any HTTP error codes besides 200 ("OK") and 302 ("Found"):
grep HTTP quickVerify.log | grep -v 200 | grep -v 302

  • Before pushing final updates, verify that "release" and "releasetest" channel match:
# on aus2-staging
$ cd /opt/aus2/snippets/staging/20090707-Firefox-3.0.12
$ find -type d -iregex '.*release.*' | perl -nle '$a = $_; $a =~ s/release/releasetest/; system("diff -r -u $_ ../20090707-Firefox-3.0.12-test/$a");'

3.0.12 -> 3.5.1 Major Update Refresh

Snippet Generation
# ssh
mkdir -p /builds/3.0.12-3.5.1-major-update/snippets
cd /builds/3.0.12-3.5.1-major-update/snippets

cvs co -d patcher-configs mozilla/tools/patcher-configs
cvs co -r UPDATE_PACKAGING_R9 -d patcher mozilla/tools/patcher
cd patcher
cvs co -r UPDATE_PACKAGING_R9 -d MozBuild mozilla/tools/release/MozBuild

# build tools
./ --build-tools --app=firefox --tools-rev=UPDATE_PACKAGING_R9 --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_build-tools.log

# download mars
./ --download --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_download.log

# FIXME - patcher needs to see that the MARs that it thinks are partials
#             are there or else it will not attempt to generate patchinfo
cd temp/firefox
ln -s 3.5.1 3.0.12-3.5.1
cd ../..

# Create partial patches and snippets
./ --create-patches --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox-create-patches.log
Quick verify

Check that releasetest = beta = release.

cd temp/firefox/3.0.12-3.5.1
# releasetest == beta
find aus2.test -type d -iregex '.*/releasetest$' | perl -nle '$a = $_; $a =~ s/releasetest/beta/; $a =~ s/aus2\.test/aus2/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/releasetest/; $a =~ s/aus2/aus2.test/; system("diff -r -u $_ $a");'
# beta == release
find aus2 -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta/release/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/beta/; system("diff -r -u $_ $a");'
Push to AUS
cd /builds/3.0.12-3.5.1-major-update/snippets/patcher/temp/firefox/3.0.12-3.5.1
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.test/
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2/
Enable test snippets
# cltbld@aus2-staging
~/bin/backupsnip 20090625-Firefox-3.0.12-3.5.1-test
~/bin/pushsnip   20090625-Firefox-3.0.12-3.5.1-test

Update verify

Ran update verify on fx-{linux,mac,win32}-1.9-slave2 as follows:

mkdir -p /builds/verify/firefox-3012-351-major/
cd /builds/verify/firefox-3012-351-major/
hg clone

# get patch to trawl "Only in" directories
cd tools/release/common
# on mac & linux
curl -sL | patch -p3
# on win32
wget --no-check-certificate -O patch
patch -p3 < patch

# everyone ...
cd ../updates
platform=linux   # or mac or win32
./ -c moz19-firefox-$platform-major.cfg 2>&1 | tee $platform.log

results everything the same as 3.0.11 -> 3.5.0 MU. aka, everything is OK

Publish Updates to Release Channel

updated AUS' config-dist.php and pushed because of the new MU at this time.

# cltbld@aus2-staging
cd /opt/aus2/snippets/staging
~/bin/backupsnip 20090707-Firefox-3.0.12
~/bin/pushsnip 20090707-Firefox-3.0.12
~/bin/pushsnip 20090721-Firefox-3.0.12-3.5.1-MU


# ffxbld@stage
cd /home/ftp/pub/firefox/releases
rm latest-3.0 && ln -s 3.0.12 latest-3.0

Release matching XULRunner build

XULRunner builds are produced by the automation, this is to groom them and push to the mirrors.

# cltbld@stage

mkdir -p ~/xulrunner-${VERSION}/batch1/stage-unsigned/{runtimes,source,sdk}
cd ~/xulrunner-${VERSION}/batch1/stage-unsigned
rsync -av --exclude=*sdk* --exclude=*info* \
  /home/ftp/pub/xulrunner/nightly/${VERSION}-candidates/build${BUILD}/ runtimes/
rsync -av --include=*sdk* --exclude=* \
  /home/ftp/pub/xulrunner/nightly/${VERSION}-candidates/build${BUILD}/ sdk/
cp /home/ftp/pub/firefox/releases/${FX_VERSION}/source/firefox-${FX_VERSION}-source.tar.bz2 \
cd ~/xulrunner-${VERSION}/batch1
rsync -av stage-unsigned/ stage-signed/

Then create detached signatures per usual process (with PRODUCT=xulrunner, VERSION=, don't need BUILD or TAG), skip the win32 signing section, do pgp, verify and upload. Back on stage:

cd ~/xulrunner-${VERSION}/batch1/stage-signed
cp /home/ftp/pub/firefox/releases/${FX_VERSION}/KEY .
~/bin/checksum-files .
chown -R cltbld:xulrunner .
find . -type f -exec chmod -v 644 {} \;
find . -type d -exec chmod -v 755 {} \;
cd ~/xulrunner-${VERSION}/batch1
rsync -nav stage-signed/* /home/ftp/pub/xulrunner/releases/${VERSION}/

If all is fine then

rsync -av stage-signed/* /home/ftp/pub/xulrunner/releases/${VERSION}/

(Mirror updating can take a bit, 10-15 mins this time)

Ping mfinkle or Mossop to update the links in these two wiki pages for the version change