Releases/Firefox 3.6.4/BuildNotes

From MozillaWiki
Jump to: navigation, search

Build Engineers

bhearsum - bug 558769

Signed-off Revision(s)

Build1: 0121ed69d08e
Build2: 1cc6c440d034
Build3: 379c0002adfa
Build4: 8fe06049502c
Build5: 0383745fc4de
Build6: 41807e6bcb14
Build7: 37f5a4a97d26


On releases/mozilla-1.9.2:

Build # Tag Changeset
1 GECKO1924_20100413_RELBRANCH 0121ed69d08e
FIREFOX_3_6_4_BUILD1 7ef41676c6e2
2 FIREFOX_3_6_4_BUILD2 1cc6c440d034
3 FIREFOX_3_6_4_BUILD3 379c0002adfa
4 FIREFOX_3_6_4_BUILD4 8fe06049502c
5 FIREFOX_3_6_4_BUILD5 0383745fc4de
6 FIREFOX_3_6_4_BUILD6 41807e6bcb14
7 FIREFOX_3_6_4_BUILD7 FIREFOX_3_6_4_RELEASE 37f5a4a97d26

Build data

Build # Type Build ID Build machine
1 Linux 20100413154310 moz2-linux-slave24
Mac 20100413152922 moz2-darwin9-slave12
Windows 20100413172113 win32-slave09
2 Linux 20100502202326 moz2-linux-slave08
Mac 20100502202401 moz2-darwin9-slave13
Windows 20100502221517 win32-slave05
3 Linux 20100503113541 mv-moz2-linux-ix-slave07
Mac 20100503113315 bm-xserve12
Windows 20100503122926 mw32-ix-slave02
4 Linux 20100513140540 mv-moz2-linux-ix-slave04
Mac 20100513134853 bm-xserve18
Windows 20100513144105 mw32-ix-slave02
5 Linux 20100523181754 mv-moz2-linux-ix-slave04
Mac 20100523180910 bm-xserve09
Windows 20100523185824 mw32-ix-slave02
6 Linux 20100527085242 mv-moz2-linux-ix-slave04
Mac 20100527084110 bm-xserve18
Windows 20100527093236 mw32-ix-slave02
7 Linux 20100611135942 mv-moz2-linux-ix-slave15
Mac 20100611134546 bm-xserve15
Windows 20100611143157 mw32-ix-slave22


Older Builds

Notes for older builds can be found on this page

Build 7

  • Marked pm02, any release, all builders for clobber with clobberer
  • Landed release config update (including updated l10n-changesets)
  • Tagged buildbot-configs, buildbotcustom, and tools with FIREFOX_3_6_4_BUILD7 and moved FIREFOX_3_6_4_RELEASE tag (to tip of default)
  • Reconfig pm02
  • started the automation with
buildbot sendchange --username=bhearsum --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6.4 build7" Go


No problems


No problems with Firefox or XULRunner source.


  • linux, mac: no problems
  • win32: no problems

Unit Tests

  • Linux: PASS
  • Mac: PASS
  • Windows: PASS


Signing Instructions
Forgot to delete en-US.xpi before signing, fixed with:

cd ~/signing-work/firefox-3.6.4/signed-build7
rm en-US.xpi *SUMS*
cd ..
make checksum-files create-sigs verify-sigs fake-upload PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1

Partner Builds

Manually created by Kev, signed by us on cm-keymaster01 using a variation of these signing instructions.

l10n verification



3.6* -> 3.6.4
  • Failed out the first time when downloading a build. Probably a caching problem.
  • Re-ran on an MPT VM.
    • Failed out when uploading snippets because the slave didn't have the full set of keys
    • Fixed the ssh keys
    • Pushed through manually with:
# cltbld@moz2-linux-slave43
rsync -av -e "ssh -oIdentityFile=~/.ssh/cltbld_dsa" aus2/
rsync -av -e "ssh -oIdentityFile=~/.ssh/cltbld_dsa" aus2.beta/
rsync -av -e "ssh -oIdentityFile=~/.ssh/cltbld_dsa" aus2.test/

And then on aus2-staging as cltbld:

~/bin/backupsnip 20100614-Firefox-3.6.4-test
~/bin/pushsnip 20100614-Firefox-3.6.4-test

Update Verify

Manually triggered the builders.

  • Linux: PASS
  • Mac: PASS
  • Win32: PASS
3.6.4buildN -> 3.6.4build7 beta and release channels

Same as build6, with updated buildids.

As cltbld@aus2-staging

cd /opt/aus2/snippets/staging
for i in beta test; do
  if  $i == test ; then
  export $channel
  mkdir -p 20100614-Firefox-3.6.4-$i-buildN-to-build7/Firefox
  # build6 --> build7
  rsync -a --exclude 'partial.txt' 20100614-Firefox-3.6.4/Firefox/3.6.3/ 20100614-Firefox-3.6.4-$i-buildN-to-build7/Firefox/3.6.4/
  mv 20100614-Firefox-3.6.4-$i-buildN-to-build7/Firefox/3.6.4/Linux_x86-gcc3/{20100401074458,20100527085242}
  mv 20100614-Firefox-3.6.4-$i-buildN-to-build7/Firefox/3.6.4/Darwin_Universal-gcc3/{20100401064631,20100527084110}
  mv 20100614-Firefox-3.6.4-$i-buildN-to-build7/Firefox/3.6.4/WINNT_x86-msvc/{20100401080539,20100527093236}
  # Change the bouncer links
  find 20100614-Firefox-3.6.4-$i-buildN-to-build7/ -type f -exec sed -i -e 's/3.6.4-complete/3.6.4build7-complete/' {} \;
  # Create "partials"
  for j in `find 20100614-Firefox-3.6.4-$i-buildN-to-build7/ -type f`; do dir=`dirname $j` && cp $j $dir/partial.txt && sed -i -e 's/type=complete/type=partial/' $dir/partial.txt; done
  # Change the update channel
  for j in `find 20100614-Firefox-3.6.4-$i-buildN-to-build7/ -mindepth 6 -maxdepth 6 -type d`; do newdir=`echo $j | sed -e "s/release$/$channel/"` && mv $j $newdir; done

  # build5 --> build7
  rsync -a 20100614-Firefox-3.6.4-$i-buildN-to-build7/Firefox/3.6.4/Linux_x86-gcc3/{20100527085242,20100523181754}/
  rsync -a 20100614-Firefox-3.6.4-$i-buildN-to-build7/Firefox/3.6.4/Darwin_Universal-gcc3/{20100527084110,20100523180910}/
  rsync -a 20100614-Firefox-3.6.4-$i-buildN-to-build7/Firefox/3.6.4/WINNT_x86-msvc/{20100527093236,20100523185824}/
  # build4 --> build7
  rsync -a 20100614-Firefox-3.6.4-$i-buildN-to-build7/Firefox/3.6.4/Linux_x86-gcc3/{20100527085242,20100513140540}/
  rsync -a 20100614-Firefox-3.6.4-$i-buildN-to-build7/Firefox/3.6.4/Darwin_Universal-gcc3/{20100527084110,20100513134853}/
  rsync -a 20100614-Firefox-3.6.4-$i-buildN-to-build7/Firefox/3.6.4/WINNT_x86-msvc/{20100527093236,20100513144105}/
  # build3 --> build7
  rsync -a 20100614-Firefox-3.6.4-$i-buildN-to-build7/Firefox/3.6.4/Linux_x86-gcc3/{20100527085242,20100503113541}/
  rsync -a 20100614-Firefox-3.6.4-$i-buildN-to-build7/Firefox/3.6.4/Darwin_Universal-gcc3/{20100527084110,20100503113315}/
  rsync -a 20100614-Firefox-3.6.4-$i-buildN-to-build7/Firefox/3.6.4/WINNT_x86-msvc/{20100527093236,20100503122926}/
  # build2 --> build7
  rsync -a 20100614-Firefox-3.6.4-$i-buildN-to-build7/Firefox/3.6.4/Linux_x86-gcc3/{20100527085242,20100502202326}/
  rsync -a 20100614-Firefox-3.6.4-$i-buildN-to-build7/Firefox/3.6.4/Darwin_Universal-gcc3/{20100527084110,20100502202401}/
  rsync -a 20100614-Firefox-3.6.4-$i-buildN-to-build7/Firefox/3.6.4/WINNT_x86-msvc/{20100527093236,20100502221517}/
  # build1 --> build7
  rsync -a 20100614-Firefox-3.6.4-$i-buildN-to-build7/Firefox/3.6.4/Linux_x86-gcc3/{20100527085242,20100413154310}/
  rsync -a 20100614-Firefox-3.6.4-$i-buildN-to-build7/Firefox/3.6.4/Darwin_Universal-gcc3/{20100527084110,20100413152922}/
  rsync -a 20100614-Firefox-3.6.4-$i-buildN-to-build7/Firefox/3.6.4/WINNT_x86-msvc/{20100527093236,20100413172113}/

# release snippets, copied from beta
rsync -a 20100614-Firefox-3.6.4{-beta,}-buildN-to-build7/
find 20100614-Firefox-3.6.4-buildN-to-build7/ -mindepth 5 -maxdepth 5 -type d -exec mv {}/beta {}/release \;
# Push test snippets
~/bin/backupsnip 20100614-Firefox-3.6.4-test-buildN-to-build7
~/bin/pushsnip   20100614-Firefox-3.6.4-test-buildN-to-build7
Update Verify

Manual checks.

3.0.19 -> 3.6.4 Major Update

This is to be unprompted straight away, tracked in bug 560551. Version bumps for patcher config and update verify config.

Snippet Generation

on fx-linux-1.9-slave2:

mkdir -p /builds/3.0.19-3.6.4-major-update/snippets
cd /builds/3.0.19-3.6.4-major-update/snippets

cvs co -d patcher-configs mozilla/tools/patcher-configs
cvs co -r UPDATE_PACKAGING_R9 -d patcher mozilla/tools/patcher
cd patcher
cvs co -r UPDATE_PACKAGING_R9 -d MozBuild mozilla/tools/release/MozBuild

# build tools
./ --build-tools --app=firefox --tools-rev=UPDATE_PACKAGING_R9 --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_build-tools.log

# download mars
./ --download --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_download.log

# FIXME - patcher needs to see that the MARs that it thinks are partials
#             are there or else it will not attempt to generate patchinfo
cd temp/firefox
ln -s 3.6.4 3.0.19-3.6.4
cd ../..

# Create partial patches and snippets
./ --create-patches --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox-create-patches.log
Quick Verify

Check that releasetest = beta = release

cd temp/firefox/3.0.19-3.6.4

# releasetest == beta
find aus2.test -type d -iregex '.*/releasetest$' | perl -nle '$a = $_; $a =~ s/releasetest/beta/; $a =~ s/aus2\.test/aus2.beta/; system("diff -r -u $_ $a");'
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/releasetest/; $a =~ s/aus2\.beta/aus2.test/; system("diff -r -u $_ $a");'

# beta == release
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/aus2\.beta/aus2/; $a =~ s/beta/release/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/beta/; $a =~ s/aus2/aus2.beta/; system("diff -r -u $_ $a");'
Push to AUS
cd /builds/3.0.19-3.6.4-major-update/snippets/patcher/temp/firefox/3.0.19-3.6.4
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.test/
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.beta/
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2/
Enable test snippets
~/bin/backupsnip 20100622-Firefox-3.0.19-3.6.4-MU-test
~/bin/pushsnip   20100622-Firefox-3.0.19-3.6.4-MU-test
Update Verify

Ran update verify on fx-{linux,mac,win32}-1.9-slave2 as follows:

 mkdir -p /builds/verify/firefox-3019-364-major/
 cd /builds/verify/firefox-3019-364-major/
 hg clone
 cd tools/release/updates
 platform=linux   # or mac or win32
 ./ -c moz19-firefox-$platform-major.cfg 2>&1 | tee $platform.log

Checked the logs carefully and found the same differences documented for 3.0.19 -> 3.6's major update.

3.6.4build7 release -> beta


Because the MARs are version-independent we can just re-use the ones from build6:

# ffxbld@stage
rsync -av /home/ftp/pub/firefox/nightly/3.6.4-candidates/build6/update/beta/ /home/ftp/pub/firefox/nightly/3.6.4-candidates/build7/update/beta/

Done as cltbld on aus2-staging:

cd ~
chmod +x *.sh
cd /opt/aus2/snippets/staging

Test snippets were pushed right after generation:

~/bin/backupsnip 20100614-Firefox-3.6.4-release-to-beta-test
~/bin/pushsnip 20100614-Firefox-3.6.4-release-to-beta-test
Update Verify

Done on fx-linux-1.9-slave2, bm-xserve05, and win32-slave18.

hg clone
cd tools/release/updates
bash -c verify.cfg | tee verify.log
  • Linux
    • PASS
  • Mac
    • PASS
  • Win32
    • PASS

Push to mirrors (candidates directory)

Because this isn't a final release we don't want to push the files anywhere, so instead, we do some editing of the exclude file. Patched /pub/ as follows (as cltbld @ stage):

--- /pub/	2010-05-19 04:27:54.013758000 -0700
+++ rsyncd-mozilla-releases.exclude	2010-05-28 06:26:03.000000000 -0700
@@ -294,6 +294,10 @@
 + firefox
 + firefox/releases
 + firefox/releases/**
-+ firefox/nightly/3.6.4-candidates/build6
-+ firefox/nightly/3.6.4-candidates/build6/**
++ firefox/nightly/3.6.4-candidates/build7
++ firefox/nightly/3.6.4-candidates/build7/**
 + mobile
 + mobile/releases
 + mobile/releases/**

Add 3.6.4build7 Bouncer Entries

Added manually through

Push 3.6.4buildN -> 3.6.4build7 updates

~/bin/backupsnip 20100614-Firefox-3.6.4-buildN-to-build7

~/bin/pushsnip 20100614-Firefox-3.6.4-buildN-to-build7
~/bin/pushsnip 20100614-Firefox-3.6.4-beta-buildN-to-build7

Push 3.6* -> 3.6.4build7 beta channel updates

~/bin/backupsnip 20100614-Firefox-3.6.4-beta

~/bin/pushsnip 20100614-Firefox-3.6.4-beta

Prepare EU Ballot builds

This is a win32 partner build for the EU browser ballot.

After the builds are created (this time by Kev), and passed QA, they were signed and moved into the candidates dir

# ffxbld@stage
# get copy of signed tarball to /tmp/
mkdir /tmp/ffxbld
cd /tmp/ffxbld
tar xfv ../partners-364-signed.tar
cd ffxbld
mv win32 win32-EUballot
mv -v win32-EUballot /pub/

The builds will be pushed when the other ones do, in 'push to mirrors'.

Add Bouncer Entries

Add entries for:

  • Firefox-3.6.4
  • Firefox-3.6.4-Complete
  • Firefox-3.6.4-Partial-3.6.3
  • Firefox-3.6.4-EUballot (points to /firefox/releases/3.6.4/win32-EUballot/:lang/Firefox Setup 3.6.4.exe)

Virus Scan

To be done the morning of the release.

# stage
$ cd /pub/
$ clamdscan -m
----------- SCAN SUMMARY -----------
Infected files: 0
Time: 3135.970 sec (52 m 15 s)

Final checks before push

# ffxbld@stage
cd /pub/
find . ! -user ffxbld 
find . ! -group firefox
find . -type f ! -perm 644 
find . -mindepth 1 -type d ! -perm 755 ! -name 'contrib*'
find . -maxdepth 1 -type d ! -perm 2775  -name 'contrib*'

It's OK for unsigned partner builds to have 644 permissions. It's also OK if contrib builds show up in the first check.

Push to mirrors

# ffxbld@stage
rsync -av --exclude=beta --exclude=*.log --exclude=*.txt --exclude=*unsigned*   --exclude=* --exclude=*.tests.tar.bz2*   --exclude=*partner-repacks*   /pub/   /pub/

Update cltbld@stage:/pub/ to include 3.6.4 instead of 3.6.3.

Final Verification

Run the 'final_verification' builder on pm02.

Several 404s from, passed when checked manually. was being slow, returning 406's or 504's. I disabled it.

WTF don't we have a script to spit out the list of errors to check?

Push updates to release channel

# cltbld @ aus2-staging
cd /opt/aus2/snippets/staging
~/bin/backupsnip 20100614-Firefox-3.6.4
~/bin/pushsnip 20100614-Firefox-3.6.4


# ffxbld@stage
cd /pub/
rm latest-3.6
ln -s 3.6.4 latest-3.6

EU Ballot

Verified the Firefox-3.6.4-EUBallot bouncer entry was setup and working. Filed bug 573904 to update the redirect on download.m.o, and bug 573905 to update

NB: We can do this with one IT bug going forward.

Push major update snippets

# cltbld@aus2-staging
~/bin/backupsnip 20100622-Firefox-3.0.19-3.6.4-MU
~/bin/pushsnip   20100622-Firefox-3.0.19-3.6.4-MU

Disable throttling using a patch like this. Ask nthomas if in any doubt, there's no huge hurry to push it out.