Releases/Firefox 3.6/BuildNotes

From MozillaWiki
Jump to: navigation, search

Until we're about to ship this page will serve as a spot to put down a rough checklist of what needs to be done to ship.

Release Engineers

Tracking Bug

bug 540666


Stage and Rename files

Much the same as Firefox 3.5.

# ffxbld@stage
mkdir firefox-3.6
cd firefox-3.6
rsync -av --exclude='*partial.mar' --exclude=source --exclude=*SUMS /pub/ .
find contrib linux-i686 update -type f | perl -nle '$nf = $_; if ( $nf =~ s/rc2// ) { system("mv","-v",$_,$nf)}' 
find mac win32 -type f | perl -nle '$nf = $_; if ( $nf =~ s/\ RC\ 2// ) { system("mv","-v",$_,$nf)};'
wget -O../firefox-3.6-shipped-locales
~cltbld/bin/ -m ~/firefox-3.6-shipped-locales -l bz2
~cltbld/bin/checksum-files .
chown -R ffxbld:firefox linux-i686 mac update win32
chmod 644 *SUMS

Content verification

find /pub/ -type f | perl -nle '$a = substr($_,41); $a =~ s/rc2//; $a =~ s/\ RC\ 2//; system("diff","-q",$_,$a);'


diff: contrib/solaris_tarball/firefox-3.6rc1-3.6.en-US.solaris-10-fcs-sparc.partial.mar: No such file or directory
diff: contrib/solaris_tarball/firefox-3.6rc1-3.6.en-US.solaris-10-fcs-i386.partial.mar: No such file or directory
diff: contrib/solaris_tarball/firefox-3.6rc1-3.6.en-US.opensolaris-i386.partial.mar: No such file or directory
diff: contrib/solaris_tarball/firefox-3.6rc1-3.6.en-US.opensolaris-sparc.partial.mar: No such file or directory
diff: source/firefox-3.6.bundle.asc: No such file or directory
diff: source/firefox-3.6.bundle: No such file or directory
diff: source/firefox-3.6.source.tar.bz2.asc: No such file or directory
diff: source/firefox-3.6.source.tar.bz2: No such file or directory
diff: update/linux-i686/af/firefox-3.6rc1-3.6.partial.mar: No such file or directory
diff: update/linux-i686/ar/firefox-3.6rc1-3.6.partial.mar: No such file or directory
diff: update/win32/zh-CN/firefox-3.6rc1-3.6.partial.mar: No such file or directory
diff: update/win32/zh-TW/firefox-3.6rc1-3.6.partial.mar: No such file or directory
Files /pub/ and MD5SUMS differ
Files /pub/ and SHA1SUMS differ

Everything above is expected: We don't want the partials, we regenerated *SUMS because of filename changes, and we haven't yet regenerated source packages yet.

Ownernship/permission verification

find . ! -user ffxbld
find . ! -group firefox
find . -type f ! -perm 644
find . -mindepth 1 -type d ! -perm 755 ! -name 'contrib*'
find . -maxdepth 1 -type d ! -perm 2775  -name 'contrib*'

Virus check

This will have been done for 3.6rc2 and background scans but double check

clamscan -r --quiet -i .

No problems found.

Add index files

curl -sL > index.html
for dir in `find . -type d -mindepth 1`; do cp -pv index.html $dir/; done

Generate major update

We currently have majortest snippets pointing at 3.6rc2 in bouncer but we should point at the 3.6 files for tracking purposes. Also need to have a releasetest channel for QA to check. Updated config.

Throttle update channels

beta and release users of 3.5.7 should need to manually check for the major update so throttling was enabled in bug 540762.

Snippet generation

On fx-linux-1.9-slave2:

mkdir -p /builds/3.5.7-3.6-major-update
cd /builds/3.5.7-3.6-major-update

cvs co -d patcher-configs mozilla/tools/patcher-configs
cvs co -r UPDATE_PACKAGING_R10 -d patcher mozilla/tools/patcher
cd patcher
cvs co -r UPDATE_PACKAGING_R10 -d MozBuild mozilla/tools/release/MozBuild

# build tools
./ --build-tools --app=firefox --tools-rev=UPDATE_PACKAGING_R10 --config=../patcher-configs/moz191-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_build-tools.log

# fake 3.5.7 mars, which aren't used for anything but take time to d/l
mkdir -p temp/firefox/3.5.7/ftp/
for l in af ar as be bg bn-BD bn-IN ca cs cy da de el en-GB en-US eo es-AR es-CL es-ES es-MX et eu fa fi fr fy-NL ga-IE gl gu-IN he hi-IN hr hu id is it ja ja-JP-mac ka kk kn ko ku lt lv mk ml mn mr nb-NO nl nn-NO oc or pa-IN pl pt-BR pt-PT rm ro ru si sk sl sq sr sv-SE ta ta-LK te th tr uk vi zh-CN zh-TW; do
  touch temp/firefox/3.5.7/ftp/firefox-3.5.7.${l}.{linux-i686,mac,win32}.complete.mar

# download mars
./ --download --app=firefox --config=../patcher-configs/moz191-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_download.log

# FIXME - patcher needs to see that the MARs that it thinks are partials
#             are there or else it will not attempt to generate patchinfo
cd temp/firefox
mkdir 3.5.7-3.6rc2
ln -s ../3.6rc2/ftp 3.5.7-3.6rc2/ftp
cd ../.. 

# Create partial patches and snippets
./ --create-patches --app=firefox --config=../patcher-configs/moz191-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox-create-patches.log

Quick verify

Check that releasetest = beta = release.

cd temp/firefox/3.5.7-3.6rc2
# releasetest == beta
find aus2.test -type d -iregex '.*/releasetest$' | perl -nle '$a = $_; $a =~ s/releasetest/beta/; $a =~ s/aus2\.test/aus2/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/releasetest/; $a =~ s/aus2/aus2.test/; system("diff -r -u $_ $a");'

# beta == release
find aus2 -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta/release/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/beta/; system("diff -r -u $_ $a");'

No output, as expected. Also verified that the only changes from what QA has tested are

+appv=3.6 RC 2

We'll run update verify to confirm that bouncer is doing the right thing.

Push to AUS

rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.test/
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2/

Enable test snippets

# cltbld@aus2-staging
~/bin/backupsnip 20100120-Firefox-3.5.7-3.6final-MU-test
~/bin/pushsnip   20100120-Firefox-3.5.7-3.6final-MU-test
Update verify

Ran update verify on fx-{linux,mac,win32}-1.9-slave2 as follows:

mkdir -p /builds/verify/firefox-357-36final-major/
cd /builds/verify/firefox-357-36final-major/
hg clone

# get patch to trawl "Only in" directories
cd tools/release/common
# on mac & linux
curl -sL | patch -p3
# on win32
wget --no-check-certificate -O patch
patch -p3 < patch

# everyone ...
cd ../updates
## get un-checked in configs
wget --no-check-certificate -O patch
patch -p3 < patch
## end of config get 
platform=linux   # or mac or win32
./ -c moz191-firefox-$platform-major.cfg 2>&1 | tee $platform.log


Locale specific (all platforms)

  • No update found for mn because that's not available for 3.6
  • bug 538570 needs to land in a point release to fix this issue with Italian
Only in source/firefox/searchplugins: demauro.xml
  • Linux: No other differences
  • Mac: Leftover empty directories
Only in source/ Plug-Ins
Contents of source/ dir only in source or target
1756043387    0 drwxr-xr-x    3 cltbld   cltbld        102 Dec 21 17:24 source/
1756043388    0 drwxr-xr-x    4 cltbld   cltbld        136 Jan 21 03:27 source/
1756043389    0 drwxr-xr-x    2 cltbld   cltbld         68 Jan 21 03:27 source/
1756043390    0 drwxr-xr-x    3 cltbld   cltbld        102 Dec 21 17:24 source/
1756043391    0 drwxr-xr-x    3 cltbld   cltbld        102 Jan 21 03:27 source/
1756043392    0 drwxr-xr-x    2 cltbld   cltbld         68 Jan 21 03:27 source/
  • Windows: All the normal suspects, no problems
Only in source/bin: .autoreg
Binary files source/bin/freebl3.chk and target/bin/freebl3.chk differ
Binary files source/bin/nssdbm3.chk and target/bin/nssdbm3.chk differ
Only in source/bin: removed-files
Binary files source/bin/softokn3.chk and target/bin/softokn3.chk differ

Mirror related

  • turned down from 1350 to 300 as I was getting 1KB/s from them. Later down to 100 to try to ensure users don't get poor transfer rates

Bouncer Setup

win32, linux, mac + 4 solaris for installer, and again for complete update. ie no WinCE done.

Tag and Source package regeneration

Used the automation to add the FIREFOX_3_6_RELEASE tag on mozilla-1.9.2 and l10n repositories. Two patches needed

Landed and reconfig master. Backout buildbotcustom patch. Set clobbers on mozilla-1.9.2, tag & source, any master. (bug 500473 is open to support this properly.)

Forced the tag step, which worked for l10n but took the top of the relbranch instead of the FIREFOX_3_6rc2_RELEASE revision. Setting 'build = 2' would have avoided this. Fix:

# cltbld@moz2-linux-slave20
cd /builds/slave/tag/mozilla-1.9.2
hg tag -f -u ffxbld -f -r 448d0d2d310c -m "Moved tag FIREFOX_3_6_RELEASE to changeset 448d0d2d310c (as release automation incorrectly chose the tip of the relbranch). CLOSED TREE" FIREFOX_3_6_RELEASE
hg push -e "ssh -l ffxbld -i ~cltbld/.ssh/ffxbld_dsa" -f ssh://

Then force the source step and then moved firefox/nightly/3.6-candidates/ away

# ffxbld @ stage
rsync -av /pub/ ~/3.6-candidates/ && rm -rf /pub/

cd ~/firefox-3.6
rsync -av ~/3.6-candidates/build1/source/ source/
~cltbld/bin/checksum-files .

NB: This actually includes revision 4289e7a61d2f, which is not part of the release build. We can't assume the tip of the relbranch

Then, pull the source/ dir and SUMS files onto the signing machine and generate a sigs for them:

mkdir -p ~/signing-work/firefox-3.6
cd ~/signing-work/firefox-3.6
cvs co -d signing release/signing/tools
rsync -av -e 'ssh -i /home/cltsign/.ssh/ffxbld_dsa' stage-unsigned/
rsync -av -e 'ssh -i /home/cltsign/.ssh/ffxbld_dsa'*SUMS stage-unsigned/
rsync -av stage-unsigned/ stage-signed/
cd stage-signed/
../signing/sign-files .
cd ..
# should only push 4 x asc here, two for source files, 2 for SUMS
rsync -nav -e 'ssh -i /home/cltsign/.ssh/ffxbld_dsa' stage-signed/
rsync  -av -e 'ssh -i /home/cltsign/.ssh/ffxbld_dsa' stage-signed/

Push to Mirrors

Push the covering index.html to mirrors ahead of the main push:

rsync -av --exclude 'contrib*' --exclude '*.exe' --exclude '*.dmg' \
 --exclude '*.bz2' --exclude '*.asc' --exclude '*.mar' --exclude '*.xpi' \
 --exclude '*.bundle' --exclude '*SUMS' --exclude KEY \
 /home/ffxbld/firefox-3.6/ /pub/

Redid 'Ownernship/permission verification', then pushed the files (complete Jan 22 20:22 PST):

rsync -av /home/ffxbld/firefox-3.6/ /pub/

Update rsync modules

Updated stage:/pub/ as cltbld to include 3.6. Removed both Thunderbird's to keep module small, and they get lots of coverage from mozilla-releases anyway.

Push Major Update

# cltbld@aus2-staging
~/bin/backupsnip 20100120-Firefox-3.5.7-3.6final-MU

After verifying billboards are in place on and go from driver

time ~/bin/pushsnip   20100120-Firefox-3.5.7-3.6final-MU
real    0m23.988s
user    0m0.023s
sys     0m0.948s

Symlink update

# ffxbld@stage
ln -s 3.6 latest-3.6
rm latest; ln -s latest-3.6 latest

Remove index.html files

Once we are fully announced we can remove the index.html files:

# ffxbld@stage
cd /pub/
find . -name index.html | less
find . -name index.html | xargs rm -v


Triggered the builds by going to the buildbot waterfall for mozilla-1.9.2 and forcing builds for linux (not x64), mac and windows xulrunner. Set the field "Revision to build" (not properties) to 'FIREFOX_3_6_RELEASE'. Email the builds to release-drivers.

Feb. 22nd, 2010: Community member filed bug since we were missing it.

# as xrbld@stage on a production slave
cd /pub/
mkdir -p 1.9.2-candidates/build1
mv 2010-02-22-07-mozilla-1.9.2/* 1.9.2-candidates/build1 
rm -rf 2010-02-22-07-mozilla-1.9.2

After receiving the OK from mfinkle of dtownsend sign the builds.

# cltbld@stage

mkdir -p ~/xulrunner-${VERSION}/batch1/stage-unsigned/{runtimes,source,sdk}
cd ~/xulrunner-${VERSION}/batch1/stage-unsigned
rsync -av --exclude=*sdk* --exclude=*info* \
  /home/ftp/pub/xulrunner/nightly/${VERSION}-candidates/build${BUILD}/ runtimes/
rsync -av --include=*sdk* --exclude=* \
  /home/ftp/pub/xulrunner/nightly/${VERSION}-candidates/build${BUILD}/ sdk/
cp -pv /pub/${FX_VERSION}/source/firefox-${FX_VERSION}.source.tar.bz2 \
cd ~/xulrunner-${VERSION}/batch1
rsync -av stage-unsigned/ stage-signed/

Then create detached signatures per usual process (with PRODUCT=xulrunner, VERSION=1.9.2, don't need BUILD, TAG, or KEYDIR), skip the win32 signing section, do pgp, verify and upload. Back on stage:

cd ~/xulrunner-${VERSION}/batch1/stage-signed
cp /pub/${FX_VERSION}/KEY .
~/bin/checksum-files .
chown -R cltbld:xulrunner .
find . -type f -exec chmod -v 644 {} \;
find . -type d -exec chmod -v 755 {} \;
cd ~/xulrunner-${VERSION}/batch1
rsync -nav stage-signed/* /home/ftp/pub/xulrunner/releases/${VERSION}/

If all is fine then

rsync -av stage-signed/* /home/ftp/pub/xulrunner/releases/${VERSION}/

(Mirror updating can take a bit, 10-15 mins this time) NOTE: Don't look for uptake on bouncer since we don't have it. If you see your builds on ftp (xulrunner/releases) it will show up on the mirrors eventually.

Update the links in these two wiki pages for the version change. NOTE: make sure to update all the links on the page!!!