Releases/Firefox 3.5.8/BuildNotes

From MozillaWiki
Jump to: navigation, search

Build Engineers

coop and armenzg
bug 543761

Signed-off Revision(s)

Build1: d8ca06471009


On releases/mozilla-1.9.1:

Build # Tag Changeset
1 GECKO1918_20100202_RELBRANCH 9165e7415f91
FIREFOX_3_5_8_BUILD1 9165e7415f91

Build data

Build # Type Build ID Build machine
1 Linux 20100202153512 moz2-linux-slave39
1 Mac 20100202152834 bm-xserve21
1 Windows 20100202165920 win32-slave49


Build 1

NOTE: We used clobberer for this releases

  • Clean up slaves
    • Linux: slave30,36,39,40,41,42,43,44,45,46,47,48,49,50
      • rm -rf /builds/slave/{tag,source,linux_build,linux_repack,updates,linux_update_verify,final_verification}/*
    • Win32: slave50,51,52,53,54,55,56,57,58,59
      • rm -rf /e/builds/moz2_slave/{win32_build,win32_repack,win32_update_verify,wince_build}/*
    • Mac: slave21,22,23,24,25,26; xserve21,22
      • rm -rf /builds/slave/{macosx_build,macosx_repack,l10n_verification, macosx_update_verify}/*

  • Landed automation version bumps, reconfigured production-master02
  • Kicked off automation
buildbot sendchange --username=armenzg --master=localhost:9010 --branch=releases/mozilla-1.9.1 -m "Firefox 3.5.8 build1" Go
  • Tag manually buildbot-configs, buildbotcustom and tools repositories
    • I didn't do it before triggering the release
    • hg pull -u && hg up -C
    • choose the revisions that were used
    • hg tag -r d2890e48e6e9 FIREFOX_3_5_8_BUILD1; hg tag -r d2890e48e6e9 FIREFOX_3_5_8_RELEASE (for buildbot-configs)
    • hg tag -r 7ad8012e2926 FIREFOX_3_5_8_RELEASE; hg tag -r 7ad8012e2926 FIREFOX_3_5_8_BUILD1 (for buildbotcustom)

Few days later

  • email received for beta snippets to be pushed live
  • snippets pushed live see "Publish updates to beta channel" section
  • patch for major update have been generated but not landed
  • logged in to fx-linux-1.9-slave2 and created the snippets by locally patching the patcher-configs
  • quick verify succeeded. Waiting for instructions on how to proceed


No problems


No problems


No problems


No Problems

  • Freed-up space in key master
    • df -h (we should have TBD GB available on /d/)
    • removed unsgined-buildN and signed-buildN of the last 4 releases, in our case:
      • ls -lrt
      • thunderbird-{2,3}
      • firefox-3.5.5
      • firefox-3.0.16
      • moved them to removed_binaries
      • starting with 23GB's free (80%)
  • Following instructions in Combined Signing


No problems

  • metadiff has output but it makes sense since we took "fr" changes for this dot release

Generate updates and push betatest snippets

No problems

Update verify

No problems

Major update (3.5.8 to 3.6.0)

Snippet generation
mkdir -p /builds/3.5.8-3.6-major-update
cd /builds/3.5.8-3.6-major-update

cvs co -d patcher-configs mozilla/tools/patcher-configs
cvs co -r UPDATE_PACKAGING_R10 -d patcher mozilla/tools/patcher
cd patcher
cvs co -r UPDATE_PACKAGING_R10 -d MozBuild mozilla/tools/release/MozBuild  

# build tools
./ --build-tools --app=firefox --tools-rev=UPDATE_PACKAGING_R10  --config=../patcher-configs/moz191-branch-major-update-patcher2.cfg 2>&1 | tee  ../firefox_build-tools.log

# fake 3.5.8 mars, which aren't used for anything but take time to d/l
mkdir -p temp/firefox/3.5.8/ftp/
for l in af ar as be bg bn-BD bn-IN ca cs cy da de el en-GB en-US eo es-AR es-CL es-ES es-MX et eu fa fi fr fy-NL ga-IE gl gu-IN he hi-IN hr hu id is it ja ja-JP-mac ka kk kn ko ku lt lv mk ml mn mr nb-NO nl nn-NO oc or pa-IN pl pt-BR pt-PT rm ro ru si sk sl sq sr sv-SE ta ta-LK te th tr uk vi zh-CN zh-TW; do
  touch temp/firefox/3.5.8/ftp/firefox-3.5.8.${l}.{linux-i686,mac,win32}.complete.mar

# download mars
./ --download --app=firefox --config=../patcher-configs/moz191-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_download.log

# FIXME - patcher needs to see that the MARs that it thinks are partials
#             are there or else it will not attempt to generate patchinfo
cd temp/firefox
mkdir 3.5.8-3.6
ln -s ../3.6/ftp 3.5.8-3.6/ftp
cd ../.. 

# Create partial patches and snippets
./ --create-patches --app=firefox --config=../patcher-configs/moz191-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox-create-patches.log
Quick verify

Check that releasetest = beta = release .

cd temp/firefox/3.5.8-3.6
# releasetest == beta
find aus2.test -type d -iregex '.*/releasetest$' | perl -nle '$a = $_; $a =~ s/releasetest/beta/; $a =~ s/aus2\.test/aus2/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/releasetest/; $a =~ s/aus2/aus2.test/; system("diff -r -u $_ $a");'

# beta == release
find aus2 -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta/release/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/beta/; system("diff -r -u $_ $a");'

# beta == betatest
find aus2.test -type d -iregex '.*/betatest$' | perl -nle '$a = $_; $a =~ s/betatest$/beta/; $a =~ s/aus2\.test/aus2/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/betatest/; $a =~ s/aus2/aus2.test/; system("diff -r -u $_ $a");'

No output, as expected.

Push to AUS
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.test/
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2/
Enable test snippets
# cltbld@aus2-staging
~/bin/backupsnip 20100210-Firefox-3.5.8-3.6-MU-test
~/bin/pushsnip   20100210-Firefox-3.5.8-3.6-MU-test
Update verify

Ran update verify on fx-{linux,mac,win32}-1.9-slave2 as follows:

mkdir -p /builds/verify/firefox-358-36-major/
cd /builds/verify/firefox-358-36-major/
hg clone

# get patch to trawl "Only in" directories
cd tools/release/common
# on mac & linux
curl -sL | patch -p3
# on win32
wget --no-check-certificate -O patch
patch -p3 < patch

# everyone ...
cd ../updates
platform=linux   # or mac or win32
./ -c moz191-firefox-$platform-major.cfg 2>&1 | tee $platform.log


  • No update found for mn, but that's not available for 3.6 so that's fine
  • Linux - no files or directories left over
  • Mac - leftover empty directories (updater not allowed to remove them)
Only in source/ Plug-Ins
Contents of source/ dir only in source or target
1775958570    0 drwxr-xr-x    3 cltbld   cltbld        102 Feb  2 16:58 source/
1775958571    0 drwxr-xr-x    4 cltbld   cltbld        136 Feb 10 15:00 source/
1775958572    0 drwxr-xr-x    2 cltbld   cltbld         68 Feb 10 15:00 source/
1775958573    0 drwxr-xr-x    3 cltbld   cltbld        102 Feb  2 16:58 source/
1775958574    0 drwxr-xr-x    3 cltbld   cltbld        102 Feb 10 15:00 source/
1775958575    0 drwxr-xr-x    2 cltbld   cltbld         68 Feb 10 15:00 source/
  • Windows: All the normal suspects, no problems with this (chk files will go away with bug 489961)
Only in source/bin: .autoreg
Binary files source/bin/freebl3.chk and target/bin/freebl3.chk differ
Binary files source/bin/nssdbm3.chk and target/bin/nssdbm3.chk differ
Only in source/bin: removed-files
Binary files source/bin/softokn3.chk and target/bin/softokn3.chk differ

Publish updates to beta channel

Feb.9th 12:30PM PST: NOTE: No email giving a heads up.

# cltbld @ aus2-staging
cd /opt/aus2/snippets/staging
# No need to run backupsnip, since jhford did it for 3.0.18
~/bin/pushsnip 20100203-Firefox-3.5.8-beta

bug 544458 for throttling changes for Fx 3.5.8 going to beta channel

Socorro product data updates

Someone else already enabled 3.5.8 and 3.5.9pre.

Update Bouncer

  • Entries added thanks to bouncer buddy

Final checks before push

# ffxbld@stage
cd /pub/
find . ! -user ffxbld #only the solaris contributed builds showed up
find . ! -group firefox
find . -type f ! -perm 644
find . -mindepth 1 -type d ! -perm 755 ! -name 'contrib*'
find . -maxdepth 1 -type d ! -perm 2775  -name 'contrib*'

Remove throttling on previous releases (3.5.7)

  • QUESTION: Do we do this everytime? or only for previous releases that have a major update on them?
  • bug 546594 - Throttling changes for Fx 3.5.8 going to release channel

Virus Scan

  • Filed a day before (Feb. 16th) release day
  • bug 546495 filed and waiting on ServOps
  • Nick run the virus check (this means that we can do it ourselves)
  • I moved this step above the step "push to mirrors" since it makes more sense
# stage
$ cd /pub/
$ clamdscan -m

----------- SCAN SUMMARY -----------
Infected files: 0
Time: 1781.993 sec (29 m 41 s) 

$ cd ~cltbld/firefox-3.0.18/stage-merged
$ clamdscan -m
/data/cltbld/firefox-3.0.18/stage-merged: OK 

----------- SCAN SUMMARY -----------
Infected files: 0
Time: 823.429 sec (13 m 43 s)

Push to mirrors

  • Feb. 17th, 2010 - 8:10AM PST - less than 25 minutes process
  • NOTE: sentry does not have a 3.5.8 entry. Pinged IT about it.
  • Filed bug 546704
# ffxbld@stage
rsync -av --exclude="*reporter*" --exclude=*.log --exclude=*.txt --exclude=*unsigned* /pub/ /pub/
# cltbld@stage
vim /pub/
# replace 3.5.7 with 3.5.8

Final Verification

  • few failures
  • Nick removed
  • All good

Push updates to release channel

  • Since bug 508406 we should run backupsnip every time we push, and not rely on it being run elsewhere
  • Upon tomorrow's GO we will have to push this: 20100203-Firefox-3.5.8 and 20100210-Firefox-3.5.8-3.6-MU
  • bug 546594 tracks unthrottling 3.5.7 updates - to land and go into production after 20100203-Firefox-3.5.8 overrides the major update currently in place (nthomas to co-ordinate)
# cltbld @ aus2-staging
cd /opt/aus2/snippets/staging
~/bin/backupsnip 20100203-Firefox-3.5.8
~/bin/pushsnip 20100203-Firefox-3.5.8
~/bin/backupsnip 20100210-Firefox-3.5.8-3.6-MU
~/bin/pushsnip 20100210-Firefox-3.5.8-3.6-MU


# ffxbld@stage
cd /home/ftp/pub/firefox/releases 
rm latest-3.5 && ln -s 3.5.8 latest-3.5

XULRunner build

  • This step doesn't have to be the last one

Triggered the builds by going to the buildbot waterfall for mozilla-1.9.1 and forcing builds for linux (not x64), mac and windows xulrunner. Set the field "Revision to build" (not properties) to 'FIREFOX_3_5_8_RELEASE'. Email the builds to release-drivers.

# as xrbld@stage on a production slave
mkdir -p /pub/
mv /pub/* /pub/ 
rm -rf 2010-02-17-10-mozilla-1.9.1
  • Waiting for reply from mfinkle or dtownsed