Releases/Firefox 3.5.2/BuildNotes

From MozillaWiki
Jump to: navigation, search

Build Engineers

nthomas, bhearsum

Signed-off Revision(s)

Build1: 7a9fe7dfac8a

Tags

On releases/mozilla-1.9.1:

Build # Tag Changeset
1 GECKO1912_20090729_RELBRANCH 7a9fe7dfac8a
FIREFOX_3_5_2_BUILD1, FIREFOX_3_5_2_RELEASE Was:b8dbd5ab1647 Became:d57f05bcdbe1 Then:a6308d41af58

Build data

Type Build ID Build machine
[Windows installer/zip]
[Mac compressed]
[Linux compressed]

Notes

Build 1

  • Cleaned up previous releases' build dirs
  • l10n-changesets updated in bug 505129
  • Land version bump, reconfig production-master
  • Kick-off automation:
buildbot sendchange --username=nthomas --master=localhost:9010 --branch=releases/mozilla-1.9.1 -m "Firefox 3.5.2 build1" goforit

Tag

No problems with tag (see below for where it got moved)

Source

No problems

Build/Repack

Had to remove en-US.xpi manually from stage FIXME No problems generating en-US builds, but linux locales started failing in the make_installers_locale step after generating tar.bz2 and complete.mar: 

/tools/python/bin/python ../../tools/update-packaging/generatesnippet.py \
         --mar-path=/builds/moz2_slave/linux_repack/build/mozilla-1.9.1/dist/update \
         --application-ini-file=/builds/moz2_slave/linux_repack/build/mozilla-1.9.1/dist/l10n-stage/firefox/application.ini \
         --locale=or \
         --product=Firefox \
         --download-base-URL= \
         --verbose
/tools/python/bin/python: can't open file '../../tools/update-packaging/generatesnippet.py': [Errno 2] No such file or directory

This is fallout from a bad merge in bug 489313, as generatesnippet.py did not land on mozilla-1.9.1. Mac also hit it because it's not platform-dependent.

Recovery

  • did a no-op reconfig on master to prevent win32 repacks from starting
  • Fix was landed on the relbranch and default, and the two tags moved up to d57f05bcdbe1
  • automation patched to disable tag and en-US builds
  • cleaned up mac and linux slaves of repack dirs and source
  • submitted another sendchange

Mac repacks were broken because of bug 489313. Root cause hasn't been determined yet, see below for fixup.

mac repack rerun
  • Backed out bug 489313 on the release branch and moved the _BUILD1 and _RELEASE tags up.
  • cleaned up the macosx_repack slave
  • Deleted all l10n builds in .../build1/mac and ../build1/update/mac on stage.
  • Landed this patch, updated the master and reconfiged
  • Did another sendchange

no problems with the re-run. Detached PGP sigs were regenerated when resigning windows repacks.

win32 repack rerun

Don't need to land anything in mozilla-1.9.1 because the backout we did for mac will suffice

  • cleaned up the win32_repack slaves
  • Moved all l10n builds in .../build1/unsigned/win32, .../build1/unsigned/update/win32, .../build1/update/win32, .../build1/win32 to ~ffxbld/3.5.2-busted/
  • Landed this patch, updated the master and reconfiged
  • did another sendchange
  • temporarily enabled some extra slaves (up to slave16) for win32 repacks to speed things up
    • this ended up backfiring a little after removing them from the l10n slaves list before they finished building - it cause the builds running on slave11 and slave14 to hang, and they had to be kicked through manually

Sign

instructions

  • Kicked off new signing on cm-keymaster01 and old style on keymaster at the same time
  • new signing bailed out at verify-asc because SUMS files were not signed, but verify-asc thought they should be (FIXME). we were trying to push out signed *SUMS files this time. to save time but since we've never done it in the past, just pushed things along with unsigned *SUMS files by running the 'fake-upload' and then 'postsign' targets
signing # 2

Had to resign after win32 l10n builds were regenerated. sign-release.py was called directly to add -p argument, then worked through the following makefile targets. asc files for linux and mac were replaced.

L10nVerify

After redoing macosx locales: PASS. All locales with real changes have an updated l10n-changesets revision (got that list by doing 'hg diff -r FIREFOX_3_5_1_RELEASE -r FIREFOX_3_5_2_RELEASE mozilla2/l10n-changesets)

After redoing win32 locales: looks fine, in particular no win32 specific problems

Generate updates and push betatest snippets

  • Update generation had 'WARNING' messages about not being able to find application.ini on Mac. This comes from get_buildid in make_incremental_updates.py, but we never actually use the output of that function which makes this a superfluous message. Noticed it in the 3.0.12 logs, too. bug 507405
  • updates were stopped midstream after the mac problem was discovered and re-kicked with 'force build' after mac builds were fixed. this caused the 'diff patcher config' step to turn red because it expected diff output, but the config had already been bumped.
  • no real problems then

After fixing win32 locales: 

do these steps to generate updats
# cltbld@moz2-linux-slave06
cd /builds/slave/updates/build
rm -rv temp/firefox/3.5.2/ftp/firefox-3.5.2.*.win32.complete.mar
rsync -a temp/firefox/3.5.1-3.5.2/ temp/firefox/3.5.1-3.5.2-bak/
rm -rf temp/firefox/3.5.1-3.5.2/ftp/firefox/nightly/3.5.2-candidates/build1/update/win32/{af,ar,as,be,bg,bn-BD,bn-IN,ca,cs,cy,da,de,el,en-GB,eo,es-AR,es-CL,es-ES,es-MX,et,eu,fa,fi,fr,fy-NL,ga-IE,gl,gu-IN,he,hi-IN,hr,hu,id,is,it,ja,ka,kk,kn,ko,ku,lt,lv,mk,ml,mn,mr,nb-NO,nl,nn-NO,oc,or,pa-IN,pl,pt-BR,pt-PT,rm,ro,ru,si,sk,sl,sq,sr,sv-SE,ta,ta-LK,te,th,tr,uk,vi,zh-CN,zh-TW}
rm -rf temp/firefox/3.5.1-3.5.2/aus2*

perl patcher2.pl --download --app=firefox --brand=Firefox --config=patcher-configs/moz191-branch-patcher2.cfg
perl patcher2.pl --create-patches --partial-patchlist-file=patchlist.cfg --app=firefox --brand=Firefox --config=patcher-configs/moz191-branch-patcher2.cfg

find temp/firefox/3.5.1-3.5.2/ftp/firefox/nightly/3.5.2-candidates/build1 -type f -exec chmod 644 {} \;
find temp/firefox/3.5.1-3.5.2/ftp/firefox/nightly/3.5.2-candidates/build1 -type d -exec chmod 755 {} \;
cd temp/firefox/
# for each of aus2, aus2.test, aus2.beta: verify that only WINNT have changed but not en-US, # of differences is the # of partials

rsync -av -e "ssh -oIdentityFile=~/.ssh/ffxbld_dsa" --exclude=*complete.mar 3.5.1-3.5.2/ftp/firefox/nightly/3.5.2-candidates/build1/update ffxbld@stage-old.mozilla.org:/pub/mozilla.org/firefox/nightly/3.5.2-candidates/build1/
rsync -av 3.5.1-3.5.2/aus2/      cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090730-Firefox-3.5.2-fixedWin32
rsync -av 3.5.1-3.5.2/aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090730-Firefox-3.5.2-fixedWin32-test
rsync -av 3.5.1-3.5.2/aus2.beta/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090730-Firefox-3.5.2-fixedWin32-beta
ssh -l cltbld aus2-staging.mozilla.org ~/bin/pushsnip 20090730-Firefox-3.5.2-fixedWin32-test

Update verify

  • win32 update verify logs showed that the helper.exe in the installers, and the helper.exe in the MARs was different. this is also fallout from bug 489313, just like the mac problems. win32 OK otherwise
  • linux - full PASS
  • mac - full PASS

Publish updates to beta channel

# login with LDAP account onto aus2-staging, then do 
# sudo su - cltbld
cd /opt/aus2/snippets/staging
time ~/bin/backupsnip 20090730-Firefox-3.5.2-fixedWin32-beta
real    47m29.661s
user    0m46.668s
sys     1m8.102s
time ~/bin/pushsnip 20090730-Firefox-3.5.2-fixedWin32-beta
real    2m28.442s
user    0m0.382s
sys     0m20.070s

Update Bouncer

done

Final checks before push

Each of the find calls should return no output. 

cd /pub/mozilla.org/firefox/nightly/3.5.2-candidates/build1/
find . ! -user ffxbld
find . ! -group firefox
find . -type f ! -perm 644
find . -type d -mindepth 1 ! -perm 755 ! -name 'contrib*'
find . -type d -maxdepth 1 ! -perm 2775  -name 'contrib*'

No Solaris builds at this point, so no output from any of these commands.

Push to mirrors

# ffxbld@stage
rsync -av --exclude=*.log --exclude=*.txt --exclude=*unsigned* /pub/mozilla.org/firefox/nightly/3.5.2-candidates/build1/ /pub/mozilla.org/firefox/releases/3.5.2/

# cltbld@stage
vim /pub/mozilla.org/zz/rsyncd-mozilla-current.exclude
# replace 3.5.1 with 3.5.2

Final Verification

  • kick off 'final_verification' builder on production-master after we have some uptake. if it goes green everything is OK, if red look at the log for 404 or 503 errors.
  • got nine FAIL's trying to query mozilla-mirror.3347.voxcdn.com and victoria.acc.umu.se, now working fine

Push updates to release channel

# cltbld @ aus2-staging
cd /opt/aus2/snippets/staging
screen
# Skipped this time, because the earlier backupsnip was just a few hours ago
time ~/bin/backupsnip 20090730-Firefox-3.5.2-fixedWin32
real    50m56.618s
user    0m46.908s
sys     1m0.209s
time ~/bin/pushsnip 20090730-Firefox-3.5.2-fixedWin32
real    1m6.163s
user    0m0.154s
sys     0m5.773s

Release

# login as cltbld to any of the production build machines. From there, do:
# ssh -l ffxbld -i ~/.ssh/ffxbld_dsa stage.m.o
cd /home/ftp/pub/firefox/releases 
rm latest-3.5 && ln -s 3.5.2 latest-3.5

XULRunner build

Tracked separately in bug 505701.

Create the builds by going to the buildbot waterfall for mozilla-1.9.1 and forcing builds for linux (not x64), mac and windows xulrunner. Set the branch to 'releases/mozilla-1.9.1' and revision to 'FIREFOX_3_5_1_RELEASE'.

When they finish, as xrbld@stage move the dir from /pub/mozilla.org/xulrunner/nightly/2009/07/2009-07-xx-xx-mozilla-1.9.1/ to ..../nightly/1.9.1.1-candidates/build1. Remove the broken symlink at xulrunner/nightly/2009-07-xx-xx-mozilla-1.9.1. Contact mfinkle and dtownsend for smoketest QA.

Publishing is done in a similar (but not identical!) way as 3.0.13: 

# cltbld@stage
VERSION=1.9.1.2
FX_VERSION=3.5.2
BUILD=1

mkdir -p ~/xulrunner-${VERSION}/batch1/stage-unsigned/{runtimes,source,sdk}
cd ~/xulrunner-${VERSION}/batch1/stage-unsigned
rsync -av --exclude=*sdk* --exclude=*info* \
  /home/ftp/pub/xulrunner/nightly/${VERSION}-candidates/build${BUILD}/ runtimes/
rsync -av --include=*sdk* --exclude=* \
  /home/ftp/pub/xulrunner/nightly/${VERSION}-candidates/build${BUILD}/ sdk/
cp -pv /pub/mozilla.org/firefox/releases/${FX_VERSION}/source/firefox-${FX_VERSION}-source.tar.bz2 \
  source/xulrunner-${VERSION}-source.tar.bz2
cd ~/xulrunner-${VERSION}/batch1
rsync -av stage-unsigned/ stage-signed/

Then create detached signatures per usual process (with PRODUCT=xulrunner, VERSION=1.9.0.13, don't need BUILD or TAG), skip the win32 signing section, do pgp, verify and upload. Back on stage: 

cd ~/xulrunner-${VERSION}/batch1/stage-signed
cp /pub/mozilla.org/firefox/releases/${FX_VERSION}/KEY .
~/bin/checksum-files .
chown -R cltbld:xulrunner .
find . -type f -exec chmod -v 644 {} \;
find . -type d -exec chmod -v 755 {} \;
cd ~/xulrunner-${VERSION}/batch1
rsync -nav stage-signed/* /home/ftp/pub/xulrunner/releases/${VERSION}/

If all is fine then 

rsync -av stage-signed/* /home/ftp/pub/xulrunner/releases/${VERSION}/

Update the links in these two wiki pages for the version change

Retrieved from "https://wiki.mozilla.org/index.php?title=Releases/Firefox_3.5.2/BuildNotes&oldid=160245"