Releases/Firefox 3.6b4/BuildNotes

From MozillaWiki
Jump to: navigation, search

NB: Text that is italicized indicates tasks still to be done, or information that needs to be filled in.

Build Engineers

Tracking Bug: bug 530112

  • PST: lsblakk
  • EST:
  • NZDT: nthomas

Signed-off Revision(s)

Build 1: af0e2b9566cc

Tags

On mozilla-central:

Build # Tag Changeset
1 GECKO192b4_20091124_RELBRANCH af0e2b9566cc
FIREFOX_3_6b4_BUILD1 FIREFOX_3_6b4_RELEASE 4c488520d1bf

Build data

Build 1:

Platform Build ID Build machine
Win32 20091124213835 moz2-win32-slave33
Mac 20091124201530 moz2-darwin9-slave09
Linux 20091124201751 moz2-linux-slave12
Wince 20091124202522 moz2-win32-slave42

Notes

Done on pm01

Build 1

  • Clean up previous releases build dirs, just for the slaves we used for 3.6b3 build1. Used cssh for linux/mac, RDP as Administrator on win32.
    • Linux: linux-slave01,02,05,06,07,08,09,10,18,20,22,28; rm -rf /builds/slave/{tag,source,linux_build,linux_repack,updates,linux_update_verify,final_verification}/*
    • Mac:darwin9-slave02,05,06,07,09,10,12, xserve07,09; rm -rf /builds/slave/{macosx_build,macosx_repack,l10n_verification, macosx_update_verify}/*
    • Win32: win32-slave01,02,05,06,07,08,09,10,16,31,40; rm -rf /e/builds/moz2_slave/{win32_build,win32_repack,win32_update_verify,wince_build}/*
  • bump mozilla-1.9.2 to 3.6b5pre with patch
  • Get review and land [buildbot-configs patch], and re-enable ftp poller patch
  • Closed the mozilla-1.9.2 tree
  • Tagged build/{buildbot-configs,buildbotcustom,tools} with FIREFOX_3_6b4_RELEASE
  • ONE-OFF - land sendchange hang fix from bug 528212 and restart master
  • Sendchange was
buildbot sendchange --username=nthomas --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6b4 build1" ShakeAndBakeBaby

Tag

  • slave: moz2-linux-slave08
  • Land version bumps patch
  • Re-opened mozilla-1.9.2
  • No problems

Source

  • slave: moz2-linux-slave42
  • No problems

Build/Repack

  • slaves
  • Linux and Win32 no problems.
  • Mac l10n bombed out because the slaves had not been cleaned up. Fix that (can't get l10n_verify and update verify on moz2-darwin9-slave05 because it needs rebooting) and then
# cltbld@production-master
cd ~/tools
hg pull
hg up -r FIREFOX_3_6b4_RELEASE
cd buildbot-helpers/
python force_release_l10n.py -m http://localhost:8010 -t FIREFOX_3_6b4_RELEASE -v -b releases/mozilla-1.9.2 -p macosx -n nthomas
  • had failed to clean up bm-xserve07 and 09 so retrigger the locales that 15 locales that failed (af ar be bg bn-BD bn-IN ca cs cy da de el en-GB eo es-AR written to shipped-locales-macosx)
python force_release_l10n.py -m http://localhost:8010 -t FIREFOX_3_6b4_RELEASE -v -s shipped-locales-macosx -p macosx -n nthomas
  • Manual verification that all l10n builds present:
# stage
cd /home/ftp/pub/firefox/nightly/3.6b4-candidates/build1
# each should be 68 = en-US  + 67 l10n builds
for p in linux-i686 mac unsigned/win32; do
  echo `ls $p | grep -v xpi | wc -l` $p
done
  • Removed en-US.xpi from the 3.6b4-candidates/build1/ directory on stage

Socorro product data updates

Since Socorro 1.1 went live this is done with an Admin control panel, so asked Sam Sidler (ss) to enable those versions. That requires at least one crash report so created those using the CrashMe extension with linux 3.6b4 and 3.6b5pre builds.

Sign

No problems

L10nVerify

  • slave: bm-xserve12
  • Got "only in..." messages for locales which are new in b4 or missed b3
  • no real differences

Generate updates and push betatest snippets

  • slave: moz2-linux-slave01

As well as the automated updates we'll need to connect 3.6b1 to b4 for af bg da, similar to what we did last time.

Fill in b1 locales
  • slave: moz2-linux-slave01

Needed to generate b1 -> b4 for 3.6b1 af bg da. Ran steps manually with this patcher config

Moved the original run out of the way: 

cd /builds/slave/updates/build
mv temp temp.original-run
cd patcher-configs
mv moz192-branch-patcher2.cfg moz192-branch-patcher2.cfg-original-run
cd ../

#download builds
perl patcher2.pl --download --app=firefox --brand=Firefox --config=patcher-configs/moz192-branch-patcher2.cfg

#create patches
perl patcher2.pl --create-patches --partial-patchlist-file=patchlist.cfg --app=firefox --brand=Firefox --config=patcher-configs/moz192-branch-patcher2.cfg

#chmod partial mars:
find temp/firefox/3.6b1-3.6b4/ftp/firefox/nightly/3.6b4-candidates/build1 -type f -exec chmod 644 {} \;

#chmod partial mar dirs:
find temp/firefox/3.6b1-3.6b4/ftp/firefox/nightly/3.6b4-candidates/build1 -type d -exec chmod 755 {} \;

#upload partial mars:
rsync -av -e 'ssh -oIdentityFile=~/.ssh/ffxbld_dsa' --exclude=*complete.mar temp/firefox/3.6b1-3.6b4/ftp/firefox/nightly/3.6b4-candidates/build1/update ffxbld@stage-old.mozilla.org:/pub/mozilla.org/firefox/nightly/3.6b4-candidates/build1/

#upload snippets:
rsync -av temp/firefox/3.6b1-3.6b4/aus2/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20091125-Firefox-3.6b1-3.6b4-locales

#upload test snippets
rsync -av temp/firefox/3.6b1-3.6b4/aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20091125-Firefox-3.6b1-3.6b4-locales-test

Then on aus2-staging as cltbld, 

cd /opt/aus2/snippets/staging
# Didn't run backupsnip because the automation one was running at the time
~/bin/pushsnip 20091125-Firefox-3.6b1-3.6b4-locales-test/
Generate 3.6b3 -> b4 for WinCE
  • slave: moz2-linux-slave01 (same slave as the automated run)
# move the previous run aside
cd /builds/slave/updates/build
mv temp temp.b1
cd patcher-configs
wget -Omoz192-branch-patcher2.cfg https://bug530112.bugzilla.mozilla.org/attachment.cgi?id=414504

cd /builds/slave/updates/build
perl patcher2.pl --download --app=firefox --brand=Firefox --config=patcher-configs/moz192-branch-patcher2.cfg
perl patcher2.pl --create-patches --partial-patchlist-file=patchlist.cfg --app=firefox --brand=Firefox --config=patcher-configs/moz192-branch-patcher2.cfg
find temp/firefox/3.6b3-3.6b4/ftp/firefox/nightly/3.6b4-candidates/build1 -type f -exec chmod 644 {} \;
find temp/firefox/3.6b3-3.6b4/ftp/firefox/nightly/3.6b4-candidates/build1 -type d -exec chmod 755 {} \;
cd /builds/slave/updates/build/temp/firefox/3.6b3-3.6b4/ftp/firefox/nightly/3.6b4-candidates/build1
rsync -av -e 'ssh -oIdentityFile=~/.ssh/ffxbld_dsa' --exclude=*complete.mar update ffxbld@stage.mozilla.org:/pub/mozilla.org/firefox/nightly/3.6b4-candidates/build1/
cd /builds/slave/updates/build/temp/firefox/3.6b3-3.6b4
rsync -av aus2/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20091125-Firefox-3.6b4-WinCE
rsync -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20091125-Firefox-3.6b4-WinCE-test

# cltbld@aus2-staging
# backupsnip in automation was still running, didn't do one
~/bin/pushsnip 20091125-Firefox-3.6b4-WinCE-test

Update verify

  • All platforms' partials had differences in removed-files caused by packaging fixes landed for the 3.5 -> 3.6 MU, bug 515421. They are not a real issue. Eg linux
diff -r source/firefox/removed-files target/firefox/removed-files
500a501,502
> res/broken-image.gif
> res/loading-image.gif
533a536,537
> icons/mozicon16.xpm
> icons/mozicon50.xpm
537a542
> old-homepage-default.properties
  • All platforms: 3.6b1 ka failed because we still don't have that locale back
  • Linux: No real problems.
  • Mac: got an error checking b2 fr where complete mars were 0-byte, which I assumed was because the files got removed while they were respun. Lead to the discovery that we need to generate b1 and b2 snippets for fr (see below)
  • Win32: Three chk file differences as normal

Respin of fr

A problem was discovered with the fr build, so we've got to respin it.

clean up the slaves

Because we can't predict which slave it will happen on we have to clean up all of the l10n dirs on the slaves.

  • Linux: 01,02,05-10
  • Mac: moz2-darwin9-slave02,05,06,07,09,10,bm-xserve07,09
  • Win32: win32-slave01,02,05-10
force the build

Used the web interface to force the fr build for each platform with the following properties:

  • en_revision = FIREFOX_3_6b4_RELEASE
  • l10n_revision = FIREFOX_3_6b4_RELEASE
  • locale = fr

builds went fine, uploads failed with (FIXME): 

Traceback (most recent call last):
  File "/usr/bin/post_upload.py", line 262, in ?
    func(options, upload_dir, files)
  File "/usr/bin/post_upload.py", line 156, in ReleaseToCandidatesDir
    os.chmod(os.path.join(root, d), 0755)
OSError: [Errno 1] Operation not permitted: '/home/ftp/pub/firefox/nightly/3.6b4-candidates/build1/contrib/solaris_pkgadd'
Encountered error while uploading
Command ['ssh', '-o', 'IdentityFile=~/.ssh/ffxbld_dsa', 'ffxbld@stage.mozilla.org', 'post_upload.py -p firefox -v 3.6b4 -n 1 --release-to-candidates-dir "/tmp/tmp.IqJdUJ9377/" "/tmp/tmp.IqJdUJ9377/linux-i686/fr/firefox-3.6b4.tar.bz2" "/tmp/tmp.IqJdUJ9377/linux-i686/xpi/fr.xpi"'] returned non-zero exit code: 1
make: *** [l10n-upload-fr] Error 2

Had to finish the upload with: 

cd /builds/slave/linux_repack/build/mozilla-1.9.2/dist/update/linux-i686
rsync -e 'ssh -oIdentityFile=~/.ssh/ffxbld_dsa' -av fr ffxbld@stage.mozilla.org:/home/ftp/pub/firefox/nightly/3.6b4-candidates/build1/update/linux-i686/

...and equivalent commands for the other platforms. Had to upload the win32 installer, too.

signing
  • Done mostly manually:
  • Only pulled win32 en-US (build1) and linux/mac/win32 zh-CN (build2), MD5SUMS, SHA1SUMS into unsigned-build2:
PRODUCT=firefox
VERSION=3.6b4
BUILD=1
TAG=FIREFOX_3_6b4_RELEASE
REPO='releases/mozilla-1.9.2'
EMAIL=bhearsum@mozilla.com
KEYDIR=d:/2009-keys
mkdir ~/signing-work/${PRODUCT}-${VERSION}-fr2
cd ~/signing-work/${PRODUCT}-${VERSION}-fr2
cp ~/hg-tools/release/signing/* .
make setup PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
rsync -av -e ssh --include=*SUMS --include=xpi --include=linux-i686/ --include=mac/ --include=unsigned/ --include=*irefox* --include=win32  --include=*fr* --include=update/ --exclude=* cltbld@stage.mozilla.org:/home/ftp/pub/${PRODUCT}/nightly/${VERSION}-candidates/build${BUILD}/ unsigned-build${BUILD}
 rsync -av -e ssh --include=unsigned/ --include=*irefox* --include=win32  --include=*en-US* --include=update/ --exclude=* cltbld@stage.mozilla.org:/home/ftp/pub/${PRODUCT}/nightly/${VERSION}-candidates/build1/ unsigned-build${BUILD}
rsync -av -e ssh --exclude=*partial* --exclude=*asc --include=win32 --include=*irefox* --include=*en-US* --include update/ --exclude=* cltbld@stage.mozilla.org:/home/ftp/pub/${PRODUCT}/nightly/${VERSION}-candidates/build1/ signed-build${BUILD}
 make stubs PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
./sign-release.py -p --keydir ${KEYDIR} -j1 --product firefox -o signed-build${BUILD} unsigned-build${BUILD} >> win32_signing_build${BUILD}.log 2>&1
rsync -av --include=*.mar --include=update --include=*SUMS --include=*dmg --include=*.tar.bz2 --include=*xpi* --include=*fr* --include=*linux* --include=*mac* --exclude=* unsigned-build1/ signed-build1/
rsync -av ./unsigned-build${BUILD}/unsigned/win32/xpi ./signed-build${BUILD}/win32
# manually updated the SUMS files with new md5/sha1 sums
cd signed-build1
../signing/sign-files .
rm win32/en-US/*.asc
cd ../
# push builds back
rsync -av -e 'ssh -oIdentityFile=~/.ssh/ffxbld_dsa' signed-build1/ ffxbld@stage.mozilla.org:/home/ftp/pub/firefox/nightly/3.6b4-candidates/build1/
updates
cd /builds/slave/updates/build
perl patcher2.pl --download --app=firefox --brand=Firefox --config=patcher-configs/moz192-branch-patcher2.cfg
perl patcher2.pl --create-patches --partial-patchlist-file=patchlist.cfg --app=firefox --brand=Firefox --config=patcher-configs/moz192-branch-patcher2.cfg
find temp/firefox/3.6b3-3.6b4/ftp/firefox/nightly/3.6b4-candidates/build1 -type f -exec chmod 644 {} \;
find temp/firefox/3.6b3-3.6b4/ftp/firefox/nightly/3.6b4-candidates/build1 -type d -exec chmod 755 {} \;
cd /builds/slave/updates/build/temp/firefox/3.6b3-3.6b4/ftp/firefox/nightly/3.6b4-candidates/build1
rsync -av -e 'ssh -oIdentityFile=~/.ssh/ffxbld_dsa' --exclude=*complete.mar update ffxbld@stage.mozilla.org:/pub/mozilla.org/firefox/nightly/3.6b4-candidates/build1/
cd /builds/slave/updates/build/temp/firefox/3.6b3-3.6b4
rsync -av aus2/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20091125-Firefox-3.6b4-fr-respin
rsync -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20091125-Firefox-3.6b4-fr-respin-test

# cltbld @ aus2-staging
cd /opt/aus2/snippets/staging
~/bin/pushsnip 20091125-Firefox-3.6b4-fr-respin-test

This only regenerated the snippets for b3 and we also need to do b1 and b2 (because the complete mar hash changed). Redo using this config 

# cltbld@moz2-linux-slave01
cd /builds/slave/updates/build/temp/firefox/3.6b3-3.6b4
mkdir old-aus
mv aus2 old-aus
cd ../../..
perl patcher2.pl --create-patchinfo --app=firefox --brand=Firefox --config=patcher-configs/moz192-branch-patcher2.cfg
cd temp/firefox/3.6b3-3.6b4/
diff -rq {old-aus/,}aus2.test
 Only in aus2.test/Firefox: 3.6b1
 Only in aus2.test/Firefox: 3.6b2
diff -rq {old-aus/,}aus2
 Only in aus2/Firefox: 3.6b1
 Only in aus2/Firefox: 3.6b2
rsync -av aus2/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20091125-Firefox-3.6b4-fr-respin-2
rsync -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20091125-Firefox-3.6b4-fr-respin-2-test

# cltbld @ aus2-staging
 ~/bin/pushsnip 20091125-Firefox-3.6b4-fr-respin-2-test

Update Bouncer

Added the usual 3 products and 9 file locations, plus the three locations for WinCE.

Added Firefox-3.6b4-Partial-3.6b1 product & 3 file locations for straggler locales. Sentry is looking for en-US copies of these files, so fake this out with zero byte files 

# ffxbld@stage
touch /pub/mozilla.org/firefox/nightly/3.6b4-candidates/build1/update/{linux-i686,mac,win32}/en-US/firefox-3.6b1-3.6b4.partial.mar

Final checks before push

cd /pub/mozilla.org/firefox/nightly/3.6b4-candidates/build1/
find . ! -user ffxbld
find . ! -group firefox
find . -type f ! -perm 644
find . -mindepth 1 -type d ! -perm 755 ! -name 'contrib*'
find . -maxdepth 1 -type d ! -perm 2775  -name 'contrib*'

Only returns Solaris builds (different user) for the user check. Had to fix the mode on the respun fr mar files and 2775 mode on the contrib dirs.

Push to mirrors

# ffxbld@stage
rsync -av --exclude=*.log --exclude=*.txt --exclude=*unsigned* --exclude=*.crashreporter-symbols.zip /pub/mozilla.org/firefox/nightly/3.6b4-candidates/build1/ /pub/mozilla.org/firefox/releases/3.6b4/

Removed 3.6b3 from /pub/mozilla.org/zz/rsyncd-mozilla-current.exclude as we don't need lots of mirror coverage and the biggest mirror syncs very quickly from the mozilla-releases module.

Final Verification

  • slave: moz2-linux-slave13
  • fails to find updates for 3.6b1 ka, as expected

Push updates to release channel

Well ahead of time 

time ~/bin/backupsnip 20091124-Firefox-3.6b4
real    61m52.014s
user    0m55.967s
sys     1m39.784s

~/bin/pushsnip 20091124-Firefox-3.6b4
~/bin/pushsnip 20091125-Firefox-3.6b4-fr-respin-2
~/bin/pushsnip 20091125-Firefox-3.6b1-3.6b4-locales

This part happened the following monday. 

~/bin/pushsnip 20091125-Firefox-3.6b4-WinCE

Release

No symlink until final

XULRunner build

Builds are forced manually, using revision of FIREFOX_3_6b4_RELEASE, name and reason. When they complete 

# xrbld@stage
cd /pub/mozilla.org/xulrunner/nightly/
mkdir -p 1.9.2b4-candidates/build1/
mv -v latest-mozilla-1.9.2/*-1.9.2b4.* 1.9.2b4-candidates/build1/

Contacted mfinkle and dtownsend for smoketest QA.

Staging and push to mirrors pending.

Retrieved from "https://wiki.mozilla.org/index.php?title=Releases/Firefox_3.6b4/BuildNotes&oldid=190067"