Releases/Firefox 3.6.3/BuildNotes

From MozillaWiki
Jump to: navigation, search

Build Engineers

bhearsum

Signed-off Revision(s)

http://hg.mozilla.org/releases/mozilla-1.9.2/rev/3093de37377d

Tags

On releases/mozilla-1.9.2:

Build # Tag Changeset
1 GECKO1922_20100315_RELBRANCH 3093de37377d
FIREFOX_3_6_3_BUILD1 FIREFOX_3_6_3_RELEASE d6e028dc1b68

Build data

Build # Type Build ID Build machine
1 Linux 20100401074458 mv-moz2-linux-ix-slave12
Mac 20100401064631 bm-xserve21
Windows 20100401080539 mw32-ix-slave17

Notes

Build 1

  • Clobbered 'mozilla-1.9.2' on pm02 with clobberer.
  • landed backout of changes to l10n-changesets_mozilla-1.9.2 since 3.6.2.
  • Landed automation config
  • Reconfiged pm02
  • Started the automation:
buildbot sendchange --username=bhearsum --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6.3 build1" Go

Tag

No problems.

Source

No problems

HALT. Had to restart after this because tagging was done improperly (baseTag was still set to FIREFOX_3_6_2, which made us move the 3.6.2 tags rather than create 3.6.3 ones).]

Build 1 (redux)

Don't need to do a build2 since the build1 tags were never created. Cleanup from the screwup:

  • Set clobberer to clobber mozilla-1.9.2 on pm02 again
  • Deleted 3.6.3-candidates from stage (only had source tarball/bundle in it)
  • Reverted tag commits on mozilla-1.9.2 repo:
hg up -C GECKO1922_20100315_RELBRANCH
hg tag -f -r f14062c981ba -m "Revert accidental movement of FIREFOX_3_6_2_RELEASE tag." FIREFOX_3_6_2_RELEASE
hg tag -f -r f14062c981ba -m "Revert accidental movement of FIREFOX_3_6_2_BUILD1 tag." FIREFOX_3_6_2_BUILD1
hg push ssh://hg.mozilla.org/releases/mozilla-1.9.2

buildbot sendchange --username=bhearsum --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6.3 build1" Go

Tag

No problems. The tag ended up on http://hg.mozilla.org/releases/mozilla-1.9.2/rev/28ef231a65a3 instead of http://hg.mozilla.org/releases/mozilla-1.9.2/rev/d6e028dc1b68, but that makes no difference as far as the builds are concerned.

Source

Dep scheduler didn't fire. No problems

Build/Repack

Dep scheduler didn't fire. Forced builds manually until fast slaves (ix/xserve) were building, then killed the other ones (per release-drivers request).
Forced l10n builds with helper scripts: 

python force_release_l10n.py -m http://production-master02.build.mozilla.org:8010 -t FIREFOX_3_6_3_RELEASE -v -b releases/mozilla-1.9.2 -p macosx -n bhearsum
python force_release_l10n.py -m http://production-master02.build.mozilla.org:8010 -t FIREFOX_3_6_3_RELEASE -v -b releases/mozilla-1.9.2 -p linux -n bhearsum
python force_release_l10n.py -m http://production-master02.build.mozilla.org:8010 -t FIREFOX_3_6_3_RELEASE -v -b releases/mozilla-1.9.2 -p win32 -n bhearsum
  • Mac: no en-US problems, no l10n problems
  • Linux: no en-US build problems, very slow uploads, no l10n problems
  • Win32: same as Linux

Deleted en-US.xpi from stage.

Partner Repacks

Can't do these from the automation until bug 554321 is resolved.

Unit tests

  • Mac: PASS
  • Win32: PASS
  • Linux: first run of mochitest-browser-chrome was orange, with an intermittent failure. Next run was green.

Sign

Started downloading builds early to speed up end to end time. Signing happened like this: 

export PRODUCT=firefox
export VERSION=3.6.3
export BUILD=1
export TAG=FIREFOX_3_6_3_RELEASE
export REPO='releases/mozilla-1.9.2'
export EMAIL=bhearsum@mozilla.com
export KEYDIR=d:/2009-keys

cd ~/hg-tools
hg pull && hg up -r ${TAG}

mkdir ~/signing-work/${PRODUCT}-${VERSION}
cd ~/signing-work/${PRODUCT}-${VERSION}
cp ~/hg-tools/release/signing/* .

make setup PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
# This step was run a few times, every 10 minutes or so until all builds finished
make download PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
make stubs sign2 stage verify fake-upload PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1

make postsign PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL}

L10nVerify

The post_signing Scheduler didn't look like it was going to fire so I forced a build of this. The Scheduler fired later, and I cancelled that build.
No differences, this is a full PASS.

Generate updates and push betatest snippets

The post_signing Scheduler didn't like it was going to fire so I forced a build of this. The Scheduler fired later, and I cancelled that build.

  • Snippet uploads failed because of changes in bug 511967. Need to fix permissions on aus2-staging, or upload with a different user for releases. bug 557195 file bug to track
  • Kicked through manually with:
# cltbld @ moz2-linux-slave43
cd /builds/slave/updates/temp/firefox/3.6.2-3.6.3/
rsync -av -e 'ssh -oIdentityFile=~/.ssh/id_dsa' aus2/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100401-Firefox-3.6.3
rsync -av -e 'ssh -oIdentityFile=~/.ssh/id_dsa' aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100401-Firefox-3.6.3-test
rsync -av -e 'ssh -oIdentityFile=~/.ssh/id_dsa' aus2.beta/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100401-Firefox-3.6.3-beta

And then, 

# cltbld @ aus2-staging
cd /opt/aus2/snippets/staging
~/bin/backupsnip 20100401-Firefox-3.6.3-test
~/bin/pushsnip 20100401-Firefox-3.6.3-test

Update verify

  • Linux: PASS
  • Win32: PASS
  • Mac: PASS

Socorro product data updates

We can do this ourselves (a subset of relengs, ask nthomas or bhearsum or joduinn). No need to file an IT bug for this.

joduinn added 3.6.3 and 3.6.4pre with the crash-stats admin page.

Update Bouncer

Added with BouncerBuddy.

Final checks before push

# ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/3.6.3-candidates/build1/
find . ! -user ffxbld #nothing showed up, as expected.
find . ! -group firefox #nothing showed up, as expected.
find . -type f ! -perm 644 # unsigned/partner-repacks have 664 permission
find . -mindepth 1 -type d ! -perm 755 ! -name 'contrib*' # unsigned/partner-repacks have 664 permission
find . -maxdepth 1 -type d ! -perm 2775  -name 'contrib*' # nothing showed up, as expected.

Virus Scan

# ffxbld@stage
$ cd /pub/mozilla.org/firefox/nightly/3.6.3-candidates/build1
$ clamdscan -m /mnt/netapp/stage/archive.mozilla.org/pub/firefox/nightly/3.6.3-candidates/build1
/mnt/netapp/stage/archive.mozilla.org/pub/firefox/nightly/3.6.3-candidates/build1: OK
----------- SCAN SUMMARY -----------
Infected files: 0
Time: 3257.886 sec (54 m 17 s)

Push to mirrors

# ffxbld@stage
rsync -av --exclude=*.log --exclude=*.txt --exclude=*unsigned* --exclude=*.crashreporter-symbols.zip --exclude=*.tests.tar.bz2* --exclude=*partner-repacks* /pub/mozilla.org/firefox/nightly/3.6.3-candidates/build1/ /pub/mozilla.org/firefox/releases/3.6.3/

Updated cltbld@stage:/pub/mozilla.org/zz/rsyncd-mozilla-current.exclude to include 3.6.3 instead of 3.6.2

Final Verification

green. there were a few 302 temporary moved responses, but all looked ok.

Push updates to beta & release channel

# cltbld@aus2-staging
cd /opt/aus2/snippets/staging/
~/bin/backupsnip 20100401-Firefox-3.6.3
...
real    3m45.063s
user    0m0.389s
sys     0m3.667s
...
~/bin/pushsnip 20100401-Firefox-3.6.3

Release

#ffxbld @ stage
 cd /pub/mozilla.org/firefox/releases
rm latest-3.6; ln -s 3.6.3 latest-3.6

Manually redo missing source tarball and bundle

  • Normal tarball as part of automation got halted because of config error, see above.
  • lsblakk triggered the source step in automation while she was prep-ing for a downtime early this morning.
  • joduinn signed tarball and bundle as follows:
# new keymaster
mkdir -p ~/signing-work/firefox-3.6.3/unsigned-build1/source
mkdir -p ~/signing-work/firefox-3.6.3/signed-build1/source
cd ~/signing-work/firefox-3.6.3/unsigned-build1/source
scp -i ~/.ssh/ffxbld_dsa ffxbld@stage.mozilla.org:/home/ftp/pub/firefox/nightly/3.6.3-candidates/build1/source .
cd ~/signing-work/firefox-3.6.3/signed-build1/source
scp -i ~/.ssh/ffxbld_dsa ffxbld@stage.mozilla.org:/home/ftp/pub/firefox/nightly/3.6.3-candidates/build1/source .
# modified MD5SUMS and SHA1SUMS
gpg -v -u releases -sba firefox-3.6.3.bundle
gpg -v -u releases -sba firefox-3.6.3.source.tar.bz2
gpg --verify firefox-3.6.3.bundle.asc
gpg --verify firefox-3.6.3.source.tar.bz2.asc
scp -i ~/.ssh/ffxbld_dsa * ffxbld@stage.mozilla.org:/home/ftp/pub/firefox/nightly/3.6.3-candidates/build1/source


3.5.9->3.6.3 MU

  • Lots of fiddling around with master (thanks bhearsum). kicked off snippets by clicking on "major snippet" on waterfall. Posted to betatest channel
  • abillings spotted that detailsURL was wrong. Seems this happened in 3.6.2 also.
  • bhearsum showed me how he fixed it for 3.6.2:
# cltbld @ aus2-staging
cd /opt/aus2/snippets/staging
rsync -av 20100402-Firefox-3.5.9-3.6.3-MU-test/ 20100402-Firefox-3.5.9-3.6.3-MU-test-url-fixed/
cd 20100402-Firefox-3.5.9-3.6.3-MU-test-url-fixed/
find . -type f -exec sed -i -e s,/3.6.3/,/3.6/, {} \;
cd ..
# verification - confirms that only the detailsUrl has changed
diff -Naur 20100402-Firefox-3.5.9-3.6.3-MU-test/ 20100402-Firefox-3.5.9-3.6.3-MU-test-url-fixed/ | grep -v '[+-][+-]' | grep '^[+-]' | grep -v detailsUrl
#
# now do the beta and release snippets, too
rsync -av 20100402-Firefox-3.5.9-3.6.3-MU-beta/ 20100402-Firefox-3.5.9-3.6.3-MU-beta-url-fixed/
cd 20100402-Firefox-3.5.9-3.6.3-MU-beta-url-fixed/
find . -type f -exec sed -i -e s,/3.6.3/,/3.6/, {} \;
#
# verification - confirms that only the detailsUrl has changed
cd ..
diff -Naur 20100402-Firefox-3.5.9-3.6.3-MU-beta/ 20100402-Firefox-3.5.9-3.6.3-MU-beta-url-fixed/ | grep -v '[+-][+-]' | grep '^[+-]' | grep -v detailsUrl
rsync -av 20100402-Firefox-3.5.9-3.6.3-MU/ 20100402-Firefox-3.5.9-3.6.3-MU-url-fixed/
cd 20100402-Firefox-3.5.9-3.6.3-MU-url-fixed/
find . -type f -exec sed -i -e s,/3.6.3/,/3.6/, {} \;
cd ..
# verification - confirms that only the detailsUrl has changed
diff -Naur 20100402-Firefox-3.5.9-3.6.3-MU/ 20100402-Firefox-3.5.9-3.6.3-MU-url-fixed/ | grep -v '[+-][+-]' | grep '^[+-]' | grep -v detailsUrl
#
# push the new test snippets
~/bin/backupsnip 20100402-Firefox-3.5.9-3.6.3-MU-test-url-fixed
~/bin/pushsnip 20100402-Firefox-3.5.9-3.6.3-MU-test-url-fixed
#
# move old, unwanted directories
mv 20100402-Firefox-3.5.9-3.6.3-MU-beta WRONG-URL-20100402-Firefox-3.5.9-3.6.3-MU-beta
mv 20100402-Firefox-3.5.9-3.6.3-MU WRONG-URL-20100402-Firefox-3.5.9-3.6.3-MU

EU Ballot Build

  • Fixed typo in bouncer: Firefox-3.6.3-EUballot win /firefox/releases/3.6.3/win32-EUballot/en-US/Firefox Setup 3.6.3.exe

XULRunner build

  • This step doesn't have to be the last one
  • Triggered the builds by going to the buildbot waterfall for mozilla-1.9.2 and forcing builds for linux (not x64), mac and windows xulrunner. Set the field "Revision to build" (not properties) to 'FIREFOX_3_6_3_RELEASE'. Email the builds to release-drivers.
  • When waterfall showed builds done, looked at logs to find location of files on stage, and gather them into one place.
# as xrbld@stage 
cd /pub/mozilla.org/xulrunner/nightly
mkdir -p 1.9.2.3-candidates/build1
cd -p 1.9.2.3-candidates/build1
scp /pub/mozilla.org/xulrunner/tinderbox-builds/mozilla-1.9.2-win32/1270238019/xulrunner-1.9.2.3.en-US.win32* .
scp /pub/mozilla.org/xulrunner/tinderbox-builds/mozilla-1.9.2-linux/1270237392/* .
scp /pub/mozilla.org/xulrunner/tinderbox-builds/mozilla-1.9.2-macosx/1270237393/* .


  • Waiting on reply from mfinkle and/or dtownsend to publish these builds

Push beta channel snippets

These were not pushed at the same time as the release snippets. 

# cltbld @ aus2-staging
cd /opt/aus2/snippets/staging
screen
# push the new test snippets
~/bin/backupsnip 20100402-Firefox-3.5.9-3.6.3-MU-beta-url-fixed
...
real    0m26.269s
user    0m0.038s
sys     0m0.537s
~/bin/pushsnip 20100402-Firefox-3.5.9-3.6.3-MU-beta-url-fixed
...
sent 217198 bytes  received 9788 bytes  7442.16 bytes/sec
total size is 180234  speedup is 0.79
Retrieved from "https://wiki.mozilla.org/index.php?title=Releases/Firefox_3.6.3/BuildNotes&oldid=214934"