Releases/Firefox 2.0.0.15:BuildNotes

From MozillaWiki
Jump to: navigation, search

Build Engineers

bhearsum + alice
Version/config bump bug

Bonsai queries

Build 1: http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=MOZILLA_1_8_BRANCH&branchtype=match&dir=&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=explicit&mindate=2008-06-10+12%3A49+PDT&maxdate=2008-06-10+12%3A50+PDT&cvsroot=%2Fcvsroot

Build 2: http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=GECKO181_20080612_RELBRANCH&branchtype=match&dir=&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=explicit&mindate=2008-06-12+12%3A35+PDT&maxdate=2008-06-12+12%3A38+PDT&cvsroot=%2Fcvsroot

Build 3: http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=GECKO181_20080612_RELBRANCH&branchtype=match&dir=&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=explicit&mindate=2008-06-12+12%3A36+PDT&maxdate=2008-06-23+12%3A38+PDT&cvsroot=%2Fcvsroot

Tags

Updated CVS Tags devmo page.

Build 1:

Module Branch Tag Pull date
cvsroot/mozilla MOZILLA_1_8_BRANCH FIREFOX_2_0_0_15_BUILD1 2008-06-10 12:51 PDT
l10n/l10n MOZILLA_1_8_BRANCH FIREFOX_2_0_0_15_BUILD1 2008-06-10 12:51 PDT

Build 2:

Module Branch Tag Pull date
cvsroot/mozilla GECKO181_20080612_RELBRANCH FIREFOX_2_0_0_15_BUILD2 N/A
l10n/l10n GECKO181_20080612_RELBRANCH FIREFOX_2_0_0_15_BUILD2 N/A

Build 3:

Module Branch Tag Pull date
cvsroot/mozilla GECKO181_20080612_RELBRANCH FIREFOX_2_0_0_15_BUILD3 FIREFOX_2_0_0_15_RELEASE N/A
l10n/l10n GECKO181_20080612_RELBRANCH FIREFOX_2_0_0_15_BUILD3 FIREFOX_2_0_0_15_RELEASE N/A

Build data

Type Build ID SHA1 Push date Build machine
[Windows installer/zip] 2008062306 production-pacifica-vm
[Mac compressed] 2008062305 bm-xserve05
[Linux compressed] 2008062305 production-prometheus-vm

Notes

Build 1

Bootstrap Tag: RELEASE_AUTOMATION_M9_2

Setup before starting:

  • Updated /builds/buildbot/Automation/buildbot-configs on production-1.8-master.
    • master.cfg is "Locally Modified" to contain passwords. Identical to repository version otherwise.
    • Turned off the nightly builders on production-1.8-master to make sure they don't interfere with the release. (We need to figure out how to avoid this in the future).
    • update mozilla1.8 tinderbox to say "no nightlies right now"
    • Ran 'buildbot reconfig'
  • update /builds/tinderbox/mozilla/tools/tinderbox to the Bootstrap tag (need to revert this before restarting nightlies)
  • ensure that machines have enough resources
    • production-prometheus-vm - 40G on /
    • production-1.8-master - 11G on /builds, 4G on /data
    • production-pacifica-vm - 36G on c:
    • bm-xserve05 - 34G on /

Tag

Respin bug found while tagging. We let it finish, and then restarted the automation for build 2.

Build 2

  • Clobbered /builds/tags/* on production-prometheus-vm
  • Updated bootstrap.cfg for build 2, re-tagged.

Tag

No problems.

Source

No problems.

Build

  • Win32 failed out because tinderbox was running in the background. When the nightly builds were stopped, the child processes didn't get killed.
  • Kill other tinderbox processes, restarted.

Repack

No problems.

Sign

No problems.

L10nVerify

Two things of note:

  • bug 351476 changed the line endings on a bunch of files. This caused some spurious output.
  • production-prometheus-vm had its kernel bumped. A preprocessed JS file in each package contains the kernel version, which obviously changed between 2.0.0.14 and 2.0.0.15, and caused more spurious output.

No real problems.

Generate Updates

No problems.

Publish Updates to Test Channels (betatest & releasetest)

  • No problems

Update Verify

No problems.

Stage

  • Failed out because cltbld wasn't part of the 'firefox' group'.
    • Created firefox group, added cltbld, restart buildbot slave

No other problems.

Push Updates to Beta Channels

# put snippets on beta
$ sudo su - cltbld
# make sure using latest version of scripts in mozilla/tools/release/bin/
$ cd bin
$ cvs update .
$ cd /opt/aus2/snippets/staging
# note the required parameter must match what will be used with pushsnip below. 
$ time ~/bin/backupsnip 20080613-Firefox-2.0.0.15-beta
real    40m37.598s
user    0m20.948s
sys     0m27.232s

After QA gives "ok" to push to beta channel, on aus2-staging do: 

 # put snippets on beta
 $ sudo su - cltbld
 # make sure using latest version of scripts in mozilla/tools/release/bin/
 $ cd bin
 $ cvs update mozilla/tools/release/bin/ 
 $ cd /opt/aus2/snippets/staging
 $ time ~/bin/pushsnip  20080613-Firefox-2.0.0.15-beta
real    2m23.129s
user    0m0.151s
sys     0m6.964s

Sign Installers

  • Had to manually delete old asc files from build2 installer signing:
# On stage
cd /data/cltbld/firefox-2.0.0.15/batch1/stage-signed
find . -iname "*.asc" -exec rm {} \;
  • Done manually using these installer-signing-instructions here
  • push signed bits back to same location on stage
  • complete stage-merged:
# on stage
cd /data/cltbld/firefox-2.0.0.15/
rsync -av batch1/mar/ stage-merged/
rsync -av batch1/stage-signed/ stage-merged/
  • Create MD5 and SHA1 checksum files
# on stage
cd /data/cltbld/firefox-2.0.0.15/stage-merged/
~/bin/checksum-files .
  • Fix permissions & ownership (on the two SUM files, and the detached sigs)
chown -R cltbld:firefox .
chmod 644 *SUMS

Stop for respin

Build 3

  • Clobbered /builds/tags/* on production-prometheus-vm
  • Updated bootstrap.cfg for build 3

Tag

No problems

Source

No problems

Build

No problems on linux & win32. Had to manually run 'upload-symbols.sh' in /builds/tinderbox/Fx-Mozilla1.8-release/Darwin_8.7.0_Depend/2008062305 on Mac because cygwin crashed when the automation tried to upload the symbols.

Repack

No problems

Sign

L10nVerify

Two things of note:

  • bug 351476 changed the line endings on a bunch of files. This caused some spurious output.
  • production-prometheus-vm had its kernel bumped. A preprocessed JS file in each package contains the kernel version, which obviously changed between 2.0.0.14 and 2.0.0.15, and caused more spurious output.

Generate Updates

  • backupsnip took 34 minutes
  • No problems
Updates for 2.0.0.15build2 to 2.0.0.15build3
export CVSROOT=":ext:cltbld@cvs.mozilla.org:/cvsroot"
cvs co -d patcher mozilla/tools/patcher
cd patcher
cvs co -d MozBuild mozilla/tools/release/MozBuild
cd ../
wget https://bugzilla.mozilla.org/attachment.cgi?id=326467
mv attachment.cgi?id=326467 moz18-branch-2.0.0.15build2-to-build3.cfg
cd patcher
./patcher2.pl --app=firefox --config=../moz18-branch-2.0.0.15build2-to-build3.cfg --build-tools --tools-revision=UPDATE_PACKAGING_R4
./patcher2.pl --app=firefox --config=../moz18-branch-2.0.0.15build2-to-build3.cfg --download | tee ../firefox2.0.0.15build2-to-build3-download.log
./patcher2.pl --app=firefox --config=../moz18-branch-2.0.0.15build2-to-build3.cfg --create-patches --partial-patchlist-file=patchlist.cfg | tee ../firefox2.0.0.15build2-to-build3-create-patches.log
rsync -av -e 'ssh -i ~/.ssh/aus' temp/firefox/2.0.0.15build2-2.0.0.15build3/aus2/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20080624-Firefox-2.0.0.15build2-to-build3
rsync -av -e 'ssh -i ~/.ssh/aus' temp/firefox/2.0.0.15build2-2.0.0.15build3/aus2.beta/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20080624-Firefox-2.0.0.15build2-to-build3-beta
rsync -av -e 'ssh -i ~/.ssh/aus' temp/firefox/2.0.0.15build2-2.0.0.15build3/aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20080624-Firefox-2.0.0.15build2-to-build3-test

Partials created here were invalid because DisableCompleteJump was set. Tried a few things to fix this, but couldn't. nthomas suspects it is because the version is the same. To work around, all partial.txt files were deleted so force aus to fallback on complete.txt. On aus2-staging: 

cd /opt/aus2/snippets/staging
cp -R 20080624-Firefox-2.0.0.15build2-to-build3 20080624-Firefox-2.0.0.15build2-to-build3-with-partials
find 20080624-Firefox-2.0.0.15build2-to-build3/ -iname partial.txt -exec rm {} \;
cp -R 20080624-Firefox-2.0.0.15build2-to-build3-beta 20080624-Firefox-2.0.0.15build2-to-build3-beta-with-partials
find 20080624-Firefox-2.0.0.15build2-to-build3-beta/ -iname partial.txt -exec rm {} \;
cp -R 20080624-Firefox-2.0.0.15build2-to-build3-test 20080624-Firefox-2.0.0.15build2-to-build3-test-with-partials
find 20080624-Firefox-2.0.0.15build2-to-build3-test/ -iname partial.txt -exec rm {} \;

Ran backupsnip and pushsnip for test channels: 

~/bin/backupsnip 20080624-Firefox-2.0.0.15build2-to-build3-test
~/bin/pushsnip 20080624-Firefox-2.0.0.15build2-to-build3-test

Backupsnip took approx. 40 minutes.

Ran backupsnip on beta && release snippets: 

-bash-3.2$ time ~/bin/backupsnip 20080624-Firefox-2.0.0.15build2-to-build3-beta
Running /bin/tar cfvj /opt/aus2/snippets/backup/20080624-2-pre-20080624-Firefox-2.0.0.15build2-to-build3-beta.tar.bz2 .
real	33m58.907s
user	0m21.440s
sys	0m28.649s

-bash-3.2$ time ~/bin/backupsnip 20080624-Firefox-2.0.0.15build2-to-build3
Running /bin/tar cfvj /opt/aus2/snippets/backup/20080624-3-pre-20080624-Firefox-2.0.0.15build2-to-build3.tar.bz2 .
real	32m36.029s
user	0m21.309s
sys	0m28.029s

Publish Updates to Test Channels (betatest & releasetest)

  • No problems

Update Verify

  • No problems

Stage

  • No problems

Push Updates to Beta Channels

Two pushes here, one for 2.0.0.14 -> 2.0.0.15build3, one for 2.0.0.15build2 -> 2.0.0.15build3. 

~/bin/pushsnip 20080623-Firefox-2.0.0.15-beta
~/bin/pushsnip 20080624-Firefox-2.0.0.15build2-to-build3-beta

Sign Installers

  • Done manually using these installer-signing-instructions here
  • push signed bits back to same location on stage
  • complete stage-merged:
# on stage
cd /data/cltbld/firefox-2.0.0.15/
rsync -av batch1/mar/ stage-merged/
rsync -av batch1/stage-signed/ stage-merged/
  • Create MD5 and SHA1 checksum files
# on stage
cd /data/cltbld/firefox-2.0.0.15/stage-merged/
~/bin/checksum-files .
  • Fix permissions & ownership (on the two SUM files, and the detached sigs)
chown -R cltbld:firefox .
chmod 644 *SUMS
# extra verification to make sure no build2 files are still here
find . -mtime +2
./contrib
./contrib-localized
# it's okay for those to have old modification dates

Update Bouncer

  • Manually added entries to download.mozilla.org

Add release to rsync module

On stage.mozilla.org, add the new release to the smaller, faster, rsync module: 

# on stage (as cltbld)
cd /pub/mozilla.org/zz/
vi rsyncd-mozilla-current.exclude

and replace the previous latest release (FF2.0.0.14) with: 

+ firefox/releases/2.0.0.15
+ firefox/releases/2.0.0.15/**

Push to mirrors

  • get the last formal "go" from QA/Dev/website/IT/release-drivers
  • push the stage-merged directory to the releases area:
# on stage (as cltbld)
rsync -av /data/cltbld/firefox-2.0.0.15/stage-merged/ /home/ftp/pub/firefox/releases/2.0.0.15/

Final Verification

  • Verify that releasetest points to valid bouncer links:
# this can be run from anywhere
cvs co mozilla/testing/release
cd mozilla/testing/release/updates
cat moz18-firefox-*.cfg | grep -v major | sed 's/betatest/releasetest/' | grep -v 2.0a | grep -v 2.0b > update.cfg
./verify.sh -t update.cfg 2>&1 | tee quickVerify.log
  • Look for any HTTP error codes besides 200 ("OK") and 302 ("Found"):
grep HTTP quickVerify.log | grep -v 200 | grep -v 302
  • Before pushing final updates,verify that "release" and "releasetest" channel match:
# on aus2-staging
cd /opt/aus2/snippets/staging/20080623-Firefox-2.0.0.15
find -type d -iregex '.*release.*' | perl -nle '$a = $_; $a =~ s/release/releasetest/; system("diff -r -u $_ ../20080623-Firefox-2.0.0.15-test/$a");'

Publish Updates to Release Channel

  • In case you need to cancel/abort updates, the quickest way to do this is here
  • While waiting for QA to finish testing, do a backup:
 # on aus2-staging
 # put snippets on release/live channel
 $ cd /opt/aus2/snippets/staging
 $ sudo su - cltbld
 $ time ~/bin/backupsnip 20080623-Firefox-2.0.0.15
real    33m19.706s
user    0m22.414s
sys     0m33.259s
  • After QA finished testing the releasetest channel, enabled release channel:
 # on aus2-staging
 # put snippets on release/live channel
 $ cd /opt/aus2/snippets/staging
 $ sudo su - cltbld
 $ time ~/bin/pushsnip 20080623-Firefox-2.0.0.15
  • VERIFYFIX: Had to chmod +x ~/bin/pushsnip

Release

Free Software Builds

  • Bump the BuildTag definition in the three tinder-config.pl, like this
  • Done on production machines:
    • production-pacifica-vm (win32)
    • bm-xserve05 (macosx)
    • production-prometheus-vm (linux)
  • Start builds:
cd /builds/tinderbox/Fx-Mozilla1.8-FS
./build-seamonkey.pl --once --depend --config-cvsup-dir `pwd`/tinderbox-configs/ 2>&1 | tee FIREFOX_2_0_0_15build3_FS.log
  • Stage
# as cltbld@stage
mkdir -p ~/firefox-2.0.0.15-fs/batch1/prestage
cd ~/firefox-2.0.0.15-fs/batch1
# rsync from push dirs
rsync -av /home/ftp/pub/firefox/nightly/2008-06-26-05-mozilla1.8-fs/ ./prestage/
rsync -av /home/ftp/pub/firefox/nightly/2008-06-26-06-mozilla1.8-fs/ ./prestage/
rsync -av prestage/ prestage-trimmed/
rm -v prestage-trimmed/*.zip
rsync -av prestage-trimmed/ stage/
cd stage
~/bin/groom-files --long="2.0.0.15.fs" .
chown -R cltbld:firefox * .
find . -type f -exec chmod -v 644 {} \;
find . -type d -exec chmod -v 755 {} \;
cd ../
rsync -av stage/ /home/ftp/pub/firefox/releases/2.0.0.15/contrib/free-software/
Retrieved from "https://wiki.mozilla.org/index.php?title=Releases/Firefox_2.0.0.15:BuildNotes&oldid=98863"