Releases/Firefox 3.0.5/BuildNotes

Build Engineers

joduinn
Tracking release bug

Bonsai queries

Bonsai query is: last cvs checkin

last l10n checkin

Updated CVS Tags devmo page.

Tags

Build 1:

Module	Branch	Tag	Pull date
cvsroot/mozilla	HEAD	FIREFOX_3_0_5_BUILD1	2008-11-28 23:01 PST
l10n/l10n	HEAD	FIREFOX_3_0_5_BUILD1	2008-12-01 12:35 PST

Build data

Type	Build ID	SHA1	Push date	Build machine
[Windows installer/zip]				fx-win32-1.9-slave2
[Mac compressed]				fx-mac-1.9-slave2
[Linux compressed]				fx-linux-1.9-slave2

Notes

Build 1

Bootstrap Tag: RELEASE_AUTOMATION_M11

Setup before starting:

• Updated master.cfg, bootstrap.cfg. Retagged bootstrap.cfg.
• On all the slaves:
  • remove /builds/verify/firefox-3.0.0.14-3-major-update, /builds/verify/firefox-3.0.0.16-*-{fake,major}-update
  • verified we're using same version of tinderbox as before (RELEASE_AUTOMATION_M11), so nothing to do.
  • On the linux slave,
  • ran 'DISPLAY=:0 xhost +' to make sure linux AliveTest works
  • removed:
    • /builds/verify/firefox-3.0.{3,4}
    • /builds/updates/firefox-3.0.4
    • /builds/source/firefox-3.0.4
    • /data/cltbld/firefox-3.0.4
  • On the mac slave, removed nothing, as there was loads of space.
• Space on slaves before starting:
  • fx-linux-1.9-slave2: 19.5G on /builds
  • fx-mac-1.9-slave2: 14.5G on /
  • fx-win32-1.9-slave2: 16.9G on d:, 4.15G on e: (disk heavy work is done on d, eg update_verify. build/repack is done on e, but mostly just overwrites existing data)

• Gulped, and kicked off automation:

#login to production-1.9-master as buildbot
#
buildbot sendchange --username=joduinn --master=localhost:9989 -m"Firefox 3.0.5build1 release" go

Tag

• No problems

Source

• No problems

Build & Repack

• linux, mac, win32: No problems

Sign

• Signing doc
• no problems.
• slave had lost connection at 12:33PST and 12:47PST today. FIXME: dont know why.
• To resume builds, I tweaked master.cfg, and did a new sendchange to resume automation from this point.

$ buildbot sendchange --username=joduinn --master=localhost:9989 -m"Firefox 3.0.5build1 release after signing" go

L10nVerify

• Automated - no problems.

Generate Updates

• Automated - no problems.

Publish Updates to Test Channels (betatest & releasetest)

• Automated - no problems.

Update Verify

• linux - no problems.
• mac - no problems.
• win32 - bug 452128 was fixed in 3.0.4, so that 3.0.4 -> 3.0.5 updates expected to work... and they did! There was the usual complaint that freebl3.chk and softokn3.chk differ (bug 404340), which is a pass in reality. However, worth noting that there were the following extra messages:

...
Binary files source/bin/freebl3.chk and target/bin/freebl3.chk differ
Binary files source/bin/softokn3.chk and target/bin/softokn3.chk differ
FAIL: binary files found in diff
FAIL: check_updates returned failure for WINNT_x86-msvc downloads/Firefox Setup 3.0.4.exe vs. downloads/firefox-3.0.5.be.win32.installer.exe: 1
Using  https://aus2.mozilla.org/update/1/Firefox/3.0.4/2008102920/WINNT_x86-msvc/be/betatest/update.xml
...

Dont understand - should the above really be comparing FF3.0.4 vs FF3.0.5?

Stage

• Automated - no problems.

Push to beta channel

• Done in tandem with 2.0.0.19, notes are over there

Sign Installers

• I AM HERE - WAITING FOR GO FROM release-drivers

Done manually using these installer-signing-instructions here

• complete stage-merged:

# on stage
cd /data/cltbld/firefox-3.0.4/
rsync -av batch1/mar/ stage-merged/
rsync -av batch1/stage-signed/ stage-merged/

• Create MD5 and SHA1 checksum files

# on stage
cd /data/cltbld/firefox-3.0.4/stage-merged/
~/bin/checksum-files .

• Fix permissions & ownership (on the two SUM files, and the detached sigs)

chown -R cltbld:firefox .
chmod 644 *SUMS

• • FIXME: I noted that the permissions on the *SUMS files were correct before I ran the chmod above. Is this still needed?

Update Bouncer

• Done.
• Note for next release: Do not remove the Check Now bit on the Firefox-3.0.4 Products until well after the change to the rsync module (to prevent the likes of bug 464566)

Push to mirrors

• push the stage-merged directory to the releases area:

# on stage
rsync -av /data/cltbld/firefox-3.0.4/stage-merged/ /home/ftp/pub/firefox/releases/3.0.4/

• edit the exclude file /pub/mozilla.org/zz/rsyncd-mozilla-current.exclude to add the new release (3.0.4) and remove the previous release (3.0.3).

Final Verification

• Verify that releasetest points to valid bouncer links:

# this can be run from anywhere
cvs co mozilla/testing/release
cd mozilla/testing/release/updates
cat moz19-firefox-*.cfg | grep -v major | sed 's/betatest/releasetest/' | grep -v 2.0a | grep -v 2.0b > update.cfg
./verify.sh -t update.cfg 2>&1 | tee quickVerify.log

• Look for any HTTP error codes besides 200 ("OK") and 302 ("Found"):

grep HTTP quickVerify.log | grep -v 200 | grep -v 302

• First testrun hit:
  • ?? occurrences of 404s

• Before pushing final updates,verify that "release" and "releasetest" channel match:

# on aus2-staging
$ cd 20081103-Firefox-3.0.4
$ find -type d -iregex '.*release.*' | perl -nle '$a = $_; $a =~ s/release/releasetest/; system("diff -r -u $_ ../20081103-Firefox-3.0.4-test/$a");'
$

Publish Updates to Release Channel

While waiting for the formal "go," ran:

time ~/bin/backupsnip 20081103-Firefox-3.0.4
real	33m58.819s
user	0m26.811s
sys	0m37.705s

Release

• In case you need to cancel/abort updates, the quickest way to do this is here

$ time ~/bin/pushsnip 20081103-Firefox-3.0.4

https://bugzilla.mozilla.org/show_bug.cgi?id=466641 was filed on November 25 - latest-3.0 symlink was not updated. I went ahead and fixed this:
# cltbld@stage
cd /home/ftp/pub/firefox/releases
rm latest-3.0 && ln -s 3.0.4 latest-3.0