Releases/Firefox 4.0b4/BuildNotes

From MozillaWiki
Jump to: navigation, search

Build Engineers

lsblakk, aki - tracking bug

Signed-off Revision(s)

L10N changesets

  • using l10n dashboard I grabbed the l10n-changesets for the config update patch, then hit "Ship it!" Note: if there are pending sign-offs, ping Axel to make sure it's ok to grab the changesets

Tags

On mozilla-central:

Build # Tag Changeset
1 GECKO20b4_20100817_RELBRANCH b51b190b9fcc
FIREFOX_4_0b4_BUILD1, FIREFOX_4_0b4_RELEASE cb39cf6342e5
2 FIREFOX_4_0b4_BUILD2, FIREFOX_4_0b4_RELEASE 9aa39b619a19
3 FIREFOX_4_0b4_BUILD3, FIREFOX_4_0b4_RELEASE 9d6448b6a677

Build data

Build # Type Build ID Build machine Time to build
2 [Linux compressed] 20100805181633 mv-moz2-linux-ix-slave10 52 mins
[Linux 64 compressed] 20100817135329 moz2-linux64-slave06 2 hrs, 7 mins
[Mac OS X] 20100817134330 bm-xserve11 3 hrs, 28 mins
[Mac OS X 64] 20100817134655 moz2-darwin10-slave05 1 hrs, 45 mins
[Win32] 20100817145336 mw32-ix-slave03 2 hrs, 44 mins
3 [Linux compressed] 20100818120848 mv-moz2-linux-ix-slave10 59 mins
[Linux 64 compressed] 20100818121643 moz2-linux64-slave06 1 hrs, 51 mins
[Mac OS X] 20100818121614 bm-xserve11 3 hrs, 31 mins
[Mac OS X 64] 20100818121922 moz2-darwin10-slave05 1 hrs, 48 mins
[Win32] 20100818132640 mw32-ix-slave03 2 hrs, 48 mins

Bugs that we have hit on this release

  • bug 588546 - Unittests for all but win32 are not being triggered, packageTests step not run either

NOTE from armenzg: There might be more on the build notes but I have added the one that I have fixed for the next beta.

Build Notes

Build 1

  • Moved the following slaves to pm:
    • from pm03:
      • mv-moz2-linux-ix-slave10
      • mw32-ix-slave18
      • mw32-ix-slave15
    • from pm01:
      • mv-moz2-linux-ix-slave02
      • mv-moz2-linux-ix-slave03
      • mv-moz2-linux-ix-slave04
      • mw32-ix-slave02
      • mw32-ix-slave03
      • bm-xserve06
      • bm-xserve11
      • bm-xserve12
      • bm-xserve16
      • moz2-linux64-slave04
      • moz2-linux64-slave06
      • moz2-darwin10-slave05
      • moz2-darwin10-slave06
  • Landed config patch
  • Reconfigured pm (delete *.pyc files)
    • needed to delete some files on pm since it was out of space (deleted /tmp/tmp* and got 2.3 GB)
  • sendchange to get things running:
buildbot sendchange --username=lsblakk --master=localhost:9010 --branch=mozilla-central -m "Firefox 4.0b4 build1" Go

Tag

mv-moz2-linux-ix-slave04

  • I had not reconfigured pm (all that fun with checkconfig messing up distracted me!) and so the first run of tag did 4.0b3 tagging instead
  • Beltzner had already given the request to cancel builds and wait for new additions to the 4.0b4 build2 so I re-did tagging (on the same slave) with force build after reconfiging and clobbering.
  • Re-did build1 tagging with the correct config to cut the relbranch.

After tagging completed we stopped for:

Build 2

  • Removed *pyc on pm moz2-master
  • Clobbered the release builders (NOTE: do not be connected to Build-VPN)
  • Used same l10n-changesets as build1 since Axel did not respond to ping
  • Landed configs
  • Reconfig & close m-c tree
  • Sendchange:
buildbot sendchange --username=lsblakk --master=localhost:9010 --branch=mozilla-central -m "Firefox 4.0b4 build2" go

Tag

mv-moz2-linux-ix-slave04

  • tagged buildbot-configs, buildbotcustom, tools with FIREFOX_4_0b4_BUILD2, FIREFOX_4_0b4_RELEASE (and also did BUILD1 since that hadn't happened yet)

no problems

Source

mv-moz2-linux-ix-slave04

no problems

Build/Repack

No Problems on all platforms (linux,linux64,macosx,macosx64,win32)

Unittests

  • Sendchanges were not correctly formed in the automation (bug 585098 was checked in but pm buildbotcustom had not been updated) so I manually did sendchanges to get linux/linux64/macosx/macosx64 results:

Unittest:

buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange-unittest --branch release-mozilla-central-linux-opt-unittest --revision 9aa39b619a19 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build2/linux-i686/en-US/firefox-4.0b4.tar.bz2 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build2/linux-i686/en-US/firefox-4.0b4.tests.zip
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange-unittest --branch release-mozilla-central-linux64-opt-unittest --revision 9aa39b619a19 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build2/linux-x86_64/en-US/firefox-4.0b4.tar.bz2 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build2/linux-x86_64/en-US/firefox-4.0b4.tests.zip
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange-unittest --branch release-mozilla-central-macosx-opt-unittest --revision 9aa39b619a19 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build2/mac/en-US/Firefox 4.0 Beta 4.dmg' 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build2/mac/en-US/Firefox 4.0 Beta 4.tests.zip'
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange-unittest --branch release-mozilla-central-macosx64-opt-unittest --revision 9aa39b619a19 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build2/mac64/en-US/Firefox 4.0 Beta 4.dmg' 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build2/mac64/en-US/Firefox 4.0 Beta 4.tests.zip'
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange-unittest --branch release-mozilla-central-win32-opt-unittest --revision 9aa39b619a19 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build2/unsigned/win32/en-US/firefox-4.0b4.zip http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build2/unsigned/win32/en-US/firefox-4.0b4.tests.zip

Talos:

buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange --branch release-mozilla-central-linux-talos --revision 9aa39b619a19 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build3/linux-i686/en-US/firefox-4.0b4.tar.bz2
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange --branch release-mozilla-central-linux64-talos --revision 9aa39b619a19 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build3/linux-x86_64/en-US/firefox-4.0b4.tar.bz2
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange --branch release-mozilla-central-win32-talos --revision 9aa39b619a19 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build3/unsigned/win32/en-US/firefox-4.0b4.zip
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange --branch release-mozilla-central-macosx-talos --revision 9aa39b619a19 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build3/mac/en-US/Firefox 4.0 Beta 4.dmg'
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange --branch release-mozilla-central-macosx64-talos --revision 9aa39b619a19 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b4-candidates/build3/mac64/en-US/Firefox 4.0 Beta 4.dmg'
Results
  • Linux/Linux64/Macosx64 builds did not make tests, upload tests, thus failed tests
  • Win32 had a green run on the tests run on pm, but had intermittent orange on the ones run on minis (two runs, since I triggered a second set manually)
  • Macosx ran orange on all but jsreftest and reftest

Signing

en-US.xpi removed from stage bug 485860:

#ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/4.0b4-candidates/build3
rm en-US.xpi

Using instructions.

L10N Verify

  • metadiff for win32_l10n had warnings - on many locales
source/bin/chrome/${locale}.jar and target/bin/chrome/${locale}.jar differ

Updates

mv-moz2-linux-ix-slave03

  • first run of updates blew up on downloading 75/363, I moved updates/builds dir aside and tried again and got past 75 on the downloading.

Build 3

  • missed the maxVer bump required for the Feedback Add-On
  • clobbered release builders
  • no l10n changes - checked with Axel in #release-drivers
  • reconfig pm (Note: updated buildbotcustom before this build, forgot to remove *.pyc will have to wait and see if that causes any problems)
  • notified #developers (changed topic even!) and closed tree for tagging
    • Why do we close the tree for tagging? Our automation doesn't cope well with losing a push race. bug 508896
  • sendchange:
buildbot sendchange --username=lsblakk --master=localhost:9010 --branch=mozilla-central -m "Firefox 4.0b4 build3" doit


Tag

mv-moz2-linux-ix-slave04

  • tagged buildbot-configs, buildbotcustom, tools with FIREFOX_4_0b4_BUILD3, FIREFOX_4_0b4_RELEASE

no problems

Source

No problems.

Build/Repack

No problems

Unittests

  • Filed bug 588546 because of a regression from bug 585098 where now only the unittestPlatforms specified in the release_config get the make packageTest step run. This results in only sendchanges/tests for win32 -- note in build2 also got test package for macosx, but see if that still happens now that buildbotcustom was updated for this run
Results
  • win32 tests run on pm had a timeout on xpcshell consistent with bug 580525

Signing

en-US.xpi removed from stage bug 485860:

#ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/4.0b4-candidates/build3
rm en-US.xpi

Using instructions.

L10N Verify

  • no problems Only in... for all platforms was mostly the same except some ja/ja-JP differences (we will be symlinking the ftp later for that locale)
  • win32 currently gives warnings on the jar diff step -- this appears to be new since bug 552668 and was not present in previous beta releases
source/bin/chrome/${locale}.jar and target/bin/chrome/${locale}.jar differ

Updates

  • no problems

Update Verify

  • After some investigation bug 589196 was filed (Windows setup layout was changed in beta 4). After landing the patch the builder became green.
  • The output has the following expected diffs:
Only in source/bin/chrome: be.manifest
Only in source/bin/defaults/profile: prefs.js

Move slaves back to active masters

Returned all but 1 slave (kept mv-moz2-linux-ix-slave02) to pm01/03:

  • to pm03:
    • mv-moz2-linux-ix-slave10
    • mw32-ix-slave18
    • mw32-ix-slave15
  • to pm01:
    • mv-moz2-linux-ix-slave03
    • mv-moz2-linux-ix-slave04
    • mw32-ix-slave02
    • mw32-ix-slave03
    • bm-xserve06
    • bm-xserve11
    • bm-xserve12
    • bm-xserve16
    • moz2-linux64-slave04
    • moz2-linux64-slave06
    • moz2-darwin10-slave05
    • moz2-darwin10-slave06

Final checks before push

# ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/4.0b4-candidates/build3/
find . ! -user ffxbld # contib/...
find . ! -group firefox
find . -type f ! -perm 644
find . -mindepth 1 -type d ! -perm 755 ! -name 'contrib*' # contib/...
find . -mindepth 1 -type d ! -perm 755 ! -name 'contrib*' # contib/...
^^ Last line is WRONG, should be |find . -maxdepth 1 -type d ! -perm 2775 -name 'contrib*'| 

Virus check

clamdscan -m /mnt/netapp/stage/archive.mozilla.org/pub/firefox/nightly/4.0b4-candidates/build3/
/mnt/netapp/stage/archive.mozilla.org/pub/firefox/nightly/4.0b4-candidates/build3: OK

----------- SCAN SUMMARY -----------
Infected files: 0
Time: 1056.363 sec (17 m 36 s)

Update Bouncer

Forced bouncer_submitter builder. Checked using web interface.

Push to mirrors

# ffbld@stage
rsync -av --exclude=*tests* --exclude=*crashreporter* --exclude=*.log \
  --exclude=*.txt --exclude=*unsigned* --exclude=*update-backup* \
  /pub/mozilla.org/firefox/nightly/4.0b4-candidates/build3/ \
  /pub/mozilla.org/firefox/releases/4.0b4

Add index.html to cover the files before release time:

# ffbld@stage
cd /pub/mozilla.org/firefox/releases/4.0b4
curl -sL https://bugzilla.mozilla.org/attachment.cgi?id=456169 | sed 's/Beta 1/Beta 4/g' > index.html
for dir in `find . -mindepth 1 -type d`; do cp -pv index.html $dir/; done

Final verify

  • went green on first run - sent remaining slave (mv-moz2-linux-ix-slave02) back to pm01
  • sent email when mirrors were absorbed for testing at 8:07am PDT
  • sent email when mirrors were absorbed for release at 9:17am PDT (more than 30K uptake)

Push snippets to beta channel

  • Upon GO from release-drivers
# cltbld@aus2-staging
cd /opt/aus2/snippets/staging
~/bin/backupsnip 20100817-Firefox-4.0b4
~/bin/pushsnip   20100817-Firefox-4.0b4
  • After investigating why the beta updates did not apply cleanly (and throttling updates temporarily it turned out that I had pushed build2 snippets live


# cltbld@aus2-staging
cd /opt/aus2/snippets/staging
~/bin/pushsnip   20100818-Firefox-4.0b4
  • Pinged QA to re-test

Remove index.html files

Once we are fully announced (by the driver) we can remove the index.html files:

# ffxbld@stage
cd /pub/mozilla.org/firefox/releases/4.0b4/
find . -name index.html
find . -name index.html | xargs rm -v

Release

  • Emailed the following to metrics@m.c
Firefox,firefox,firefox,4.0b4,4.0
Firefox,firefox,firefox,4.0b5pre,4.0