Releases/Firefox 4.0b3/BuildNotes

From MozillaWiki
Jump to: navigation, search

note re: bug 582700. If Windows 2000 is still busted in 4.0b3 we should set updateType=major in windows snippets. Per Rob Strong's comment it should NOT prompt despite being a MU.. Windows 2000 support was restored for 4.0b3, nothing to worry about now.

Build Engineers

armenzg, aki - tracking bug

Signed-off Revision(s)

Tags

On mozilla-central:

Build # Tag Changeset
1 GECKO20b3_20100804_RELBRANCH 2f187db8f5f6
FIREFOX_4_0b3_BUILD1, FIREFOX_4_0b3_RELEASE 1fdcff2dbfdf
2 FIREFOX_4_0b3_BUILD2, FIREFOX_4_0b3_RELEASE 69f2d0457750
3 FIREFOX_4_0b3_BUILD3, FIREFOX_4_0b3_RELEASE 900fdd7fb8b2

Build data

Build # Type Build ID Build machine
1 [Linux compressed] 20100804111201 mv-moz2-linux-ix-slave22
2 [Windows installer/zip] 20100804193205 mw32-ix-slave05
[Mac dmg] 20100804182810 bm-xserve15
[Mac64 dmg] 20100804183435 moz2-darwin10-slave07
[Linux compressed] 20100804182707 mv-moz2-linux-ix-slave22
[Linux64 compressed] 20100804183236 moz2-linux64-slave03
3 [Windows installer/zip] 20100805192522 mw32-ix-slave05
[Mac dmg] 20100805181943 bm-xserve15
[Mac64 dmg] 20100805182338 moz2-darwin10-slave07
[Linux compressed] 20100805181633 mv-moz2-linux-ix-slave22
[Linux64 compressed] 20100805182321 moz2-linux64-slave03

Bugs that we have hit on this release

  • bug 565640 discovering which URL to pull l10n-changesets is not discoverable
  • bug 584574 - revision not found in buildbot-configs on hg.m.o
  • bug 584818 - release clobber branch broken
    • select a master name and select "Any branch"
  • bug 584620 - Clobberer isn't working for 64bit release builders
  • No bug - Fix ParseLocalesFile to handle multiple lines with the same locale.
  • bug 584747 - Send sendchanges for releases to pm01:9009 after switchover to schedulerDB
  • bug 570816 - Fix branch name for unittest release sendchanges
  • bug 585082 - graphs does not recognise the "Firefox-Release" branch
  • bug 585098 - Port branch name refactoring for talos sendchanges to 0.7.10/default masters (releases and tigers)
  • bug 585108 - We don't need unit test builders for "mozilla-central" and "mozilla-2.0" releases
  • bug 586230 - No contrib directory for Firefox 4.0b3

Notes

Done on pm. Had to be restarted, because the master had 2 pending tag jobs.

Build 1

  • Marked all builders for all releases on pm for a clobber (done by nthomas)
  • Moved the following slaves to pm:
    • from pm03:
      • mv-moz2-linux-ix-slave22
      • mv-moz2-linux-ix-slave23
      • mw32-ix-slave05
      • mw32-ix-slave13
    • from pm01:
      • mv-moz2-linux-ix-slave20
      • mv-moz2-linux-ix-slave11
      • mw32-ix-slave02
      • mw32-ix-slave19
      • bm-xserve06
      • bm-xserve11
      • bm-xserve12
      • bm-xserve15
  • Confirmed beta 3 l10n changesets
  • Downloaded l10n-changesets from here
    • I only found the URL because Rail had mentioned previously. Axel filed bug 565640 to make it more discoverable
  • Landed changes
  • Reconfigured pm
  • sendchange to get things running:
buildbot sendchange --username=armenzg --master=localhost:9010 --branch=mozilla-central -m "Firefox 4.0b3 build1" Go

Tag

mv-moz2-linux-ix-slave11: No problems

  • Full stop at 12:11pm PDT
  • only Linux had finished and had to stop the repackages for it

Build 2

  • Removed *pyc and waiting
  • Clobbered the release builders (NOTE: do not be connected to Build-VPN)
  • Got the l10n-changesets from here
  • Landed configs and l10n-changesets
  • Stopped pm after checkconfig took too long
  • Restarted pm after a successful checkconfig
  • Tagging of our repos should have happened before the following sendchange
    • This is what has to be done: Tagged tip of default branch of buildbot-configs, buildbotcustom, tools with FIREFOX_4_0b3_BUILD2 and FIREFOX_4_0b3_RELEASE
  • Sendchange:
buildbot sendchange --username=aki --master=localhost:9010 --branch=mozilla-central -m "Firefox 4.0b3 build2" go

Tag

mv-moz2-linux-ix-slave11

  • Closed m-c.
  • Kept hitting "abort: destination 'be' already exists" in clone_be. The step didn't actually clobber the tag directory; I had to go back and clobber all release builders on pm in clobberer before it would work (nthomas says it was requesting branch=nothing)
  • This meant I did 3 sendchanges to get over the 2 failed tag steps. Luckily it went smoothly after that.
  • Re-opened m-c.
  • I tagged buildbot-configs, but ran into bug 584574 and decided to hold off on tagging buildbotcustom.
  • Once bug 584574 was fixed, tagged buildbotcustom.

Source

mv-moz2-linux-ix-slave11 - no problems

Build/Repack

  • Some L10N repacks failed due to bug 584620 (linux64 and macosx64).
  • Clobbered and forced l10n repacks for the failed locales/platforms (see instructions:
# cltbld@pm
PYTHONPATH=/tools/buildbotcustom:$PYTHONPATH python force_release_l10n.py -m http://localhost:8010 -t FIREFOX_4_0b3_RELEASE -v -s locales-file -n rail@mozilla.com

locales-file content:

it linux64
es-ES linux64
ru linux64
is linux64
de linux64
pt-PT linux64
nn-NO linux64
uk macosx64
pl macosx64
sv-SE macosx64
nn-NO macosx64
zh-TW macosx64
  • nn-NO on linux64 never got triggered because the "nn-NO macosx64" line overwrote it. bhearsum landed a fix so this can't happen again
  • I forced nn-NO for linux64

Unit tests

  • There has been a switch over to scheduler db last night and the sendchanges are probably being sent to the wrong place. I am going to trigger them.
  • bug 584747 send sendchanges to the schedulerDB
  • bug 570816 fix branch name for unit test releases sendchanges
  • bug 585098 fix branch name for talos release sendchanges

NOTE: I send the sendchanges to production-master01.build.mozilla.org:9009 and I changed the branch name for each sendchange to "release-$branch-$platform-opt-unittest' instead of '$branch-release-$platform-opt-unittest' NOTE2: Note the white spaces on Mac's filenames and that I had to add an apostrophe. You will see mac unit tests red because for missing them.

buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange-unittest --branch release-mozilla-central-linux-opt-unittest --revision 69f2d0457750 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build2/linux-i686/en-US/firefox-4.0b3.tar.bz2 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build2/linux-i686/en-US/firefox-4.0b3.tests.zip
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange-unittest --branch release-mozilla-central-linux64-opt-unittest --revision 69f2d0457750 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build2/linux-x86_64/en-US/firefox-4.0b3.tar.bz2 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build2/linux-x86_64/en-US/firefox-4.0b3.tests.zip
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange-unittest --branch release-mozilla-central-win32-opt-unittest --revision 69f2d0457750 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build2/unsigned/win32/en-US/firefox-4.0b3.zip http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build2/unsigned/win32/en-US/firefox-4.0b3.tests.zip
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange-unittest --branch release-mozilla-central-macosx-opt-unittest --revision 69f2d0457750 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build2/mac/en-US/Firefox 4.0 Beta 3.dmg' 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build2/mac/en-US/Firefox 4.0 Beta 3.tests.zip'
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange-unittest --branch release-mozilla-central-macosx64-opt-unittest --revision 69f2d0457750 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build2/mac64/en-US/Firefox 4.0 Beta 3.dmg' 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build2/mac64/en-US/Firefox 4.0 Beta 3.tests.zip'
  • No need to trigger the talos sendchanges because we got a full stop for build2

Sign

Using instructions.

NOTE: Watch for bug 586230 in the next release.

en-US.xpi removed from stage:

#ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/4.0b3-candidates/build2
rm en-US.xpi

Run usual signing script:

#!/bin/bash
set -x
set -e

PRODUCT=firefox
VERSION=4.0b3
BUILD=2
TAG=FIREFOX_4_0b3_RELEASE
REPO="mozilla-central"
EMAIL=armenzg@mozilla.com
KEYDIR=d:/2009-keys

cd ~/hg-tools
hg pull && hg up -r ${TAG}
mkdir -p ~/signing-work/${PRODUCT}-${VERSION}
cd ~/signing-work/${PRODUCT}-${VERSION}
cp ~/hg-tools/release/signing/* .

make PRODUCT=${PRODUCT} VERSION=${VERSION} \
  BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} \
  KEYDIR=${KEYDIR} USE_NEW=1

Inspected stdout, signing log. They looked good. Ran the following:

make postsign PRODUCT=${PRODUCT} VERSION=${VERSION} \
   BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL}

L10N verification

No problems

Updates

No problems

Update verify

No problems

Build 3

  • Full stop of build 2 because of bug 584057
  • Taken this fix
  • Clobbered all release builders
  • Took no l10n-changesets at Axel's request.
  • Landed config changes
  • reconfiged pm
  • sendchange:
buildbot sendchange --username=aki --master=localhost:9010 --branch=mozilla-central -m "Firefox 4.0b3 build3" doit
  • tagged buildbot-configs and buildbotcustom

Tag

mv-moz2-linux-ix-slave11 - No problems.

Source

mv-moz2-linux-ix-slave11 - No problems.

Build/Repack

No problems.

Unit tests

  • We fixed prior build3 the name of the unit test branch name for its sendchanges with this branch name fix but we still need to land the where the sendchanges are sent
  • NOTE I don't know why some talos sendchanges triggered 5 pending talos jobs instead of 1. For next release and after the aforementioned bugs are fixed let's see if it happens again.

List of sendchanges run manually:

buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange --branch release-mozilla-central-linux-talos --revision 900fdd7fb8b2 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build3/linux-i686/en-US/firefox-4.0b3.tar.bz2
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange --branch release-mozilla-central-linux64-talos --revision 900fdd7fb8b2 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build3/linux-x86_64/en-US/firefox-4.0b3.tar.bz2
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange --branch release-mozilla-central-win32-talos --revision 900fdd7fb8b2 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build3/unsigned/win32/en-US/firefox-4.0b3.zip
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange --branch release-mozilla-central-macosx-talos --revision 900fdd7fb8b2 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build3/mac/en-US/Firefox 4.0 Beta 3.dmg'
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange --branch release-mozilla-central-macosx64-talos --revision 900fdd7fb8b2 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build3/mac64/en-US/Firefox 4.0 Beta 3.dmg'
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange-unittest --branch release-mozilla-central-linux-opt-unittest --revision 900fdd7fb8b2 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build3/linux-i686/en-US/firefox-4.0b3.tar.bz2 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build3/linux-i686/en-US/firefox-4.0b3.tests.zip
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange-unittest --branch release-mozilla-central-linux64-opt-unittest --revision 900fdd7fb8b2 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build3/linux-x86_64/en-US/firefox-4.0b3.tar.bz2 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build3/linux-x86_64/en-US/firefox-4.0b3.tests.zip
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange-unittest --branch release-mozilla-central-win32-opt-unittest --revision 900fdd7fb8b2 http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build3/unsigned/win32/en-US/firefox-4.0b3.zip http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build3/unsigned/win32/en-US/firefox-4.0b3.tests.zip
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange-unittest --branch release-mozilla-central-macosx-opt-unittest --revision 900fdd7fb8b2 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build3/mac/en-US/Firefox 4.0 Beta 3.dmg' 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build3/mac/en-US/Firefox 4.0 Beta 3.tests.zip'
buildbot sendchange --master production-master01.build.mozilla.org:9009 --username sendchange-unittest --branch release-mozilla-central-macosx64-opt-unittest --revision 900fdd7fb8b2 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build3/mac64/en-US/Firefox 4.0 Beta 3.dmg' 'http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b3-candidates/build3/mac64/en-US/Firefox 4.0 Beta 3.tests.zip'
Results

These were the only oranges we had. The oranges are known and the last one is permanent:

Sign

Using instructions.

  • tagged tools

en-US.xpi removed from stage:

#ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/4.0b3-candidates/build2
rm en-US.xpi

Run the following:

PRODUCT=firefox
VERSION=4.0b3
BUILD=3
TAG=FIREFOX_4_0b3_RELEASE
REPO="mozilla-central"
EMAIL=armenzg@mozilla.com
KEYDIR=d:/2009-keys

cd ~/hg-tools
hg pull && hg up -r ${TAG}
mkdir -p ~/signing-work/${PRODUCT}-${VERSION}
cd ~/signing-work/${PRODUCT}-${VERSION}
cp ~/hg-tools/release/signing/* .

make PRODUCT=${PRODUCT} VERSION=${VERSION} \
  BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} \
  KEYDIR=${KEYDIR} USE_NEW=1

Inspected stdout, signing log. They looked good. Ran the following:

make postsign PRODUCT=${PRODUCT} VERSION=${VERSION} \
   BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL}

L10N verification

No problems

Updates

No problems

Update verify

No problems

Final checks before push

# ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/4.0b3-candidates/build3/
find . ! -user ffxbld
find . ! -group firefox
find . -type f ! -perm 644
find . -mindepth 1 -type d ! -perm 755 ! -name 'contrib*'
find . -mindepth 1 -type d ! -perm 755 ! -name 'contrib*'
^^ Last line is WRONG, should be |find . -maxdepth 1 -type d -perm 2775 ! -name 'contrib*'|

Virus check

# ffxbld@stage
clamdscan -m /mnt/netapp/stage/archive.mozilla.org/pub/firefox/nightly/4.0b3-candidates/build3/
/mnt/netapp/stage/archive.mozilla.org/pub/firefox/nightly/4.0b3-candidates/build3: OK

----------- SCAN SUMMARY -----------
Infected files: 0
Time: 1626.579 sec (27 m 6 s)

Update Bouncer

  • bug 571869 has been verified to be fixed
  • Moved mv-moz2-linux-ix-slave16 from pm02 to pm
  • verified release_config.py still said 4.0b3 build3
  • Forced bouncer_submitter build
  • This went green!
  • TODO verify the step went smoothly: Nick says it shouldn't have submitted the various languages. I think Nick was making reference to:
Locales: be, ca, cs, da, de, el, en-US, es-AR, es-ES, et, fi, fr, ga-IE, hu, is, it, ko, ku, lt, nb-NO, nl, nn-NO, pa-IN, pl, pt-BR, pt-PT, ro, ru, sk, sv-SE, tr, uk, zh-CN, zh-TW
Server response:
<?xml version="1.0" encoding="utf-8"?><products><product id="824" name="Firefox-4.0b3"><language locale="be"/><language locale="ca"/><language locale="cs"/><language locale="da"/><language locale="de"/><language locale="el"/><language locale="en-US"/><language locale="es-AR"/><language locale="es-ES"/><language locale="et"/><language locale="fi"/><language locale="fr"/><language locale="ga-IE"/><language locale="hu"/><language locale="is"/><language locale="it"/><language locale="ko"/><language locale="ku"/><language locale="lt"/><language locale="nb-NO"/><language locale="nl"/><language locale="nn-NO"/><language locale="pa-IN"/><language locale="pl"/><language locale="pt-BR"/><language locale="pt-PT"/><language locale="ro"/><language locale="ru"/><language locale="sk"/><language locale="sv-SE"/><language locale="tr"/><language locale="uk"/><language locale="zh-CN"/><language locale="zh-TW"/></product></products>
Adding product: Firefox-4.0b3-Complete
Locales: be, ca, cs, da, de, el, en-US, es-AR, es-ES, et, fi, fr, ga-IE, hu, is, it, ko, ku, lt, nb-NO, nl, nn-NO, pa-IN, pl, pt-BR, pt-PT, ro, ru, sk, sv-SE, tr, uk, zh-CN, zh-TW
Server response:
<?xml version="1.0" encoding="utf-8"?><products><product id="825" name="Firefox-4.0b3-Complete"><language locale="be"/><language locale="ca"/><language locale="cs"/><language locale="da"/><language locale="de"/><language locale="el"/><language locale="en-US"/><language locale="es-AR"/><language locale="es-ES"/><language locale="et"/><language locale="fi"/><language locale="fr"/><language locale="ga-IE"/><language locale="hu"/><language locale="is"/><language locale="it"/><language locale="ko"/><language locale="ku"/><language locale="lt"/><language locale="nb-NO"/><language locale="nl"/><language locale="nn-NO"/><language locale="pa-IN"/><language locale="pl"/><language locale="pt-BR"/><language locale="pt-PT"/><language locale="ro"/><language locale="ru"/><language locale="sk"/><language locale="sv-SE"/><language locale="tr"/><language locale="uk"/><language locale="zh-CN"/><language locale="zh-TW"/></product></products>
Adding product: Firefox-4.0b3-Partial-4.0b2
Locales: be, ca, cs, da, de, el, en-US, es-AR, es-ES, et, fi, fr, ga-IE, hu, is, it, ko, ku, lt, nb-NO, nl, nn-NO, pa-IN, pl, pt-BR, pt-PT, ro, ru, sk, sv-SE, tr, uk, zh-CN, zh-TW
Server response:
<?xml version="1.0" encoding="utf-8"?><products><product id="826" name="Firefox-4.0b3-Partial-4.0b2"><language locale="be"/><language locale="ca"/><language locale="cs"/><language locale="da"/><language locale="de"/><language locale="el"/><language locale="en-US"/><language locale="es-AR"/><language locale="es-ES"/><language locale="et"/><language locale="fi"/><language locale="fr"/><language locale="ga-IE"/><language locale="hu"/><language locale="is"/><language locale="it"/><language locale="ko"/><language locale="ku"/><language locale="lt"/><language locale="nb-NO"/><language locale="nl"/><language locale="nn-NO"/><language locale="pa-IN"/><language locale="pl"/><language locale="pt-BR"/><language locale="pt-PT"/><language locale="ro"/><language locale="ru"/><language locale="sk"/><language locale="sv-SE"/><language locale="tr"/><language locale="uk"/><language locale="zh-CN"/><language locale="zh-TW"/></product></products>

Push to mirrors

# ffxbld@stage
cd /home/ftp/pub/firefox/releases
mkdir 4.0b3
cd 4.0b3
rsync -av --exclude=*tests* --exclude=*crashreporter* --exclude=*.log --exclude=*.txt --exclude=*unsigned* --exclude=*update-backup* /pub/mozilla.org/firefox/nightly/4.0b3-candidates/build3/. .

index.html:

curl -sL https://bugzilla.mozilla.org/attachment.cgi?id=456169 > index.html
vi index.html
# edit to change Beta 1 to Beta 3
for dir in `find . -mindepth 1 -type d`; do cp -pv index.html $dir/; done

Final verify

  • went green after 2nd force build (after a wait). Moved mv-moz2-linux-ix-slave16 back to pm02.
  • sent email when mirrors were absorbed for testing (11:58pm PDT)
  • sent email when mirrors were absorbed for release ( 7:51pm PDT)

Push snippets to beta channel

  • Upon GO from release-drivers
# cltbld@aus2-staging
cd /opt/aus2/snippets/staging
~/bin/backupsnip 20100805-Firefox-4.0b3
~/bin/pushsnip   20100805-Firefox-4.0b3

Remove index.html files

Once we are fully announced (by the driver) we can remove the index.html files:

# ffxbld@stage
cd /pub/mozilla.org/firefox/releases/4.0b3/
find . -name index.html
find . -name index.html | xargs rm -v

Release

  • Emailed the following to metrics@m.c
Firefox,firefox,firefox,4.0b3,4.0
Firefox,firefox,firefox,4.0b4pre,4.0