Personal tools

Releases/Firefox 4.0b8/BuildNotes

From MozillaWiki

Jump to: navigation, search

Contents

Notes About Releasing

Please update the Release:Primer for future releases (bug fixes, changes to automation) as needed

Build Engineers

rail - bug 615555

Signed-off Revision(s)

L10N changesets

Tags

Build # Tag Changeset
1 GECKO20b8_20101214_RELBRANCH abe884259481
FIREFOX_4_0b8_BUILD1/FIREFOX_4_0b8_RELEASE d78f9cb65e91

Build data

Build # Type Build ID Build machine Time to build
1 Linux [compressed .tar.bz2]
Linux64 [compressed .tar.bz2]
Macosx64 [.dmg]
Win32 [.exe/.zip]

Bugs hit

Build 1

Done on pm01

  • No clobberer necessary because this is the first 0.8.0 release on this branch.
  • Landed configs
  • tagged buildbot-configs (production branch), buildbotcustom (production-0.8 branch), and tools (default branch) with FIREFOX_4_0b8_BUILD1 and FIREFOX_4_0b8_RELEASE
  • updated /tools/buildbotcustom/buildbotcustom, /builds/buildbot/configs, and /builds/buildbot/builder_master/tools to latest
  • reconfiged master
  • Left tree open, 0.8.0 automation doesn't require that it closes
  • kicked off automation with:
PYTHONPATH=/builds/buildbot/builder_master1:/builds/buildbot/builder_master1/tools/lib/python python /builds/buildbot/builder_master1/tools/buildbot-helpers/release_sanity.py -u rail -V 4.0b8 --branch mozilla-central --build-number 1 --dryrun localhost:9010
PYTHONPATH=/builds/buildbot/builder_master1:/builds/buildbot/builder_master1/tools/lib/python python /builds/buildbot/builder_master1/tools/buildbot-helpers/release_sanity.py -u rail -V 4.0b8 --branch mozilla-central --build-number 1  localhost:9010

Setup

Tag

The first sendchage was without script_repo_revision:FIREFOX_4_0b8_RELEASE set, and failed in the beginning. The second one sent by release_sanity.py worked fine.

Source

No problems

Build/Repack

Failed l10n repacks: en-ZA, nso, pt-BR

These locales had missing files and compare-locales doesn't handle this case.

Filed bug 619258


  • clobbered all slaves used for repacks (all chunks, all platforms)
  • removed all locales except of en-US from ftp (installers+complete mars)
  • landed the patch and tagged the revision with FIREFOX_4_0b8_BUILD1 and FIREFOX_4_0b8_RELEASE
  • to make sure if the patch works, rebuilt 2nd (failed previously) chunk for linux.
  • win32 repacks didn't work. Landed patch

Later, we noticed that "ja" was missing from 64-bit Linux and "ja-JP-mac" was missing from 64-bit Mac. To fix, did the following manually on moz2-linux64-slave01:

export   HG_SHARE_BASE_DIR=/builds/hg-shared
export   BUILDBOTCUSTOM=http://hg.mozilla.org/build/buildbotcustom
export   BUILDBOT_CONFIGS=http://hg.mozilla.org/build/buildbot-configs
export   CC=/tools/gcc/bin/gcc
export   CXX=/tools/gcc/bin/g++
export   PROPERTIES_FILE=buildprops.json
cd /builds/slave/rel-cen-lnx64-rpk-1/build
/tools/python/bin/python /builds/slave/rel-cen-lnx64-rpk-1/build/scripts/scripts/l10n/create-release-repacks.py -c mozilla/production_config.py -r mozilla/release-firefox-mozilla-central.py -b http://hg.mozilla.org/build/buildbot-configs -t FIREFOX_4_0b8_RELEASE -p linux64 -l ja

And the following on moz2-darwin10-slave13:

export   BUILDBOTCUSTOM=http://hg.mozilla.org/build/buildbotcustom
export   BUILDBOT_CONFIGS=http://hg.mozilla.org/build/buildbot-configs
export   CC=/tools/gcc/bin/gcc
export   CXX=/tools/gcc/bin/g++
export   PROPERTIES_FILE=buildprops.json
cd /builds/slave/rel-cen-osx64-rpk-2/build
/tools/python/bin/python /builds/slave/rel-cen-osx64-rpk-2/build/scripts/scripts/l10n/create-release-repacks.py -c mozilla/production_config.py -r mozilla/release-firefox-mozilla-central.py -b http://hg.mozilla.org/build/buildbot-configs -t FIREFOX_4_0b8_RELEASE -p macosx64 -l ja-JP-mac

Filed bug 619522 on the issue.

Repack 3

Due to bug 619664 we had to resping l10n repacks.

Unittests

Results

Signing

To speed up signing, ran the following prior to all repacks being completed:

PRODUCT=firefox
VERSION=4.0b8
BUILD=1
TAG=FIREFOX_4_0b8_RELEASE
REPO='mozilla-central'
EMAIL=rail@mozilla.com
KEYDIR=d:/2010-keys
cd ~/hg-tools
hg pull && hg up -r ${TAG}
mkdir ~/signing-work/${PRODUCT}-${VERSION}
cd ~/signing-work/${PRODUCT}-${VERSION}
cp ~/hg-tools/release/signing/* .
make df verify-gpg-passphrase setup verify-signcode-passphrase PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
# download target run multiple times to get things as they complete
make download PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1

Once all the l10n and partner repacks finished, ran the following in the same window to do the signing:

# Need to download one more time to make sure we've got everything
make download PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
make stubs sign2 stage verify verify-sigs fake-upload PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
  • verify-asc (last step of verify, before fake-upload) failed due to 3 *.checksum files:
./linux-i686/en-US/firefox-4.0b8.checksums
./linux-x86_64/en-US/firefox-4.0b8.checksums
./mac64/en-US/Firefox 4.0 Beta 8.checksums
  • removed the files manually and rerun fake-upload
  • filed bug 619572 with a patch

Finally:

make postsign PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL}

Got "Signature Verification Succeeded" mail later.

  • We had to respin l10n repacks and sign the files again
    • removed signed files from stage
    • rerun the signing step w/o any problem

L10N Verify

  • A lot of harmless warnings looking like errors: bug 619066

Updates

First attempt failed during unpacking mar

Extracting /builds/slave/rel-cen-updates/build/temp/firefox/4.0b7/ftp/firefox-4.0b7.he.win32.complete.mar to /builds/slave/rel-cen-updates/build/tmpeWytLO-fastmode/195/from

IN SHUTDOWN...
Failed to remove patcher2.pid: No such file or directory
FAIL shell call timed out after 10800 seconds at patcher2.pl line 1619.
program finished with exit code 2
elapsedTime=11041.982708

dmesg on linux-ix-slave13:

ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x0)
ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x12)
ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x2)
ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x0)
ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x0)
ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x1)
ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x0)

Gracefully shutdown linux-ix-slave13, buildbot.tac -> buildbot.tac.off. Filed bug 619624

Special case for old mac universal builds

See Releases/Firefox_4.0b7/BuildNotes#Updates for the details.

#cltbld@aus-staging
cd /opt/aus2/snippets/staging/
cp -r 20101214-Firefox-4.0b8-test 20101214-Firefox-4.0b8-test-fixed
cd 20101214-Firefox-4.0b8-test-fixed/Firefox

mkdir -p 3.7a1/Darwin_Universal-gcc3/20100208064157
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a1/Darwin_Universal-gcc3/20100208064157
mkdir -p 3.7a2/Darwin_Universal-gcc3/20100228155422
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a2/Darwin_Universal-gcc3/20100228155422/
mkdir -p 3.7a3/Darwin_Universal-gcc3/20100315065900
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a3/Darwin_Universal-gcc3/20100315065900/
mkdir -p 3.7a4/Darwin_Universal-gcc3/20100407105356
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a4/Darwin_Universal-gcc3/20100407105356/
mkdir -p 3.7a5/Darwin_Universal-gcc3/20100610053455
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a5/Darwin_Universal-gcc3/20100610053455
mkdir -p 4.0b1/Darwin_Universal-gcc3/20100630131607
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 4.0b1/Darwin_Universal-gcc3/20100630131607/
mkdir -p 4.0b2/Darwin_Universal-gcc3/20100720175703
rsync -av --exclude partial.txt 4.0b2/Darwin_x86_64-gcc3/20100720180051/ 4.0b2/Darwin_Universal-gcc3/20100720175703/
mkdir -p 4.0b3/Darwin_Universal-gcc3/20100805181943/
rsync -av --exclude partial.txt 4.0b3/Darwin_x86_64-gcc3/20100805182338/ 4.0b3/Darwin_Universal-gcc3/20100805181943/
mkdir -p 4.0b4/Darwin_Universal-gcc3/20100818121614
rsync -av --exclude partial.txt 4.0b4/Darwin_x86_64-gcc3/20100818121922/ 4.0b4/Darwin_Universal-gcc3/20100818121614/
mkdir -p 4.0b5/Darwin_x86-gcc3-u-ppc-i386/20100831070010
rsync -av --exclude partial.txt 4.0b5/Darwin_x86_64-gcc3/20100831070403/ 4.0b5/Darwin_x86-gcc3-u-ppc-i386/20100831070010/
mkdir -p 4.0b6/Darwin_x86-gcc3-u-ppc-i386/20100914072643
rsync -av --exclude partial.txt 4.0b6/Darwin_x86_64-gcc3/20100914073111/ 4.0b6/Darwin_x86-gcc3-u-ppc-i386/20100914072643/

mkdir -p 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/{beta,release}test


# setup snippet
cat > 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/releasetest/complete.txt << EOF
version=1
type=complete
url=http://download.mozilla.org/?product=firefox-4.0b8-complete&os=osx64&lang=ja-JP-mac
hashFunction=SHA512
hashValue=66ffd1fbcf11d90680774891dc5b4c57122d0e91e77184a98c58d437f56dbf871203bc449cfb5094daf0cdb1bc8d867c6aa4ec66f4025bf9302172833f282707
size=27373764
build=20101214165004
appv=4.0 Beta 8
extv=4.0b8
detailsUrl=http://www.mozilla.com/ja-JP-mac/firefox/4.0b8/releasenotes/
EOF

sed -e 's!^url=.*!url=http://stage-old.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/update/mac64/ja-JP-mac/firefox-4.0b8.complete.mar!' \
   4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/releasetest/complete.txt \
 > 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/betatest/complete.txt

# copy to other locations, ja was added at b5
dirs=`find 4.0b{4,5,6,7} -maxdepth 1 -mindepth 1 -type d -name '*Darwin*' ! -wholename '4.0b6/Darwin_x86_64-gcc3' | sort`
for d in $dirs; do
  rsync -av 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac $d/*/
done

# fix paths/os
find */Darwin* -type f -exec sed -i -e 's!update/mac64/!update/mac/!' -e 's!os=osx64!os=osx!' {} \;

cd ../../

rsync -a 20101214-Firefox-4.0b8-test-fixed/ 20101214-Firefox-4.0b8-fixed/
cd 20101214-Firefox-4.0b8-fixed/Firefox
find . -depth -name 'betatest' -type d -exec rm -rf {} \;
find . -mindepth 4 -maxdepth 4 -type d -exec mv {}/{releasetest,beta} \;


~/bin/backupsnip 20101214-Firefox-4.0b8-test-fixed
~/bin/pushsnip   20101214-Firefox-4.0b8-test-fixed
# ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/update 
ln -s mac64 mac
/pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/
ln -s mac64 mac


Realized that we don't generate snippets for linux64/ja and a typo in the previous script (-wholename '4.0b6/Darwin_x86_64-gcc3' should be -wholename '4.0b7/Darwin_x86_64-gcc3'), generated snippets again:

#cltbld@aus-staging
cd /opt/aus2/snippets/staging/
rsync -a 20101214-Firefox-4.0b8-test/ 20101214-Firefox-4.0b8-test-fixed2/
cd 20101214-Firefox-4.0b8-test-fixed2/Firefox

mkdir -p 3.7a1/Darwin_Universal-gcc3/20100208064157
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a1/Darwin_Universal-gcc3/20100208064157
mkdir -p 3.7a2/Darwin_Universal-gcc3/20100228155422
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a2/Darwin_Universal-gcc3/20100228155422/
mkdir -p 3.7a3/Darwin_Universal-gcc3/20100315065900
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a3/Darwin_Universal-gcc3/20100315065900/
mkdir -p 3.7a4/Darwin_Universal-gcc3/20100407105356
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a4/Darwin_Universal-gcc3/20100407105356/
mkdir -p 3.7a5/Darwin_Universal-gcc3/20100610053455
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a5/Darwin_Universal-gcc3/20100610053455
mkdir -p 4.0b1/Darwin_Universal-gcc3/20100630131607
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 4.0b1/Darwin_Universal-gcc3/20100630131607/
mkdir -p 4.0b2/Darwin_Universal-gcc3/20100720175703
rsync -av --exclude partial.txt 4.0b2/Darwin_x86_64-gcc3/20100720180051/ 4.0b2/Darwin_Universal-gcc3/20100720175703/
mkdir -p 4.0b3/Darwin_Universal-gcc3/20100805181943/
rsync -av --exclude partial.txt 4.0b3/Darwin_x86_64-gcc3/20100805182338/ 4.0b3/Darwin_Universal-gcc3/20100805181943/
mkdir -p 4.0b4/Darwin_Universal-gcc3/20100818121614
rsync -av --exclude partial.txt 4.0b4/Darwin_x86_64-gcc3/20100818121922/ 4.0b4/Darwin_Universal-gcc3/20100818121614/
mkdir -p 4.0b5/Darwin_x86-gcc3-u-ppc-i386/20100831070010
rsync -av --exclude partial.txt 4.0b5/Darwin_x86_64-gcc3/20100831070403/ 4.0b5/Darwin_x86-gcc3-u-ppc-i386/20100831070010/
mkdir -p 4.0b6/Darwin_x86-gcc3-u-ppc-i386/20100914072643
rsync -av --exclude partial.txt 4.0b6/Darwin_x86_64-gcc3/20100914073111/ 4.0b6/Darwin_x86-gcc3-u-ppc-i386/20100914072643/

mkdir -p 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/{beta,release}test

# setup snippet
cat > 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/releasetest/complete.txt << EOF
version=1
type=complete
url=http://download.mozilla.org/?product=firefox-4.0b8-complete&os=osx64&lang=ja-JP-mac
hashFunction=SHA512
hashValue=66ffd1fbcf11d90680774891dc5b4c57122d0e91e77184a98c58d437f56dbf871203bc449cfb5094daf0cdb1bc8d867c6aa4ec66f4025bf9302172833f282707
size=27373764
build=20101214165004
appv=4.0 Beta 8
extv=4.0b8
detailsUrl=http://www.mozilla.com/ja-JP-mac/firefox/4.0b8/releasenotes/
EOF

sed -e 's!^url=.*!url=http://stage-old.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/update/mac64/ja-JP-mac/firefox-4.0b8.complete.mar!' \
   4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/releasetest/complete.txt \
 > 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/betatest/complete.txt

# copy to other locations, ja was added at b5
dirs=`find 4.0b{4,5,6,7} -maxdepth 1 -mindepth 1 -type d -name '*Darwin*' ! -wholename '4.0b7/Darwin_x86_64-gcc3' | sort`
for d in $dirs; do
  rsync -av 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac $d/*/
done

# fix paths/os
find */Darwin* -type f -exec sed -i -e 's!update/mac64/!update/mac/!' -e 's!os=osx64!os=osx!' {} \;

mkdir -p 4.0b7/Linux_x86_64-gcc3/20101104131204/ja/{beta,release}test


# setup snippet
cat > 4.0b7/Linux_x86_64-gcc3/20101104131204/ja/releasetest/complete.txt << EOF
version=1
type=complete
url=http://download.mozilla.org/?product=firefox-4.0b8-complete&os=linux64&lang=ja
hashFunction=SHA512
hashValue=0f3f22fc577bb06a01c4ab011448fb97bda99ede7ad4af6fbea8b89a819d2c4decf67d6bf9890ba533b395d2d2133eb0c60e7e4048f6a5151dd8e53dba0178a7
size=15999286
build=20101214180840
appv=4.0 Beta 8
extv=4.0b8
detailsUrl=http://www.mozilla.com/ja/firefox/4.0b8/releasenotes/
EOF

sed -e 's!^url=.*!url=http://stage-old.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/update/linux-x86_64/ja/firefox-4.0b8.complete.mar!' \
   4.0b7/Linux_x86_64-gcc3/20101104131204/ja/releasetest/complete.txt \
 > 4.0b7/Linux_x86_64-gcc3/20101104131204/ja/betatest/complete.txt

# copy to other locations, ja was added at b5
dirs=`find 4.0b{4,5,6,7} -maxdepth 1 -mindepth 1 -type d -name '*Linux_x86_64-gcc3*' ! -wholename '4.0b7/Linux_x86_64-gcc3' | sort`
for d in $dirs; do
  rsync -av 4.0b7/Linux_x86_64-gcc3/20101104131204/ja $d/*/
done


cd ../../

rsync -a 20101214-Firefox-4.0b8-test-fixed2/ 20101214-Firefox-4.0b8-fixed2/
cd 20101214-Firefox-4.0b8-fixed2/Firefox
find . -depth -name 'betatest' -type d -exec rm -rf {} \;
find . -mindepth 4 -maxdepth 4 -type d -exec mv {}/{releasetest,beta} \;


~/bin/backupsnip 20101214-Firefox-4.0b8-test-fixed2
~/bin/pushsnip   20101214-Firefox-4.0b8-test-fixed2

TODO: add ja/ja-JP-mac to update verify configs fpr linux64 and mac64 and rerun them.

Offering completes as partials for win32

Due to signing changes that landed just before b7, win32 complete MARs for b7 were broken by signing. This means that win32 partial MARs for b8 will be broken until we regenerate/sign complete MARs for b7, and then regenerate the partial MARs for b8.

In the interim, we'll be offering the win32 complete MARs as partials. Other platforms are not affected.

#cltbld@aus2-staging
cd /opt/aus2/incoming/3/Firefox/4.0b7/WINNT_x86-msvc/20101104142426
time ~coop/completes_as_partials.bash | tee ~/4.0b7_completes_as_partials.log
Updated 114 partial snippets with completes

 real    0m9.897s
 user    0m0.192s
 sys     0m1.329s

Root cause of win32 updates issues

Starting with bug 554321, we introduce a patch that lets us sign partner-repacks at the same time as everything else (rather than the previous practice of signing locale repacks, then generating partner-repacks off of signed builds and resigning). This patch was landed shortly before the tools repo was tagged for beta 7. Beta 7 proceeded as normal, using this signing patch and passed verify_signatures, and update_verify.

However, we later hit problems in staging before the 3.5/3.6 releases where the signed MARs were inconsistent with the installers due to a problem with how the list of files to be signed was sorted in the partner-repack patch, causing MARs to be signed ahead of installers, which tripped up our caching. The issue was fixed in bug 613320. The problems discovered in that bug are confirmed *not* affecting the stable branches, so the next 3.5/3.6 releases should not encounter these problems.

Since we could generate valid partials from the busted b7 MARs, for this release, we have switched the partial update snippet to point to the complete MAR which applies correctly regardless of the busted beta 7 MARs. Since the signing issue has already been fixed, this means that going forward, the beta 8->beta 9 (and onwards) partials should be able to be generated without issues.

The reason that this slipped through in the first place was because I only tested for consistency between locales when testing the signed builds, and did not run update_verify on builds signed with the original partner-repack patch. As part of the signing changes that have landed, we now verify the checksums of the MAR and installer internals as part of verify_signatures, which will catch the case where we have inconsistent checksums on any installer/MAR internals.

Looking forward, the best way to decrease the possibility of things like this happening in the future would probably be to expand the coverage provided by update_verify (we should have a way to find out about inconsistencies in applying updates before we try to generate updates for the next release). We already have the checksum verification steps in the signing code now to guard against the specific cases of checksum differences between the MARs and installers.

Post repack 3 snippet fixes

#cltbld@aus-staging
cd /opt/aus2/snippets/staging/
rsync -a 20101214-Firefox-4.0b8-test/ 20101214-Firefox-4.0b8-test-fixed3/
cd 20101214-Firefox-4.0b8-test-fixed3/Firefox

# Fix mac platform name
mkdir -p 3.7a1/Darwin_Universal-gcc3/20100208064157
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a1/Darwin_Universal-gcc3/20100208064157
mkdir -p 3.7a2/Darwin_Universal-gcc3/20100228155422
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a2/Darwin_Universal-gcc3/20100228155422/
mkdir -p 3.7a3/Darwin_Universal-gcc3/20100315065900
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a3/Darwin_Universal-gcc3/20100315065900/
mkdir -p 3.7a4/Darwin_Universal-gcc3/20100407105356
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a4/Darwin_Universal-gcc3/20100407105356/
mkdir -p 3.7a5/Darwin_Universal-gcc3/20100610053455
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a5/Darwin_Universal-gcc3/20100610053455
mkdir -p 4.0b1/Darwin_Universal-gcc3/20100630131607
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 4.0b1/Darwin_Universal-gcc3/20100630131607/
mkdir -p 4.0b2/Darwin_Universal-gcc3/20100720175703
rsync -av --exclude partial.txt 4.0b2/Darwin_x86_64-gcc3/20100720180051/ 4.0b2/Darwin_Universal-gcc3/20100720175703/
mkdir -p 4.0b3/Darwin_Universal-gcc3/20100805181943/
rsync -av --exclude partial.txt 4.0b3/Darwin_x86_64-gcc3/20100805182338/ 4.0b3/Darwin_Universal-gcc3/20100805181943/
mkdir -p 4.0b4/Darwin_Universal-gcc3/20100818121614
rsync -av --exclude partial.txt 4.0b4/Darwin_x86_64-gcc3/20100818121922/ 4.0b4/Darwin_Universal-gcc3/20100818121614/
mkdir -p 4.0b5/Darwin_x86-gcc3-u-ppc-i386/20100831070010
rsync -av --exclude partial.txt 4.0b5/Darwin_x86_64-gcc3/20100831070403/ 4.0b5/Darwin_x86-gcc3-u-ppc-i386/20100831070010/
mkdir -p 4.0b6/Darwin_x86-gcc3-u-ppc-i386/20100914072643
rsync -av --exclude partial.txt 4.0b6/Darwin_x86_64-gcc3/20100914073111/ 4.0b6/Darwin_x86-gcc3-u-ppc-i386/20100914072643/

# Generate ja-JP-mac snippet
mkdir -p 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/{beta,release}test

cat > 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/releasetest/complete.txt << EOF
version=1
type=complete
url=http://download.mozilla.org/?product=firefox-4.0b8-complete&os=osx64&lang=ja-JP-mac
hashFunction=SHA512
hashValue=fea614985261dc8e0cae6f05d19a55375b49f0560222b2892d703ad7b555a6fe2ad884a3fd8e876ca87ab5ccd1cdf338932af13977b2c61173815e9eeb6a0c5b
size=27373799
build=20101214165004
appv=4.0 Beta 8
extv=4.0b8
detailsUrl=http://www.mozilla.com/ja-JP-mac/firefox/4.0b8/releasenotes/
EOF

sed -e 's!^url=.*!url=http://stage-old.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/update/mac64/ja-JP-mac/firefox-4.0b8.complete.mar!' \
   4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/releasetest/complete.txt \
 > 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/betatest/complete.txt
 
# copy to other locations, ja was added at b5
dirs=`find 4.0b{4,5,6,7} -maxdepth 1 -mindepth 1 -type d -name '*Darwin*' ! -wholename '4.0b7/Darwin_x86_64-gcc3' | sort`
for d in $dirs; do
  rsync -av 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac $d/*/
done

# fix paths/os
find */Darwin* -type f -exec sed -i -e 's!update/mac64/!update/mac/!' -e 's!os=osx64!os=osx!' {} \;

# Generate ja linux64 snippet
mkdir -p 4.0b7/Linux_x86_64-gcc3/20101104131204/ja/{beta,release}test

# setup snippet
cat > 4.0b7/Linux_x86_64-gcc3/20101104131204/ja/releasetest/complete.txt << EOF
version=1
type=complete
url=http://download.mozilla.org/?product=firefox-4.0b8-complete&os=linux64&lang=ja
hashFunction=SHA512
hashValue=482d120eebecccafce5e870e8a9e3fc1295d964960d21523c8cf7a1079c6a55d830b39eaa992f36e372f19998e920ce601a866849f9001a8aac5bafdd14a41e1
size=15996568
build=20101214180840
appv=4.0 Beta 8
extv=4.0b8
detailsUrl=http://www.mozilla.com/ja/firefox/4.0b8/releasenotes/
EOF

sed -e 's!^url=.*!url=http://stage-old.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/update/linux-x86_64/ja/firefox-4.0b8.complete.mar!' \
   4.0b7/Linux_x86_64-gcc3/20101104131204/ja/releasetest/complete.txt \
 > 4.0b7/Linux_x86_64-gcc3/20101104131204/ja/betatest/complete.txt

# copy to other locations, ja was added at b5
dirs=`find 4.0b{4,5,6,7} -maxdepth 1 -mindepth 1 -type d -name '*Linux_x86_64-gcc3*' ! -wholename '4.0b7/Linux_x86_64-gcc3' | sort`
for d in $dirs; do
  rsync -av 4.0b7/Linux_x86_64-gcc3/20101104131204/ja $d/*/
done


# win beta7 partial -> complete

dirs=`find 4.0b7/WINNT_x86-msvc/20101104142426 -maxdepth 1 -mindepth 1 -type d | sort`
for d in $dirs; do
    rsync -av $d/betatest/complete.txt $d/betatest/partial.txt
    rsync -av $d/releasetest/complete.txt $d/releasetest/partial.txt
    sed -i 's/type=complete/type=partial/' $d/{beta,release}test/partial.txt
done

cd ../../

rsync -a 20101214-Firefox-4.0b8-test-fixed3/ 20101214-Firefox-4.0b8-fixed3/
cd 20101214-Firefox-4.0b8-fixed3/Firefox
find . -depth -name 'betatest' -type d -exec rm -rf {} \;
find . -mindepth 4 -maxdepth 4 -type d -exec mv {}/{releasetest,beta} \;


~/bin/backupsnip 20101214-Firefox-4.0b8-test-fixed3
~/bin/pushsnip   20101214-Firefox-4.0b8-test-fixed3

Fix "ja" locale snippet type

cd /opt/aus2/snippets/staging/
rsync -a 20101214-Firefox-4.0b8-test-fixed3/ 20101214-Firefox-4.0b8-test-fixed4/
cd 20101214-Firefox-4.0b8-test-fixed4/Firefox


for f in */*Darwin*/*/ja*/*/complete.txt */*Linux_x86_64-gcc3*/*/ja*/*/complete.txt; do
  t=$(dirname $f)/partial.txt
  rsync -av $f $t
  sed -i 's/type=complete/type=partial/' $t
done

~/bin/backupsnip 20101214-Firefox-4.0b8-test-fixed4
~/bin/pushsnip   20101214-Firefox-4.0b8-test-fixed4
  
cd ../../

rsync -a 20101214-Firefox-4.0b8-test-fixed4/ 20101214-Firefox-4.0b8-fixed4/
cd 20101214-Firefox-4.0b8-fixed4/Firefox
find . -depth -name 'betatest' -type d -exec rm -rf {} \;
find . -mindepth 4 -maxdepth 4 -type d -exec mv {}/{releasetest,beta} \;

Add ja-JP-mac and swap to mac in dir names

Fix 4.0b7 -> 4.0b8 Mac updates

Patcher generates Mac updates using the "Darwin_x86_64-gcc3" OS label, but incoming clients use "Darwin_x86_64-gcc3-u-i386-x86_64" and "Darwin_x86-gcc3-u-i386-x86_64" in their requests. To fix, we added symlinks:

cd /opt/aus2/incoming/3/Firefox/4.0b7
ln -s Darwin_x86_64-gcc3 Darwin_x86_64-gcc3-u-i386-x86_64
ln -s Darwin_x86_64-gcc3 Darwin_x86-gcc3-u-i386-x86_64

Update Bouncer

Used "Force Build" on "release-mozilla-central-bouncer_submitter" builder. No problems with it.

Naming fixup

The buildbot setup leaves us with mac64 directories on the ftp server, and an expectation to use os=osx64 in bouncer. The mozilla.com maintainers would really rather not change from os=osx, so we're making everything match what we did for 4.0b6.

Rename on ftp server

Fix the file locations, but leave a symlink so that betatest keeps working for QA, and updates will work for b9.

# ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1
rm mac && mv mac64 mac && ln -s mac mac64
cd update
rm mac && mv mac64 mac && ln -s mac mac64

Fix-up Snippets

Done above, as part of the "-fixed" sets of snippets.

Fix up *SUMS

On signing box

cd ~/signing-work/firefox-4.0b8/
mkdir signed-build1-redo-SUMS
cp signed-build1/*SUMS signed-build1-redo-SUMS/
cd signed-build1-redo-SUMS
sed -i -e 's/mac64/mac/' *SUMS
../signing/sign-files .
rsync -av -e "ssh -i $HOME/.ssh/ffxbld_dsa" * \
 ffxbld@stage.mozilla.org:/pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/

Adjust Bouncer Entries

Manually changed all "osx64" entries to "osx" platform, and edited locations, changing "mac64" to "mac".

Update Verify

  • landed a patch to enable ja/ja-JP-mac update verify checks.
  • linux64 update verify failure for "ja". The current code expects that 4.0b7-4.0b8 snippet should be type="partial", while AUS2 returns a snippet with type="complete" for "ja". patcher2 doesn't generate partial mars for 64bit platforms, because they are not in shipped-locales file. I checked it manually: downloaded 4.0b7, switched to betatest channel, checked for updates, applied updates, checked version.
  • The same issue for jp-JP-mac. No mac around to test using the same scenario, but I believe that it'll work. If someone could check this, I'd be glad.
  • update verify of win32 went red. Update logs look good, no update failure. The errors were produced by file size sanity check, which tries to compare partial and complete mar sizes and fails if the partial is bigger than the complete mar. This happens only for versions before b7 (N-2) (we don't run full check for them, just sanity check). I suspect that the file size sanity check part is buggy, because we don't provide partial mars for versions N-2 and the snippets say the same.

Final checks before push and virus check

bug 540598 adds a new pre_push_checks builder which should be run with at least 2 properties set: revision and release_config. Web form submitted with the following fields set:

  • branch: mozilla-central
  • revision: FIREFOX_4_0b8_RELEASE
  • release_config: mozilla/release-firefox-mozilla-2.0.py

First run failed because the scripts were chmod 644. chmoded to 755, commited, tagged as FIREFOX_4_0b8_BUILD1 and FIREFOX_4_0b8_RELEASE. Second run passed.

Socorro product data updates

Push to mirrors

bug 540598 adds a new push_to_mirrors builder which should be run with at least 2 properties set: revision and release_config. Web form submitted with the following fields set:

  • branch: mozilla-central
  • revision: FIREFOX_4_0b8_RELEASE
  • release_config: mozilla/release-firefox-mozilla-2.0.py

Final verify

Failed for ja linux64 and ja-JP-mac. As in case of update verify, final verification expects partial updates for b7, but the current patcher2.pl doesn't generate partials for ja+64bit. Can be ignored.

Push snippets to beta channel

As cltbld@aus2-staging:

cd /opt/aus2/snippets/staging
~/bin/backupsnip 20101214-Firefox-4.0b8-fixed4
...
cd /opt/aus2/snippets/staging
time ~/bin/pushsnip 20101214-Firefox-4.0b8-fixed4
real    2m0.066s
user    0m0.219s
sys     0m3.201s

Send mail to metrics

Sent mail to metrics@mozilla.com with the following contents:

Firefox,firefox,firefox,4.0b8,4.0
Firefox,firefox,firefox,4.0b9pre,4.0

Release

added rsyncd exclude entries for 4.0b6 and 4.0b7:

# cltbld@stage.mozilla.org
grep '4.0b\(6\|7\)' /pub/mozilla.org/zz/rsyncd-mozilla-releases.exclude
- firefox/releases/4.0b6
- firefox/releases/4.0b7