Releases/Firefox 4.0b8/BuildNotes
From MozillaWiki
< Releases | Firefox 4.0b8
Contents
- 1 Notes About Releasing
- 2 Build Engineers
- 3 Signed-off Revision(s)
- 4 L10N changesets
- 5 Tags
- 6 Build data
- 7 Bugs hit
- 8 Build 1
- 8.1 Setup
- 8.2 Tag
- 8.3 Source
- 8.4 Build/Repack
- 8.5 Unittests
- 8.6 Signing
- 8.7 L10N Verify
- 8.8 Updates
- 8.9 Update Bouncer
- 8.10 Naming fixup
- 8.11 Update Verify
- 8.12 Final checks before push and virus check
- 8.13 Socorro product data updates
- 8.14 Push to mirrors
- 8.15 Final verify
- 8.16 Push snippets to beta channel
- 8.17 Send mail to metrics
- 8.18 Release
Notes About Releasing
Please update the Release:Primer for future releases (bug fixes, changes to automation) as needed
Build Engineers
rail - bug 615555
Signed-off Revision(s)
- Build 1: abe884259481
L10N changesets
- wget https://l10n-stage-sj.mozilla.org/shipping/l10n-changesets?ms=fx4.0b8
- hit "ship it" from FF4 beta8 page
Tags
| Build # | Tag | Changeset |
| 1 | GECKO20b8_20101214_RELBRANCH | abe884259481 |
| FIREFOX_4_0b8_BUILD1/FIREFOX_4_0b8_RELEASE | d78f9cb65e91 |
Build data
| Build # | Type | Build ID | Build machine | Time to build |
| 1 | Linux [compressed .tar.bz2] | |||
| Linux64 [compressed .tar.bz2] | ||||
| Macosx64 [.dmg] | ||||
| Win32 [.exe/.zip] |
Bugs hit
Build 1
Done on pm01
- No clobberer necessary because this is the first 0.8.0 release on this branch.
- Landed configs
- tagged buildbot-configs (production branch), buildbotcustom (production-0.8 branch), and tools (default branch) with FIREFOX_4_0b8_BUILD1 and FIREFOX_4_0b8_RELEASE
- updated /tools/buildbotcustom/buildbotcustom, /builds/buildbot/configs, and /builds/buildbot/builder_master/tools to latest
- reconfiged master
- Left tree open, 0.8.0 automation doesn't require that it closes
- kicked off automation with:
PYTHONPATH=/builds/buildbot/builder_master1:/builds/buildbot/builder_master1/tools/lib/python python /builds/buildbot/builder_master1/tools/buildbot-helpers/release_sanity.py -u rail -V 4.0b8 --branch mozilla-central --build-number 1 --dryrun localhost:9010 PYTHONPATH=/builds/buildbot/builder_master1:/builds/buildbot/builder_master1/tools/lib/python python /builds/buildbot/builder_master1/tools/buildbot-helpers/release_sanity.py -u rail -V 4.0b8 --branch mozilla-central --build-number 1 localhost:9010
Setup
Tag
The first sendchage was without script_repo_revision:FIREFOX_4_0b8_RELEASE set, and failed in the beginning. The second one sent by release_sanity.py worked fine.
Source
No problems
Build/Repack
Failed l10n repacks: en-ZA, nso, pt-BR
These locales had missing files and compare-locales doesn't handle this case.
Filed bug 619258
- clobbered all slaves used for repacks (all chunks, all platforms)
- removed all locales except of en-US from ftp (installers+complete mars)
- landed the patch and tagged the revision with FIREFOX_4_0b8_BUILD1 and FIREFOX_4_0b8_RELEASE
- to make sure if the patch works, rebuilt 2nd (failed previously) chunk for linux.
- win32 repacks didn't work. Landed patch
Later, we noticed that "ja" was missing from 64-bit Linux and "ja-JP-mac" was missing from 64-bit Mac. To fix, did the following manually on moz2-linux64-slave01:
export HG_SHARE_BASE_DIR=/builds/hg-shared export BUILDBOTCUSTOM=http://hg.mozilla.org/build/buildbotcustom export BUILDBOT_CONFIGS=http://hg.mozilla.org/build/buildbot-configs export CC=/tools/gcc/bin/gcc export CXX=/tools/gcc/bin/g++ export PROPERTIES_FILE=buildprops.json cd /builds/slave/rel-cen-lnx64-rpk-1/build /tools/python/bin/python /builds/slave/rel-cen-lnx64-rpk-1/build/scripts/scripts/l10n/create-release-repacks.py -c mozilla/production_config.py -r mozilla/release-firefox-mozilla-central.py -b http://hg.mozilla.org/build/buildbot-configs -t FIREFOX_4_0b8_RELEASE -p linux64 -l ja
And the following on moz2-darwin10-slave13:
export BUILDBOTCUSTOM=http://hg.mozilla.org/build/buildbotcustom export BUILDBOT_CONFIGS=http://hg.mozilla.org/build/buildbot-configs export CC=/tools/gcc/bin/gcc export CXX=/tools/gcc/bin/g++ export PROPERTIES_FILE=buildprops.json cd /builds/slave/rel-cen-osx64-rpk-2/build /tools/python/bin/python /builds/slave/rel-cen-osx64-rpk-2/build/scripts/scripts/l10n/create-release-repacks.py -c mozilla/production_config.py -r mozilla/release-firefox-mozilla-central.py -b http://hg.mozilla.org/build/buildbot-configs -t FIREFOX_4_0b8_RELEASE -p macosx64 -l ja-JP-mac
Filed bug 619522 on the issue.
Repack 3
Due to bug 619664 we had to resping l10n repacks.
Unittests
Results
Signing
To speed up signing, ran the following prior to all repacks being completed:
PRODUCT=firefox
VERSION=4.0b8
BUILD=1
TAG=FIREFOX_4_0b8_RELEASE
REPO='mozilla-central'
EMAIL=rail@mozilla.com
KEYDIR=d:/2010-keys
cd ~/hg-tools
hg pull && hg up -r ${TAG}
mkdir ~/signing-work/${PRODUCT}-${VERSION}
cd ~/signing-work/${PRODUCT}-${VERSION}
cp ~/hg-tools/release/signing/* .
make df verify-gpg-passphrase setup verify-signcode-passphrase PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
# download target run multiple times to get things as they complete
make download PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
Once all the l10n and partner repacks finished, ran the following in the same window to do the signing:
# Need to download one more time to make sure we've got everything
make download PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
make stubs sign2 stage verify verify-sigs fake-upload PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
- verify-asc (last step of verify, before fake-upload) failed due to 3 *.checksum files:
./linux-i686/en-US/firefox-4.0b8.checksums ./linux-x86_64/en-US/firefox-4.0b8.checksums ./mac64/en-US/Firefox 4.0 Beta 8.checksums
- removed the files manually and rerun fake-upload
- filed bug 619572 with a patch
Finally:
make postsign PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL}
Got "Signature Verification Succeeded" mail later.
- We had to respin l10n repacks and sign the files again
- removed signed files from stage
- rerun the signing step w/o any problem
L10N Verify
- A lot of harmless warnings looking like errors: bug 619066
Updates
First attempt failed during unpacking mar
Extracting /builds/slave/rel-cen-updates/build/temp/firefox/4.0b7/ftp/firefox-4.0b7.he.win32.complete.mar to /builds/slave/rel-cen-updates/build/tmpeWytLO-fastmode/195/from IN SHUTDOWN... Failed to remove patcher2.pid: No such file or directory FAIL shell call timed out after 10800 seconds at patcher2.pl line 1619. program finished with exit code 2 elapsedTime=11041.982708
dmesg on linux-ix-slave13:
ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x0) ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x12) ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x2) ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x0) ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x0) ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x1) ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x0)
Gracefully shutdown linux-ix-slave13, buildbot.tac -> buildbot.tac.off. Filed bug 619624
Special case for old mac universal builds
See Releases/Firefox_4.0b7/BuildNotes#Updates for the details.
#cltbld@aus-staging
cd /opt/aus2/snippets/staging/
cp -r 20101214-Firefox-4.0b8-test 20101214-Firefox-4.0b8-test-fixed
cd 20101214-Firefox-4.0b8-test-fixed/Firefox
mkdir -p 3.7a1/Darwin_Universal-gcc3/20100208064157
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a1/Darwin_Universal-gcc3/20100208064157
mkdir -p 3.7a2/Darwin_Universal-gcc3/20100228155422
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a2/Darwin_Universal-gcc3/20100228155422/
mkdir -p 3.7a3/Darwin_Universal-gcc3/20100315065900
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a3/Darwin_Universal-gcc3/20100315065900/
mkdir -p 3.7a4/Darwin_Universal-gcc3/20100407105356
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a4/Darwin_Universal-gcc3/20100407105356/
mkdir -p 3.7a5/Darwin_Universal-gcc3/20100610053455
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a5/Darwin_Universal-gcc3/20100610053455
mkdir -p 4.0b1/Darwin_Universal-gcc3/20100630131607
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 4.0b1/Darwin_Universal-gcc3/20100630131607/
mkdir -p 4.0b2/Darwin_Universal-gcc3/20100720175703
rsync -av --exclude partial.txt 4.0b2/Darwin_x86_64-gcc3/20100720180051/ 4.0b2/Darwin_Universal-gcc3/20100720175703/
mkdir -p 4.0b3/Darwin_Universal-gcc3/20100805181943/
rsync -av --exclude partial.txt 4.0b3/Darwin_x86_64-gcc3/20100805182338/ 4.0b3/Darwin_Universal-gcc3/20100805181943/
mkdir -p 4.0b4/Darwin_Universal-gcc3/20100818121614
rsync -av --exclude partial.txt 4.0b4/Darwin_x86_64-gcc3/20100818121922/ 4.0b4/Darwin_Universal-gcc3/20100818121614/
mkdir -p 4.0b5/Darwin_x86-gcc3-u-ppc-i386/20100831070010
rsync -av --exclude partial.txt 4.0b5/Darwin_x86_64-gcc3/20100831070403/ 4.0b5/Darwin_x86-gcc3-u-ppc-i386/20100831070010/
mkdir -p 4.0b6/Darwin_x86-gcc3-u-ppc-i386/20100914072643
rsync -av --exclude partial.txt 4.0b6/Darwin_x86_64-gcc3/20100914073111/ 4.0b6/Darwin_x86-gcc3-u-ppc-i386/20100914072643/
mkdir -p 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/{beta,release}test
# setup snippet
cat > 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/releasetest/complete.txt << EOF
version=1
type=complete
url=http://download.mozilla.org/?product=firefox-4.0b8-complete&os=osx64&lang=ja-JP-mac
hashFunction=SHA512
hashValue=66ffd1fbcf11d90680774891dc5b4c57122d0e91e77184a98c58d437f56dbf871203bc449cfb5094daf0cdb1bc8d867c6aa4ec66f4025bf9302172833f282707
size=27373764
build=20101214165004
appv=4.0 Beta 8
extv=4.0b8
detailsUrl=http://www.mozilla.com/ja-JP-mac/firefox/4.0b8/releasenotes/
EOF
sed -e 's!^url=.*!url=http://stage-old.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/update/mac64/ja-JP-mac/firefox-4.0b8.complete.mar!' \
4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/releasetest/complete.txt \
> 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/betatest/complete.txt
# copy to other locations, ja was added at b5
dirs=`find 4.0b{4,5,6,7} -maxdepth 1 -mindepth 1 -type d -name '*Darwin*' ! -wholename '4.0b6/Darwin_x86_64-gcc3' | sort`
for d in $dirs; do
rsync -av 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac $d/*/
done
# fix paths/os
find */Darwin* -type f -exec sed -i -e 's!update/mac64/!update/mac/!' -e 's!os=osx64!os=osx!' {} \;
cd ../../
rsync -a 20101214-Firefox-4.0b8-test-fixed/ 20101214-Firefox-4.0b8-fixed/
cd 20101214-Firefox-4.0b8-fixed/Firefox
find . -depth -name 'betatest' -type d -exec rm -rf {} \;
find . -mindepth 4 -maxdepth 4 -type d -exec mv {}/{releasetest,beta} \;
~/bin/backupsnip 20101214-Firefox-4.0b8-test-fixed ~/bin/pushsnip 20101214-Firefox-4.0b8-test-fixed
# ffxbld@stage cd /pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/update ln -s mac64 mac /pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/ ln -s mac64 mac
Realized that we don't generate snippets for linux64/ja and a typo in the previous script (-wholename '4.0b6/Darwin_x86_64-gcc3' should be -wholename '4.0b7/Darwin_x86_64-gcc3'), generated snippets again:
#cltbld@aus-staging
cd /opt/aus2/snippets/staging/
rsync -a 20101214-Firefox-4.0b8-test/ 20101214-Firefox-4.0b8-test-fixed2/
cd 20101214-Firefox-4.0b8-test-fixed2/Firefox
mkdir -p 3.7a1/Darwin_Universal-gcc3/20100208064157
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a1/Darwin_Universal-gcc3/20100208064157
mkdir -p 3.7a2/Darwin_Universal-gcc3/20100228155422
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a2/Darwin_Universal-gcc3/20100228155422/
mkdir -p 3.7a3/Darwin_Universal-gcc3/20100315065900
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a3/Darwin_Universal-gcc3/20100315065900/
mkdir -p 3.7a4/Darwin_Universal-gcc3/20100407105356
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a4/Darwin_Universal-gcc3/20100407105356/
mkdir -p 3.7a5/Darwin_Universal-gcc3/20100610053455
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a5/Darwin_Universal-gcc3/20100610053455
mkdir -p 4.0b1/Darwin_Universal-gcc3/20100630131607
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 4.0b1/Darwin_Universal-gcc3/20100630131607/
mkdir -p 4.0b2/Darwin_Universal-gcc3/20100720175703
rsync -av --exclude partial.txt 4.0b2/Darwin_x86_64-gcc3/20100720180051/ 4.0b2/Darwin_Universal-gcc3/20100720175703/
mkdir -p 4.0b3/Darwin_Universal-gcc3/20100805181943/
rsync -av --exclude partial.txt 4.0b3/Darwin_x86_64-gcc3/20100805182338/ 4.0b3/Darwin_Universal-gcc3/20100805181943/
mkdir -p 4.0b4/Darwin_Universal-gcc3/20100818121614
rsync -av --exclude partial.txt 4.0b4/Darwin_x86_64-gcc3/20100818121922/ 4.0b4/Darwin_Universal-gcc3/20100818121614/
mkdir -p 4.0b5/Darwin_x86-gcc3-u-ppc-i386/20100831070010
rsync -av --exclude partial.txt 4.0b5/Darwin_x86_64-gcc3/20100831070403/ 4.0b5/Darwin_x86-gcc3-u-ppc-i386/20100831070010/
mkdir -p 4.0b6/Darwin_x86-gcc3-u-ppc-i386/20100914072643
rsync -av --exclude partial.txt 4.0b6/Darwin_x86_64-gcc3/20100914073111/ 4.0b6/Darwin_x86-gcc3-u-ppc-i386/20100914072643/
mkdir -p 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/{beta,release}test
# setup snippet
cat > 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/releasetest/complete.txt << EOF
version=1
type=complete
url=http://download.mozilla.org/?product=firefox-4.0b8-complete&os=osx64&lang=ja-JP-mac
hashFunction=SHA512
hashValue=66ffd1fbcf11d90680774891dc5b4c57122d0e91e77184a98c58d437f56dbf871203bc449cfb5094daf0cdb1bc8d867c6aa4ec66f4025bf9302172833f282707
size=27373764
build=20101214165004
appv=4.0 Beta 8
extv=4.0b8
detailsUrl=http://www.mozilla.com/ja-JP-mac/firefox/4.0b8/releasenotes/
EOF
sed -e 's!^url=.*!url=http://stage-old.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/update/mac64/ja-JP-mac/firefox-4.0b8.complete.mar!' \
4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/releasetest/complete.txt \
> 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/betatest/complete.txt
# copy to other locations, ja was added at b5
dirs=`find 4.0b{4,5,6,7} -maxdepth 1 -mindepth 1 -type d -name '*Darwin*' ! -wholename '4.0b7/Darwin_x86_64-gcc3' | sort`
for d in $dirs; do
rsync -av 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac $d/*/
done
# fix paths/os
find */Darwin* -type f -exec sed -i -e 's!update/mac64/!update/mac/!' -e 's!os=osx64!os=osx!' {} \;
mkdir -p 4.0b7/Linux_x86_64-gcc3/20101104131204/ja/{beta,release}test
# setup snippet
cat > 4.0b7/Linux_x86_64-gcc3/20101104131204/ja/releasetest/complete.txt << EOF
version=1
type=complete
url=http://download.mozilla.org/?product=firefox-4.0b8-complete&os=linux64&lang=ja
hashFunction=SHA512
hashValue=0f3f22fc577bb06a01c4ab011448fb97bda99ede7ad4af6fbea8b89a819d2c4decf67d6bf9890ba533b395d2d2133eb0c60e7e4048f6a5151dd8e53dba0178a7
size=15999286
build=20101214180840
appv=4.0 Beta 8
extv=4.0b8
detailsUrl=http://www.mozilla.com/ja/firefox/4.0b8/releasenotes/
EOF
sed -e 's!^url=.*!url=http://stage-old.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/update/linux-x86_64/ja/firefox-4.0b8.complete.mar!' \
4.0b7/Linux_x86_64-gcc3/20101104131204/ja/releasetest/complete.txt \
> 4.0b7/Linux_x86_64-gcc3/20101104131204/ja/betatest/complete.txt
# copy to other locations, ja was added at b5
dirs=`find 4.0b{4,5,6,7} -maxdepth 1 -mindepth 1 -type d -name '*Linux_x86_64-gcc3*' ! -wholename '4.0b7/Linux_x86_64-gcc3' | sort`
for d in $dirs; do
rsync -av 4.0b7/Linux_x86_64-gcc3/20101104131204/ja $d/*/
done
cd ../../
rsync -a 20101214-Firefox-4.0b8-test-fixed2/ 20101214-Firefox-4.0b8-fixed2/
cd 20101214-Firefox-4.0b8-fixed2/Firefox
find . -depth -name 'betatest' -type d -exec rm -rf {} \;
find . -mindepth 4 -maxdepth 4 -type d -exec mv {}/{releasetest,beta} \;
~/bin/backupsnip 20101214-Firefox-4.0b8-test-fixed2
~/bin/pushsnip 20101214-Firefox-4.0b8-test-fixed2
TODO: add ja/ja-JP-mac to update verify configs fpr linux64 and mac64 and rerun them.
Offering completes as partials for win32
Due to signing changes that landed just before b7, win32 complete MARs for b7 were broken by signing. This means that win32 partial MARs for b8 will be broken until we regenerate/sign complete MARs for b7, and then regenerate the partial MARs for b8.
In the interim, we'll be offering the win32 complete MARs as partials. Other platforms are not affected.
#cltbld@aus2-staging cd /opt/aus2/incoming/3/Firefox/4.0b7/WINNT_x86-msvc/20101104142426 time ~coop/completes_as_partials.bash | tee ~/4.0b7_completes_as_partials.log Updated 114 partial snippets with completes real 0m9.897s user 0m0.192s sys 0m1.329s
Root cause of win32 updates issues
Starting with bug 554321, we introduce a patch that lets us sign partner-repacks at the same time as everything else (rather than the previous practice of signing locale repacks, then generating partner-repacks off of signed builds and resigning). This patch was landed shortly before the tools repo was tagged for beta 7. Beta 7 proceeded as normal, using this signing patch and passed verify_signatures, and update_verify.
However, we later hit problems in staging before the 3.5/3.6 releases where the signed MARs were inconsistent with the installers due to a problem with how the list of files to be signed was sorted in the partner-repack patch, causing MARs to be signed ahead of installers, which tripped up our caching. The issue was fixed in bug 613320. The problems discovered in that bug are confirmed *not* affecting the stable branches, so the next 3.5/3.6 releases should not encounter these problems.
Since we could generate valid partials from the busted b7 MARs, for this release, we have switched the partial update snippet to point to the complete MAR which applies correctly regardless of the busted beta 7 MARs. Since the signing issue has already been fixed, this means that going forward, the beta 8->beta 9 (and onwards) partials should be able to be generated without issues.
The reason that this slipped through in the first place was because I only tested for consistency between locales when testing the signed builds, and did not run update_verify on builds signed with the original partner-repack patch. As part of the signing changes that have landed, we now verify the checksums of the MAR and installer internals as part of verify_signatures, which will catch the case where we have inconsistent checksums on any installer/MAR internals.
Looking forward, the best way to decrease the possibility of things like this happening in the future would probably be to expand the coverage provided by update_verify (we should have a way to find out about inconsistencies in applying updates before we try to generate updates for the next release). We already have the checksum verification steps in the signing code now to guard against the specific cases of checksum differences between the MARs and installers.
Post repack 3 snippet fixes
#cltbld@aus-staging
cd /opt/aus2/snippets/staging/
rsync -a 20101214-Firefox-4.0b8-test/ 20101214-Firefox-4.0b8-test-fixed3/
cd 20101214-Firefox-4.0b8-test-fixed3/Firefox
# Fix mac platform name
mkdir -p 3.7a1/Darwin_Universal-gcc3/20100208064157
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a1/Darwin_Universal-gcc3/20100208064157
mkdir -p 3.7a2/Darwin_Universal-gcc3/20100228155422
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a2/Darwin_Universal-gcc3/20100228155422/
mkdir -p 3.7a3/Darwin_Universal-gcc3/20100315065900
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a3/Darwin_Universal-gcc3/20100315065900/
mkdir -p 3.7a4/Darwin_Universal-gcc3/20100407105356
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a4/Darwin_Universal-gcc3/20100407105356/
mkdir -p 3.7a5/Darwin_Universal-gcc3/20100610053455
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a5/Darwin_Universal-gcc3/20100610053455
mkdir -p 4.0b1/Darwin_Universal-gcc3/20100630131607
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 4.0b1/Darwin_Universal-gcc3/20100630131607/
mkdir -p 4.0b2/Darwin_Universal-gcc3/20100720175703
rsync -av --exclude partial.txt 4.0b2/Darwin_x86_64-gcc3/20100720180051/ 4.0b2/Darwin_Universal-gcc3/20100720175703/
mkdir -p 4.0b3/Darwin_Universal-gcc3/20100805181943/
rsync -av --exclude partial.txt 4.0b3/Darwin_x86_64-gcc3/20100805182338/ 4.0b3/Darwin_Universal-gcc3/20100805181943/
mkdir -p 4.0b4/Darwin_Universal-gcc3/20100818121614
rsync -av --exclude partial.txt 4.0b4/Darwin_x86_64-gcc3/20100818121922/ 4.0b4/Darwin_Universal-gcc3/20100818121614/
mkdir -p 4.0b5/Darwin_x86-gcc3-u-ppc-i386/20100831070010
rsync -av --exclude partial.txt 4.0b5/Darwin_x86_64-gcc3/20100831070403/ 4.0b5/Darwin_x86-gcc3-u-ppc-i386/20100831070010/
mkdir -p 4.0b6/Darwin_x86-gcc3-u-ppc-i386/20100914072643
rsync -av --exclude partial.txt 4.0b6/Darwin_x86_64-gcc3/20100914073111/ 4.0b6/Darwin_x86-gcc3-u-ppc-i386/20100914072643/
# Generate ja-JP-mac snippet
mkdir -p 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/{beta,release}test
cat > 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/releasetest/complete.txt << EOF
version=1
type=complete
url=http://download.mozilla.org/?product=firefox-4.0b8-complete&os=osx64&lang=ja-JP-mac
hashFunction=SHA512
hashValue=fea614985261dc8e0cae6f05d19a55375b49f0560222b2892d703ad7b555a6fe2ad884a3fd8e876ca87ab5ccd1cdf338932af13977b2c61173815e9eeb6a0c5b
size=27373799
build=20101214165004
appv=4.0 Beta 8
extv=4.0b8
detailsUrl=http://www.mozilla.com/ja-JP-mac/firefox/4.0b8/releasenotes/
EOF
sed -e 's!^url=.*!url=http://stage-old.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/update/mac64/ja-JP-mac/firefox-4.0b8.complete.mar!' \
4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/releasetest/complete.txt \
> 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/betatest/complete.txt
# copy to other locations, ja was added at b5
dirs=`find 4.0b{4,5,6,7} -maxdepth 1 -mindepth 1 -type d -name '*Darwin*' ! -wholename '4.0b7/Darwin_x86_64-gcc3' | sort`
for d in $dirs; do
rsync -av 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac $d/*/
done
# fix paths/os
find */Darwin* -type f -exec sed -i -e 's!update/mac64/!update/mac/!' -e 's!os=osx64!os=osx!' {} \;
# Generate ja linux64 snippet
mkdir -p 4.0b7/Linux_x86_64-gcc3/20101104131204/ja/{beta,release}test
# setup snippet
cat > 4.0b7/Linux_x86_64-gcc3/20101104131204/ja/releasetest/complete.txt << EOF
version=1
type=complete
url=http://download.mozilla.org/?product=firefox-4.0b8-complete&os=linux64&lang=ja
hashFunction=SHA512
hashValue=482d120eebecccafce5e870e8a9e3fc1295d964960d21523c8cf7a1079c6a55d830b39eaa992f36e372f19998e920ce601a866849f9001a8aac5bafdd14a41e1
size=15996568
build=20101214180840
appv=4.0 Beta 8
extv=4.0b8
detailsUrl=http://www.mozilla.com/ja/firefox/4.0b8/releasenotes/
EOF
sed -e 's!^url=.*!url=http://stage-old.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/update/linux-x86_64/ja/firefox-4.0b8.complete.mar!' \
4.0b7/Linux_x86_64-gcc3/20101104131204/ja/releasetest/complete.txt \
> 4.0b7/Linux_x86_64-gcc3/20101104131204/ja/betatest/complete.txt
# copy to other locations, ja was added at b5
dirs=`find 4.0b{4,5,6,7} -maxdepth 1 -mindepth 1 -type d -name '*Linux_x86_64-gcc3*' ! -wholename '4.0b7/Linux_x86_64-gcc3' | sort`
for d in $dirs; do
rsync -av 4.0b7/Linux_x86_64-gcc3/20101104131204/ja $d/*/
done
# win beta7 partial -> complete
dirs=`find 4.0b7/WINNT_x86-msvc/20101104142426 -maxdepth 1 -mindepth 1 -type d | sort`
for d in $dirs; do
rsync -av $d/betatest/complete.txt $d/betatest/partial.txt
rsync -av $d/releasetest/complete.txt $d/releasetest/partial.txt
sed -i 's/type=complete/type=partial/' $d/{beta,release}test/partial.txt
done
cd ../../
rsync -a 20101214-Firefox-4.0b8-test-fixed3/ 20101214-Firefox-4.0b8-fixed3/
cd 20101214-Firefox-4.0b8-fixed3/Firefox
find . -depth -name 'betatest' -type d -exec rm -rf {} \;
find . -mindepth 4 -maxdepth 4 -type d -exec mv {}/{releasetest,beta} \;
~/bin/backupsnip 20101214-Firefox-4.0b8-test-fixed3
~/bin/pushsnip 20101214-Firefox-4.0b8-test-fixed3
Fix "ja" locale snippet type
cd /opt/aus2/snippets/staging/
rsync -a 20101214-Firefox-4.0b8-test-fixed3/ 20101214-Firefox-4.0b8-test-fixed4/
cd 20101214-Firefox-4.0b8-test-fixed4/Firefox
for f in */*Darwin*/*/ja*/*/complete.txt */*Linux_x86_64-gcc3*/*/ja*/*/complete.txt; do
t=$(dirname $f)/partial.txt
rsync -av $f $t
sed -i 's/type=complete/type=partial/' $t
done
~/bin/backupsnip 20101214-Firefox-4.0b8-test-fixed4
~/bin/pushsnip 20101214-Firefox-4.0b8-test-fixed4
cd ../../
rsync -a 20101214-Firefox-4.0b8-test-fixed4/ 20101214-Firefox-4.0b8-fixed4/
cd 20101214-Firefox-4.0b8-fixed4/Firefox
find . -depth -name 'betatest' -type d -exec rm -rf {} \;
find . -mindepth 4 -maxdepth 4 -type d -exec mv {}/{releasetest,beta} \;
Add ja-JP-mac and swap to mac in dir names
Fix 4.0b7 -> 4.0b8 Mac updates
Patcher generates Mac updates using the "Darwin_x86_64-gcc3" OS label, but incoming clients use "Darwin_x86_64-gcc3-u-i386-x86_64" and "Darwin_x86-gcc3-u-i386-x86_64" in their requests. To fix, we added symlinks:
cd /opt/aus2/incoming/3/Firefox/4.0b7 ln -s Darwin_x86_64-gcc3 Darwin_x86_64-gcc3-u-i386-x86_64 ln -s Darwin_x86_64-gcc3 Darwin_x86-gcc3-u-i386-x86_64
Update Bouncer
Used "Force Build" on "release-mozilla-central-bouncer_submitter" builder. No problems with it.
Naming fixup
The buildbot setup leaves us with mac64 directories on the ftp server, and an expectation to use os=osx64 in bouncer. The mozilla.com maintainers would really rather not change from os=osx, so we're making everything match what we did for 4.0b6.
Rename on ftp server
Fix the file locations, but leave a symlink so that betatest keeps working for QA, and updates will work for b9.
# ffxbld@stage cd /pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1 rm mac && mv mac64 mac && ln -s mac mac64 cd update rm mac && mv mac64 mac && ln -s mac mac64
Fix-up Snippets
Done above, as part of the "-fixed" sets of snippets.
Fix up *SUMS
On signing box
cd ~/signing-work/firefox-4.0b8/ mkdir signed-build1-redo-SUMS cp signed-build1/*SUMS signed-build1-redo-SUMS/ cd signed-build1-redo-SUMS sed -i -e 's/mac64/mac/' *SUMS ../signing/sign-files . rsync -av -e "ssh -i $HOME/.ssh/ffxbld_dsa" * \ ffxbld@stage.mozilla.org:/pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/
Adjust Bouncer Entries
Manually changed all "osx64" entries to "osx" platform, and edited locations, changing "mac64" to "mac".
Update Verify
- landed a patch to enable ja/ja-JP-mac update verify checks.
- linux64 update verify failure for "ja". The current code expects that 4.0b7-4.0b8 snippet should be type="partial", while AUS2 returns a snippet with type="complete" for "ja". patcher2 doesn't generate partial mars for 64bit platforms, because they are not in shipped-locales file. I checked it manually: downloaded 4.0b7, switched to betatest channel, checked for updates, applied updates, checked version.
- The same issue for jp-JP-mac. No mac around to test using the same scenario, but I believe that it'll work. If someone could check this, I'd be glad.
- update verify of win32 went red. Update logs look good, no update failure. The errors were produced by file size sanity check, which tries to compare partial and complete mar sizes and fails if the partial is bigger than the complete mar. This happens only for versions before b7 (N-2) (we don't run full check for them, just sanity check). I suspect that the file size sanity check part is buggy, because we don't provide partial mars for versions N-2 and the snippets say the same.
Final checks before push and virus check
bug 540598 adds a new pre_push_checks builder which should be run with at least 2 properties set: revision and release_config. Web form submitted with the following fields set:
- branch: mozilla-central
- revision: FIREFOX_4_0b8_RELEASE
- release_config: mozilla/release-firefox-mozilla-2.0.py
First run failed because the scripts were chmod 644. chmoded to 755, commited, tagged as FIREFOX_4_0b8_BUILD1 and FIREFOX_4_0b8_RELEASE. Second run passed.
Socorro product data updates
Push to mirrors
bug 540598 adds a new push_to_mirrors builder which should be run with at least 2 properties set: revision and release_config. Web form submitted with the following fields set:
- branch: mozilla-central
- revision: FIREFOX_4_0b8_RELEASE
- release_config: mozilla/release-firefox-mozilla-2.0.py
Final verify
Failed for ja linux64 and ja-JP-mac. As in case of update verify, final verification expects partial updates for b7, but the current patcher2.pl doesn't generate partials for ja+64bit. Can be ignored.
Push snippets to beta channel
As cltbld@aus2-staging:
cd /opt/aus2/snippets/staging ~/bin/backupsnip 20101214-Firefox-4.0b8-fixed4 ... cd /opt/aus2/snippets/staging time ~/bin/pushsnip 20101214-Firefox-4.0b8-fixed4 real 2m0.066s user 0m0.219s sys 0m3.201s
Send mail to metrics
Sent mail to metrics@mozilla.com with the following contents:
Firefox,firefox,firefox,4.0b8,4.0 Firefox,firefox,firefox,4.0b9pre,4.0
Release
added rsyncd exclude entries for 4.0b6 and 4.0b7:
# cltbld@stage.mozilla.org grep '4.0b\(6\|7\)' /pub/mozilla.org/zz/rsyncd-mozilla-releases.exclude - firefox/releases/4.0b6 - firefox/releases/4.0b7
