Releases/Firefox 4.0b8/BuildNotes: Difference between revisions
< Releases | Firefox 4.0b8
Jump to navigation
Jump to search
| Line 399: | Line 399: | ||
* [http://people.mozilla.org/~coop/4.0b7_completes_as_partials.log output log] | * [http://people.mozilla.org/~coop/4.0b7_completes_as_partials.log output log] | ||
==== Signing issues ==== | |||
Starting with {{Bug|554321}}, we introduce a patch that lets us sign partner-repacks at the same time as everything else (rather than the previous practice of signing locale repacks, then generating partner-repacks off of signed builds and resigning). This patch was landed shortly before the tools repo was tagged for beta 7. Beta 7 proceeded as normal, using this signing patch and passed verify_signatures, and update_verify. | |||
However, we later hit problems in staging before the 3.5/3.6 releases where the signed MARs were inconsistent with the installers due to a problem with how the list of files to be signed was sorted in the partner-repack patch, causing MARs to be signed ahead of installers, which tripped up our caching. The issue was fixed in {{bug|613320}}. The problems discovered in that bug are confirmed *not* affecting the stable branches, so the next 3.5/3.6 releases should not encounter these problems. | |||
Since we could generate valid partials from the busted b7 MARs, for this release, we have switched the partial update snippet to point to the complete MAR which applies correctly regardless of the busted beta 7 MARs. Since the signing issue has already been fixed, this means that going forward, the beta 8->beta 9 (and onwards) partials should be able to be generated without issues. | |||
The reason that this slipped through in the first place was because I only tested for consistency between locales when testing the signed builds, and did not run update_verify on builds signed with the original partner-repack patch. | |||
As part of the signing changes that have landed, we now verify the checksums of the MAR and installer internals as part of verify_signatures, which will catch the case where we have inconsistent checksums on any installer/MAR internals. | |||
Looking forward, the best way to decrease the possibility of things like this happening in the future would probably be to expand the coverage provided by update_verify (we should have a way to find out about inconsistencies in applying updates before we try to generate updates for the next release). We already have the checksum verification steps in the signing code now to guard against the specific cases of checksum differences between the MARs and installers. | |||
=== Post repack 3 snippet fixes === | === Post repack 3 snippet fixes === | ||
Revision as of 00:58, 17 December 2010
Notes About Releasing
Please update the Release:Primer for future releases (bug fixes, changes to automation) as needed
Build Engineers
rail - bug 615555
Signed-off Revision(s)
- Build 1: abe884259481
L10N changesets
- wget https://l10n-stage-sj.mozilla.org/shipping/l10n-changesets?ms=fx4.0b8
- hit "ship it" from FF4 beta8 page
Tags
| Build # | Tag | Changeset |
| 1 | GECKO20b8_20101214_RELBRANCH | abe884259481 |
| FIREFOX_4_0b8_BUILD1/FIREFOX_4_0b8_RELEASE | d78f9cb65e91 |
Build data
| Build # | Type | Build ID | Build machine | Time to build |
| 1 | Linux [compressed .tar.bz2] | |||
| Linux64 [compressed .tar.bz2] | ||||
| Macosx64 [.dmg] | ||||
| Win32 [.exe/.zip] |
Bugs hit
Build 1
Done on pm01
- No clobberer necessary because this is the first 0.8.0 release on this branch.
- Landed configs
- tagged buildbot-configs (production branch), buildbotcustom (production-0.8 branch), and tools (default branch) with FIREFOX_4_0b8_BUILD1 and FIREFOX_4_0b8_RELEASE
- updated /tools/buildbotcustom/buildbotcustom, /builds/buildbot/configs, and /builds/buildbot/builder_master/tools to latest
- reconfiged master
- Left tree open, 0.8.0 automation doesn't require that it closes
- kicked off automation with:
PYTHONPATH=/builds/buildbot/builder_master1:/builds/buildbot/builder_master1/tools/lib/python python /builds/buildbot/builder_master1/tools/buildbot-helpers/release_sanity.py -u rail -V 4.0b8 --branch mozilla-central --build-number 1 --dryrun localhost:9010 PYTHONPATH=/builds/buildbot/builder_master1:/builds/buildbot/builder_master1/tools/lib/python python /builds/buildbot/builder_master1/tools/buildbot-helpers/release_sanity.py -u rail -V 4.0b8 --branch mozilla-central --build-number 1 localhost:9010
Setup
Tag
The first sendchage was without script_repo_revision:FIREFOX_4_0b8_RELEASE set, and failed in the beginning. The second one sent by release_sanity.py worked fine.
Source
No problems
Build/Repack
Failed l10n repacks: en-ZA, nso, pt-BR
These locales had missing files and compare-locales doesn't handle this case.
Filed bug 619258
- clobbered all slaves used for repacks (all chunks, all platforms)
- removed all locales except of en-US from ftp (installers+complete mars)
- landed the patch and tagged the revision with FIREFOX_4_0b8_BUILD1 and FIREFOX_4_0b8_RELEASE
- to make sure if the patch works, rebuilt 2nd (failed previously) chunk for linux.
- win32 repacks didn't work. Landed patch
Later, we noticed that "ja" was missing from 64-bit Linux and "ja-JP-mac" was missing from 64-bit Mac. To fix, did the following manually on moz2-linux64-slave01:
export HG_SHARE_BASE_DIR=/builds/hg-shared export BUILDBOTCUSTOM=http://hg.mozilla.org/build/buildbotcustom export BUILDBOT_CONFIGS=http://hg.mozilla.org/build/buildbot-configs export CC=/tools/gcc/bin/gcc export CXX=/tools/gcc/bin/g++ export PROPERTIES_FILE=buildprops.json cd /builds/slave/rel-cen-lnx64-rpk-1/build /tools/python/bin/python /builds/slave/rel-cen-lnx64-rpk-1/build/scripts/scripts/l10n/create-release-repacks.py -c mozilla/production_config.py -r mozilla/release-firefox-mozilla-central.py -b http://hg.mozilla.org/build/buildbot-configs -t FIREFOX_4_0b8_RELEASE -p linux64 -l ja
And the following on moz2-darwin10-slave13:
export BUILDBOTCUSTOM=http://hg.mozilla.org/build/buildbotcustom export BUILDBOT_CONFIGS=http://hg.mozilla.org/build/buildbot-configs export CC=/tools/gcc/bin/gcc export CXX=/tools/gcc/bin/g++ export PROPERTIES_FILE=buildprops.json cd /builds/slave/rel-cen-osx64-rpk-2/build /tools/python/bin/python /builds/slave/rel-cen-osx64-rpk-2/build/scripts/scripts/l10n/create-release-repacks.py -c mozilla/production_config.py -r mozilla/release-firefox-mozilla-central.py -b http://hg.mozilla.org/build/buildbot-configs -t FIREFOX_4_0b8_RELEASE -p macosx64 -l ja-JP-mac
Filed bug 619522 on the issue.
Repack 3
Due to bug 619664 we had to resping l10n repacks.
Unittests
Results
Signing
To speed up signing, ran the following prior to all repacks being completed:
PRODUCT=firefox
VERSION=4.0b8
BUILD=1
TAG=FIREFOX_4_0b8_RELEASE
REPO='mozilla-central'
EMAIL=rail@mozilla.com
KEYDIR=d:/2010-keys
cd ~/hg-tools
hg pull && hg up -r ${TAG}
mkdir ~/signing-work/${PRODUCT}-${VERSION}
cd ~/signing-work/${PRODUCT}-${VERSION}
cp ~/hg-tools/release/signing/* .
make df verify-gpg-passphrase setup verify-signcode-passphrase PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
# download target run multiple times to get things as they complete
make download PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
Once all the l10n and partner repacks finished, ran the following in the same window to do the signing:
# Need to download one more time to make sure we've got everything
make download PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
make stubs sign2 stage verify verify-sigs fake-upload PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
- verify-asc (last step of verify, before fake-upload) failed due to 3 *.checksum files:
./linux-i686/en-US/firefox-4.0b8.checksums ./linux-x86_64/en-US/firefox-4.0b8.checksums ./mac64/en-US/Firefox 4.0 Beta 8.checksums
- removed the files manually and rerun fake-upload
- filed bug 619572 with a patch
Finally:
make postsign PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL}
Got "Signature Verification Succeeded" mail later.
- We had to respin l10n repacks and sign the files again
- removed signed files from stage
- rerun the signing step w/o any problem
L10N Verify
- A lot of harmless warnings looking like errors: bug 619066
Updates
First attempt failed during unpacking mar
Extracting /builds/slave/rel-cen-updates/build/temp/firefox/4.0b7/ftp/firefox-4.0b7.he.win32.complete.mar to /builds/slave/rel-cen-updates/build/tmpeWytLO-fastmode/195/from IN SHUTDOWN... Failed to remove patcher2.pid: No such file or directory FAIL shell call timed out after 10800 seconds at patcher2.pl line 1619. program finished with exit code 2 elapsedTime=11041.982708
dmesg on linux-ix-slave13:
ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x0) ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x12) ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x2) ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x0) ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x0) ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x1) ata1: spurious interrupt (irq_stat 0x8 active_tag -84148995 sactive 0x0)
Gracefully shutdown linux-ix-slave13, buildbot.tac -> buildbot.tac.off. Filed bug 619624
Special case for old mac universal builds
See Releases/Firefox_4.0b7/BuildNotes#Updates for the details.
#cltbld@aus-staging
cd /opt/aus2/snippets/staging/
cp -r 20101214-Firefox-4.0b8-test 20101214-Firefox-4.0b8-test-fixed
cd 20101214-Firefox-4.0b8-test-fixed/Firefox
mkdir -p 3.7a1/Darwin_Universal-gcc3/20100208064157
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a1/Darwin_Universal-gcc3/20100208064157
mkdir -p 3.7a2/Darwin_Universal-gcc3/20100228155422
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a2/Darwin_Universal-gcc3/20100228155422/
mkdir -p 3.7a3/Darwin_Universal-gcc3/20100315065900
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a3/Darwin_Universal-gcc3/20100315065900/
mkdir -p 3.7a4/Darwin_Universal-gcc3/20100407105356
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a4/Darwin_Universal-gcc3/20100407105356/
mkdir -p 3.7a5/Darwin_Universal-gcc3/20100610053455
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a5/Darwin_Universal-gcc3/20100610053455
mkdir -p 4.0b1/Darwin_Universal-gcc3/20100630131607
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 4.0b1/Darwin_Universal-gcc3/20100630131607/
mkdir -p 4.0b2/Darwin_Universal-gcc3/20100720175703
rsync -av --exclude partial.txt 4.0b2/Darwin_x86_64-gcc3/20100720180051/ 4.0b2/Darwin_Universal-gcc3/20100720175703/
mkdir -p 4.0b3/Darwin_Universal-gcc3/20100805181943/
rsync -av --exclude partial.txt 4.0b3/Darwin_x86_64-gcc3/20100805182338/ 4.0b3/Darwin_Universal-gcc3/20100805181943/
mkdir -p 4.0b4/Darwin_Universal-gcc3/20100818121614
rsync -av --exclude partial.txt 4.0b4/Darwin_x86_64-gcc3/20100818121922/ 4.0b4/Darwin_Universal-gcc3/20100818121614/
mkdir -p 4.0b5/Darwin_x86-gcc3-u-ppc-i386/20100831070010
rsync -av --exclude partial.txt 4.0b5/Darwin_x86_64-gcc3/20100831070403/ 4.0b5/Darwin_x86-gcc3-u-ppc-i386/20100831070010/
mkdir -p 4.0b6/Darwin_x86-gcc3-u-ppc-i386/20100914072643
rsync -av --exclude partial.txt 4.0b6/Darwin_x86_64-gcc3/20100914073111/ 4.0b6/Darwin_x86-gcc3-u-ppc-i386/20100914072643/
mkdir -p 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/{beta,release}test
# setup snippet
cat > 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/releasetest/complete.txt << EOF
version=1
type=complete
url=http://download.mozilla.org/?product=firefox-4.0b8-complete&os=osx64&lang=ja-JP-mac
hashFunction=SHA512
hashValue=66ffd1fbcf11d90680774891dc5b4c57122d0e91e77184a98c58d437f56dbf871203bc449cfb5094daf0cdb1bc8d867c6aa4ec66f4025bf9302172833f282707
size=27373764
build=20101214165004
appv=4.0 Beta 8
extv=4.0b8
detailsUrl=http://www.mozilla.com/ja-JP-mac/firefox/4.0b8/releasenotes/
EOF
sed -e 's!^url=.*!url=http://stage-old.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/update/mac64/ja-JP-mac/firefox-4.0b8.complete.mar!' \
4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/releasetest/complete.txt \
> 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/betatest/complete.txt
# copy to other locations, ja was added at b5
dirs=`find 4.0b{4,5,6,7} -maxdepth 1 -mindepth 1 -type d -name '*Darwin*' ! -wholename '4.0b6/Darwin_x86_64-gcc3' | sort`
for d in $dirs; do
rsync -av 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac $d/*/
done
# fix paths/os
find */Darwin* -type f -exec sed -i -e 's!update/mac64/!update/mac/!' -e 's!os=osx64!os=osx!' {} \;
cd ../../
rsync -a 20101214-Firefox-4.0b8-test-fixed/ 20101214-Firefox-4.0b8-fixed/
cd 20101214-Firefox-4.0b8-fixed/Firefox
find . -depth -name 'betatest' -type d -exec rm -rf {} \;
find . -mindepth 4 -maxdepth 4 -type d -exec mv {}/{releasetest,beta} \;
~/bin/backupsnip 20101214-Firefox-4.0b8-test-fixed ~/bin/pushsnip 20101214-Firefox-4.0b8-test-fixed
# ffxbld@stage cd /pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/update ln -s mac64 mac /pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/ ln -s mac64 mac
Realized that we don't generate snippets for linux64/ja and a typo in the previous script (-wholename '4.0b6/Darwin_x86_64-gcc3' should be -wholename '4.0b7/Darwin_x86_64-gcc3'), generated snippets again:
#cltbld@aus-staging
cd /opt/aus2/snippets/staging/
rsync -a 20101214-Firefox-4.0b8-test/ 20101214-Firefox-4.0b8-test-fixed2/
cd 20101214-Firefox-4.0b8-test-fixed2/Firefox
mkdir -p 3.7a1/Darwin_Universal-gcc3/20100208064157
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a1/Darwin_Universal-gcc3/20100208064157
mkdir -p 3.7a2/Darwin_Universal-gcc3/20100228155422
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a2/Darwin_Universal-gcc3/20100228155422/
mkdir -p 3.7a3/Darwin_Universal-gcc3/20100315065900
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a3/Darwin_Universal-gcc3/20100315065900/
mkdir -p 3.7a4/Darwin_Universal-gcc3/20100407105356
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a4/Darwin_Universal-gcc3/20100407105356/
mkdir -p 3.7a5/Darwin_Universal-gcc3/20100610053455
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a5/Darwin_Universal-gcc3/20100610053455
mkdir -p 4.0b1/Darwin_Universal-gcc3/20100630131607
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 4.0b1/Darwin_Universal-gcc3/20100630131607/
mkdir -p 4.0b2/Darwin_Universal-gcc3/20100720175703
rsync -av --exclude partial.txt 4.0b2/Darwin_x86_64-gcc3/20100720180051/ 4.0b2/Darwin_Universal-gcc3/20100720175703/
mkdir -p 4.0b3/Darwin_Universal-gcc3/20100805181943/
rsync -av --exclude partial.txt 4.0b3/Darwin_x86_64-gcc3/20100805182338/ 4.0b3/Darwin_Universal-gcc3/20100805181943/
mkdir -p 4.0b4/Darwin_Universal-gcc3/20100818121614
rsync -av --exclude partial.txt 4.0b4/Darwin_x86_64-gcc3/20100818121922/ 4.0b4/Darwin_Universal-gcc3/20100818121614/
mkdir -p 4.0b5/Darwin_x86-gcc3-u-ppc-i386/20100831070010
rsync -av --exclude partial.txt 4.0b5/Darwin_x86_64-gcc3/20100831070403/ 4.0b5/Darwin_x86-gcc3-u-ppc-i386/20100831070010/
mkdir -p 4.0b6/Darwin_x86-gcc3-u-ppc-i386/20100914072643
rsync -av --exclude partial.txt 4.0b6/Darwin_x86_64-gcc3/20100914073111/ 4.0b6/Darwin_x86-gcc3-u-ppc-i386/20100914072643/
mkdir -p 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/{beta,release}test
# setup snippet
cat > 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/releasetest/complete.txt << EOF
version=1
type=complete
url=http://download.mozilla.org/?product=firefox-4.0b8-complete&os=osx64&lang=ja-JP-mac
hashFunction=SHA512
hashValue=66ffd1fbcf11d90680774891dc5b4c57122d0e91e77184a98c58d437f56dbf871203bc449cfb5094daf0cdb1bc8d867c6aa4ec66f4025bf9302172833f282707
size=27373764
build=20101214165004
appv=4.0 Beta 8
extv=4.0b8
detailsUrl=http://www.mozilla.com/ja-JP-mac/firefox/4.0b8/releasenotes/
EOF
sed -e 's!^url=.*!url=http://stage-old.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/update/mac64/ja-JP-mac/firefox-4.0b8.complete.mar!' \
4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/releasetest/complete.txt \
> 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/betatest/complete.txt
# copy to other locations, ja was added at b5
dirs=`find 4.0b{4,5,6,7} -maxdepth 1 -mindepth 1 -type d -name '*Darwin*' ! -wholename '4.0b7/Darwin_x86_64-gcc3' | sort`
for d in $dirs; do
rsync -av 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac $d/*/
done
# fix paths/os
find */Darwin* -type f -exec sed -i -e 's!update/mac64/!update/mac/!' -e 's!os=osx64!os=osx!' {} \;
mkdir -p 4.0b7/Linux_x86_64-gcc3/20101104131204/ja/{beta,release}test
# setup snippet
cat > 4.0b7/Linux_x86_64-gcc3/20101104131204/ja/releasetest/complete.txt << EOF
version=1
type=complete
url=http://download.mozilla.org/?product=firefox-4.0b8-complete&os=linux64&lang=ja
hashFunction=SHA512
hashValue=0f3f22fc577bb06a01c4ab011448fb97bda99ede7ad4af6fbea8b89a819d2c4decf67d6bf9890ba533b395d2d2133eb0c60e7e4048f6a5151dd8e53dba0178a7
size=15999286
build=20101214180840
appv=4.0 Beta 8
extv=4.0b8
detailsUrl=http://www.mozilla.com/ja/firefox/4.0b8/releasenotes/
EOF
sed -e 's!^url=.*!url=http://stage-old.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/update/linux-x86_64/ja/firefox-4.0b8.complete.mar!' \
4.0b7/Linux_x86_64-gcc3/20101104131204/ja/releasetest/complete.txt \
> 4.0b7/Linux_x86_64-gcc3/20101104131204/ja/betatest/complete.txt
# copy to other locations, ja was added at b5
dirs=`find 4.0b{4,5,6,7} -maxdepth 1 -mindepth 1 -type d -name '*Linux_x86_64-gcc3*' ! -wholename '4.0b7/Linux_x86_64-gcc3' | sort`
for d in $dirs; do
rsync -av 4.0b7/Linux_x86_64-gcc3/20101104131204/ja $d/*/
done
cd ../../
rsync -a 20101214-Firefox-4.0b8-test-fixed2/ 20101214-Firefox-4.0b8-fixed2/
cd 20101214-Firefox-4.0b8-fixed2/Firefox
find . -depth -name 'betatest' -type d -exec rm -rf {} \;
find . -mindepth 4 -maxdepth 4 -type d -exec mv {}/{releasetest,beta} \;
~/bin/backupsnip 20101214-Firefox-4.0b8-test-fixed2
~/bin/pushsnip 20101214-Firefox-4.0b8-test-fixed2
TODO: add ja/ja-JP-mac to update verify configs fpr linux64 and mac64 and rerun them.
Offering completes as partials for win32
Due to signing changes that landed just before b7, win32 complete MARs for b7 were broken by signing. This means that win32 partial MARs for b8 will be broken until we regenerate/sign complete MARs for b7, and then regenerate the partial MARs for b8.
In the interim, we'll be offering the win32 complete MARs as partials. Other platforms are not affected.
#cltbld@aus2-staging cd /opt/aus2/incoming/3/Firefox/4.0b7/WINNT_x86-msvc/20101104142426 time ~coop/completes_as_partials.bash | tee ~/4.0b7_completes_as_partials.log Updated 114 partial snippets with completes real 0m9.897s user 0m0.192s sys 0m1.329s
Signing issues
Starting with bug 554321, we introduce a patch that lets us sign partner-repacks at the same time as everything else (rather than the previous practice of signing locale repacks, then generating partner-repacks off of signed builds and resigning). This patch was landed shortly before the tools repo was tagged for beta 7. Beta 7 proceeded as normal, using this signing patch and passed verify_signatures, and update_verify.
However, we later hit problems in staging before the 3.5/3.6 releases where the signed MARs were inconsistent with the installers due to a problem with how the list of files to be signed was sorted in the partner-repack patch, causing MARs to be signed ahead of installers, which tripped up our caching. The issue was fixed in bug 613320. The problems discovered in that bug are confirmed *not* affecting the stable branches, so the next 3.5/3.6 releases should not encounter these problems.
Since we could generate valid partials from the busted b7 MARs, for this release, we have switched the partial update snippet to point to the complete MAR which applies correctly regardless of the busted beta 7 MARs. Since the signing issue has already been fixed, this means that going forward, the beta 8->beta 9 (and onwards) partials should be able to be generated without issues.
The reason that this slipped through in the first place was because I only tested for consistency between locales when testing the signed builds, and did not run update_verify on builds signed with the original partner-repack patch. As part of the signing changes that have landed, we now verify the checksums of the MAR and installer internals as part of verify_signatures, which will catch the case where we have inconsistent checksums on any installer/MAR internals.
Looking forward, the best way to decrease the possibility of things like this happening in the future would probably be to expand the coverage provided by update_verify (we should have a way to find out about inconsistencies in applying updates before we try to generate updates for the next release). We already have the checksum verification steps in the signing code now to guard against the specific cases of checksum differences between the MARs and installers.
Post repack 3 snippet fixes
#cltbld@aus-staging
cd /opt/aus2/snippets/staging/
rsync -a 20101214-Firefox-4.0b8-test/ 20101214-Firefox-4.0b8-test-fixed3/
cd 20101214-Firefox-4.0b8-test-fixed3/Firefox
# Fix mac platform name
mkdir -p 3.7a1/Darwin_Universal-gcc3/20100208064157
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a1/Darwin_Universal-gcc3/20100208064157
mkdir -p 3.7a2/Darwin_Universal-gcc3/20100228155422
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a2/Darwin_Universal-gcc3/20100228155422/
mkdir -p 3.7a3/Darwin_Universal-gcc3/20100315065900
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a3/Darwin_Universal-gcc3/20100315065900/
mkdir -p 3.7a4/Darwin_Universal-gcc3/20100407105356
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a4/Darwin_Universal-gcc3/20100407105356/
mkdir -p 3.7a5/Darwin_Universal-gcc3/20100610053455
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 3.7a5/Darwin_Universal-gcc3/20100610053455
mkdir -p 4.0b1/Darwin_Universal-gcc3/20100630131607
rsync -av --exclude partial.txt 4.0b1/Darwin_x86_64-gcc3/20100630131936/ 4.0b1/Darwin_Universal-gcc3/20100630131607/
mkdir -p 4.0b2/Darwin_Universal-gcc3/20100720175703
rsync -av --exclude partial.txt 4.0b2/Darwin_x86_64-gcc3/20100720180051/ 4.0b2/Darwin_Universal-gcc3/20100720175703/
mkdir -p 4.0b3/Darwin_Universal-gcc3/20100805181943/
rsync -av --exclude partial.txt 4.0b3/Darwin_x86_64-gcc3/20100805182338/ 4.0b3/Darwin_Universal-gcc3/20100805181943/
mkdir -p 4.0b4/Darwin_Universal-gcc3/20100818121614
rsync -av --exclude partial.txt 4.0b4/Darwin_x86_64-gcc3/20100818121922/ 4.0b4/Darwin_Universal-gcc3/20100818121614/
mkdir -p 4.0b5/Darwin_x86-gcc3-u-ppc-i386/20100831070010
rsync -av --exclude partial.txt 4.0b5/Darwin_x86_64-gcc3/20100831070403/ 4.0b5/Darwin_x86-gcc3-u-ppc-i386/20100831070010/
mkdir -p 4.0b6/Darwin_x86-gcc3-u-ppc-i386/20100914072643
rsync -av --exclude partial.txt 4.0b6/Darwin_x86_64-gcc3/20100914073111/ 4.0b6/Darwin_x86-gcc3-u-ppc-i386/20100914072643/
# Generate ja-JP-mac snippet
mkdir -p 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/{beta,release}test
cat > 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/releasetest/complete.txt << EOF
version=1
type=complete
url=http://download.mozilla.org/?product=firefox-4.0b8-complete&os=osx64&lang=ja-JP-mac
hashFunction=SHA512
hashValue=fea614985261dc8e0cae6f05d19a55375b49f0560222b2892d703ad7b555a6fe2ad884a3fd8e876ca87ab5ccd1cdf338932af13977b2c61173815e9eeb6a0c5b
size=27373799
build=20101214165004
appv=4.0 Beta 8
extv=4.0b8
detailsUrl=http://www.mozilla.com/ja-JP-mac/firefox/4.0b8/releasenotes/
EOF
sed -e 's!^url=.*!url=http://stage-old.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/update/mac64/ja-JP-mac/firefox-4.0b8.complete.mar!' \
4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/releasetest/complete.txt \
> 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac/betatest/complete.txt
# copy to other locations, ja was added at b5
dirs=`find 4.0b{4,5,6,7} -maxdepth 1 -mindepth 1 -type d -name '*Darwin*' ! -wholename '4.0b7/Darwin_x86_64-gcc3' | sort`
for d in $dirs; do
rsync -av 4.0b7/Darwin_x86_64-gcc3/20101104131842/ja-JP-mac $d/*/
done
# fix paths/os
find */Darwin* -type f -exec sed -i -e 's!update/mac64/!update/mac/!' -e 's!os=osx64!os=osx!' {} \;
# Generate ja linux64 snippet
mkdir -p 4.0b7/Linux_x86_64-gcc3/20101104131204/ja/{beta,release}test
# setup snippet
cat > 4.0b7/Linux_x86_64-gcc3/20101104131204/ja/releasetest/complete.txt << EOF
version=1
type=complete
url=http://download.mozilla.org/?product=firefox-4.0b8-complete&os=linux64&lang=ja
hashFunction=SHA512
hashValue=482d120eebecccafce5e870e8a9e3fc1295d964960d21523c8cf7a1079c6a55d830b39eaa992f36e372f19998e920ce601a866849f9001a8aac5bafdd14a41e1
size=15996568
build=20101214180840
appv=4.0 Beta 8
extv=4.0b8
detailsUrl=http://www.mozilla.com/ja/firefox/4.0b8/releasenotes/
EOF
sed -e 's!^url=.*!url=http://stage-old.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b8-candidates/build1/update/linux-x86_64/ja/firefox-4.0b8.complete.mar!' \
4.0b7/Linux_x86_64-gcc3/20101104131204/ja/releasetest/complete.txt \
> 4.0b7/Linux_x86_64-gcc3/20101104131204/ja/betatest/complete.txt
# copy to other locations, ja was added at b5
dirs=`find 4.0b{4,5,6,7} -maxdepth 1 -mindepth 1 -type d -name '*Linux_x86_64-gcc3*' ! -wholename '4.0b7/Linux_x86_64-gcc3' | sort`
for d in $dirs; do
rsync -av 4.0b7/Linux_x86_64-gcc3/20101104131204/ja $d/*/
done
# win beta7 partial -> complete
dirs=`find 4.0b7/WINNT_x86-msvc/20101104142426 -maxdepth 1 -mindepth 1 -type d | sort`
for d in $dirs; do
rsync -av $d/betatest/complete.txt $d/betatest/partial.txt
rsync -av $d/releasetest/complete.txt $d/releasetest/partial.txt
sed -i 's/type=complete/type=partial/' $d/{beta,release}test/partial.txt
done
cd ../../
rsync -a 20101214-Firefox-4.0b8-test-fixed3/ 20101214-Firefox-4.0b8-fixed3/
cd 20101214-Firefox-4.0b8-fixed3/Firefox
find . -depth -name 'betatest' -type d -exec rm -rf {} \;
find . -mindepth 4 -maxdepth 4 -type d -exec mv {}/{releasetest,beta} \;
~/bin/backupsnip 20101214-Firefox-4.0b8-test-fixed3
~/bin/pushsnip 20101214-Firefox-4.0b8-test-fixed3