Releases/Firefox 14.0.1/BuildNotes-Updates

Updates

First problem: Failed to bump the patcher config

perl ../tools/release/patcher-config-bump.pl -p firefox -r Firefox -v 14.0.1 -a 14.0.1 \
-o 13.0.2 -b 1 ....
ASSERT: BumpFilePath() - Could not bump path: 'firefox/nightly/13.0.1-candidates/build1/update/%platform%/%locale%/firefox-13.0-13.0.1.partial.mar' from '13.0.2' to '14.0.1' at ../tools/lib/perl/Release/Patcher/Config.pm line 148.

due to incorrect oldVersion in release config.

Fix:

# buildbot-configs @ ed8561aa1056
sed -e 's/13_0_2/13_0_1/' -e 's/13\.0\.2/13.0.1/' -i  mozilla/release-firefox-mozilla-release.py
hg ci -m "Bug 758795, fix up oldVersion for FF 14.0.1, r=version-numbers-are-confusing"
hg up -r production
hg transplant 1aa1d4209bb1
hg tag -f -r production {FIREFOX,FENNEC}_14_0_1_{RELEASE,BUILD1}
hg push ssh://hg.mozilla.org/build/buildbot-configs

# bm12
cd /builds/buildbot/build1
source bin/activate 
make update reconfig

Note that the tagging was wrong for Fennec, revert it

hg tag -f -r b52a7634bca669ea1205ad34ab297b845ec193c1 FENNEC_14_0_1_RELEASE
hg tag -f -r b2b6a37879f23438c3fb29124f9b3d37e61cff13 FENNEC_14_0_1_BUILD1
# after verification using hg serve
hg push ssh://hg.mozilla.org/build/buildbot-configs

Forced builder.

Second problem: Failed to upload aus2-test because in the create_snippets step we got

Partial patch info - 0 to create
Complete patch info - 0 to create
Past release patch info - 6168 to create
No known patch for locale af, 10.0 -> 14.0.1; skipping...

instead of generating snippets. This is because the checksums files have lines like this

26dee2baa2b425fb3d8057879558bf89 md5 7771770 update/win32/en-US/firefox-13.0.2-14.0.1.partial.mar

Which is the same problem with the wrong version in the config. In fact we have generated partials for 13.0.2 to 14.0.1, which won't be any use, and will need to create 13.0.1 to 14.0.1 partials.

Manually generate 13.0.1-14.0.1 partials

Working from the howto doc, but no config changes are necessary here.

## on linux-ix-slave36
cd /builds/slave/rel-m-rel-updates/build
mkdir -p logs/13.0.1-14.0.1

## build tools (9 min)
export HGROOT=http://hg.mozilla.org/releases/mozilla-release
perl patcher2.pl --build-tools-hg --tools-revision=FIREFOX_14_0_1_RELEASE \
  --app=firefox --brand=Firefox \
  --config=patcher-configs/mozRelease-branch-patcher2.cfg

## get complete mar files   (75 min, very badly needs parallelization)
perl patcher2.pl --download \
  --app=firefox --brand=Firefox \
  --config=patcher-configs/mozRelease-branch-patcher2.cfg \
  2>&1 | tee logs/13.0.1-14.0.1/download.log

## generate unsigned partial updates (100 min)
perl patcher2.pl --create-patches --partial-patchlist-file=patchlist.cfg \
 --app=firefox --brand=Firefox \
 --config=patcher-configs/mozRelease-branch-patcher2.cfg \
  2>&1 | tee logs/13.0.1-14.0.1/create-partials.log

## Make a backup of the new partials/snippets, destroy snippets (6 min)
rsync -a temp/firefox/13.0.1-14.0.1{,-unsigned}/
rm -rf temp/firefox/13.0.1-14.0.1/aus2*

## Sign the mar files (< 1 min)
# get a token
# look up the values for --user in the release-signing part of passwords.py, on the master
unset HISTFILE
IP=`/sbin/ifconfig eth0 | grep "inet addr" | awk -F: '{print $2}' | awk '{print $1}'`
ssh -oBatchMode=no cltbld@buildbot-master12.build.mozilla.org \
  "curl --fail -k -Fslave_ip=$IP -Fduration=3600 --user reda:cted \
   https://signing1.build.scl1.mozilla.com:9120/token" \
     > /builds/slave/rel-m-rel-updates/token
rm /builds/slave/rel-m-rel-updates/nonce

# update the channel id and version, then sign (25 min)
(for f in `find temp/firefox/13.0.1-14.0.1/ftp -type f -name *partial.mar | sort`; do
  echo Signing $f
  mozilla/obj/dist/host/bin/mar -H firefox-mozilla-release -V 14.0.1 -i $f;
  /tools/python-2.6.5/bin/python \
    /builds/slave/rel-m-rel-updates/tools/release/signing/signtool.py \
    -t /builds/slave/rel-m-rel-updates/token -n /builds/slave/rel-m-rel-updates/nonce \
    -c /builds/slave/rel-m-rel-updates/tools/release/signing/host.cert \
    -H signing1.build.scl1.mozilla.com:9120 -H signing2.build.scl1.mozilla.com:9120 -f gpg -f mar \
    "$f"
done ) 2>&1 | tee logs/13.0.1-14.0.1/sign-partials.log

## Regenerate the snippets (4 min)
perl patcher2.pl --create-patchinfo \
  --app=firefox --brand=Firefox \
  --config=patcher-configs/mozRelease-branch-patcher2.cfg \
  2>&1 | tee logs/13.0.1-14.0.1/generate_snippets.log

# Snippet verification
cd temp/firefox/13.0.1-14.0.1
bash /builds/slave/rel-m-rel-updates/tools/release/compare-channel-snippets.sh \
  aus2 release aus2.test  releasetest

# Tidy and upload mar files (6 min)
find ftp/firefox/nightly/14.0.1-candidates/build1 -type f -exec chmod 644 {} \; 
find ftp/firefox/nightly/14.0.1-candidates/build1 -type d -exec chmod 755 {} \;
# $BUILD is the build number, eg 1, 2, ...
rsync -nav -e 'ssh -oIdentityFile=~/.ssh/ffxbld_dsa' '--exclude=*complete.mar' \
  ftp/firefox/nightly/14.0.1-candidates/build1/update \
  ffxbld@stage.mozilla.org:/pub/mozilla.org/firefox/nightly/14.0.1-candidates/build1/
# then repeat without the -n

# Upload snippets (2 min, NB: no  --exclude=complete.txt)
rsync -av -e 'ssh -oIdentityFile=~/.ssh/auspush' \
 aus2.test/ \
 ffxbld@aus3-staging.mozilla.org:/opt/aus2/snippets/staging/Firefox-14.0.1-build1-test
rsync -av -e 'ssh -oIdentityFile=~/.ssh/auspush' \
 aus2/ \
 ffxbld@aus3-staging.mozilla.org:/opt/aus2/snippets/staging/Firefox-14.0.1-build1

Defer backupsnip until after other changes.

Adjust What's New in snippets

From bug 767207

locales="en-US en-GB fr it de es-ES es-AR es-CL es-MX ja ja-JP-mac ko ru"

# backup auto-generated snippets
cd /opt/aus2/snippets/staging
rsync -a Firefox-14.0.1-build1{,-withoutWN}/
rsync -a Firefox-14.0.1-build1-test{,-withoutWN}/

for l in $locales; do
  for f in `find Firefox-14.0.1-build1{,-test}/*/*/*/*/$l -type f`; do
    echo tweaking $f
    sed -i '/^actions.*$/d' $f
  done
done

Did some quick checks using diff, looks OK.

Generate 12.0 to 14.0.1 partial update for windows

## on linux-ix-slave36
cd /builds/slave/rel-m-rel-updates/build
mkdir -p logs/12.0-14.0.1
curl -sL https://bugzilla.mozilla.org/attachment.cgi?id=642529 \
  > patcher-configs/mozRelease-branch-patcher2-758795.cfg

# save the previous partial working space, but duplicate 14.0.1
# win32 completes as a speed up (2 min)
mv temp/ temp-13.0.1-14.0.1-partial/
mkdir -p temp/firefox/14.0.1/ftp
cp -pv temp-13.0.1-14.0.1-partial/firefox/14.0.1/ftp/*win32* temp/firefox/14.0.1/ftp/

## build tools - skipped as we had this from 12.0.1 - 13.0.1

## Get the complete mar files (8 min)
perl patcher2.pl --download \
  --app=firefox --brand=Firefox \
  --config=patcher-configs/mozRelease-branch-patcher2-758795.cfg

## Generate unsigned partial updates (28 min)
perl patcher2.pl --create-patches --partial-patchlist-file=patchlist.cfg \
 --app=firefox --brand=Firefox \
 --config=patcher-configs/mozRelease-branch-patcher2-758795.cfg \
  2>&1 | tee logs/12.0-14.0.1/create-partials.log

## Make a backup of the new partials/snippets, destroy snippets (< 1 min)
rsync -a temp/firefox/12.0-14.0.1{,-unsigned}/
rm -rf temp/firefox/12.0-14.0.1/aus2*

## Sign the mar files (< 1 min)
# get a token
# look up the values for --user in the release-signing part of passwords.py, on the master
unset HISTFILE
IP=`/sbin/ifconfig eth0 | grep "inet addr" | awk -F: '{print $2}' | awk '{print $1}'`
ssh -oBatchMode=no cltbld@buildbot-master12.build.mozilla.org \
  "curl --fail -k -Fslave_ip=$IP -Fduration=3600 --user reda:cted \
   https://signing1.build.scl1.mozilla.com:9120/token" \
     > /builds/slave/rel-m-rel-updates/token
rm /builds/slave/rel-m-rel-updates/nonce

# update the channel id and version, then sign (8 min)
(for f in `find temp/firefox/12.0-14.0.1/ftp -type f -name *partial.mar | sort`; do
  echo Signing $f
  mozilla/obj/dist/host/bin/mar -H firefox-mozilla-release -V 14.0.1 -i $f;
  /tools/python-2.6.5/bin/python \
    /builds/slave/rel-m-rel-updates/tools/release/signing/signtool.py \
    -t /builds/slave/rel-m-rel-updates/token -n /builds/slave/rel-m-rel-updates/nonce \
    -c /builds/slave/rel-m-rel-updates/tools/release/signing/host.cert \
    -H signing1.build.scl1.mozilla.com:9120 -H signing2.build.scl1.mozilla.com:9120 -f gpg -f mar \
    "$f"
done ) 2>&1 | tee logs/12.0-14.0.1/sign-partials.log

## Regenerate the snippets (1 min)
perl patcher2.pl --create-patchinfo \
  --app=firefox --brand=Firefox \
  --config=patcher-configs/mozRelease-branch-patcher2-758795.cfg \
  2>&1 | tee logs/12.0-14.0.1/generate_snippets.log

# Snippet verification (<1 min)
cd temp/firefox/12.0-14.0.1
bash /builds/slave/rel-m-rel-updates/tools/release/compare-channel-snippets.sh \
  aus2 release aus2.test  releasetest

# Tidy and upload mar files (3 min)
find ftp/firefox/nightly/14.0.1-candidates/build1 -type f -exec chmod 644 {} \; 
find ftp/firefox/nightly/14.0.1-candidates/build1 -type d -exec chmod 755 {} \;
rsync -nav -e 'ssh -oIdentityFile=~/.ssh/ffxbld_dsa' '--exclude=*complete.mar' \
  ftp/firefox/nightly/14.0.1-candidates/build1/update \
  ffxbld@stage.mozilla.org:/pub/mozilla.org/firefox/nightly/14.0.1-candidates/build1/
# then repeat without the -n

# Upload snippets (1 min)
rsync -av -e 'ssh -oIdentityFile=~/.ssh/auspush' --exclude=complete.txt \
 aus2.test/ \
 ffxbld@aus3-staging.mozilla.org:/opt/aus2/snippets/staging/Firefox-14.0.1-build1-test
rsync -av -e 'ssh -oIdentityFile=~/.ssh/auspush' --exclude=complete.txt \
 aus2/ \
 ffxbld@aus3-staging.mozilla.org:/opt/aus2/snippets/staging/Firefox-14.0.1-build1

# aus3-staging
backupsnip Firefox-14.0.1-build1-test
pushsnip   Firefox-14.0.1-build1-test

Forced update verify builders.

Generate release-cck-mozilla11/12 snippets

By duplicating the 13.0.1 release/releasetest snippets and modifying the bouncer product, see bug for more info.

# ffxbld@aus3-staging
cd /opt/aus2/snippets/staging
mkdir -p Firefox-14.0.1-build1-funnelcake11_12-test/Firefox
rsync -a --exclude=Linux* --exclude=betatest \
  Firefox-14.0.1-build1{,-funnelcake11_12}-test/Firefox/13.0.1/
for d in `find Firefox-14.0.1-build1-funnelcake11_12-test -mindepth 6 -type d`; do
  mv $d $d-cck-mozilla11
  rsync -a $d-cck-mozilla11/ $d-cck-mozilla12/
  sed -i -e 's/\-complete/-complete-funnelcake11/' \
            -e 's/-partial-13\.0\.1/-partial-13.0.1-funnelcake11/' $d-cck-mozilla11/*.txt
  sed -i -e 's/\-complete/-complete-funnelcake12/' \
            -e 's/-partial-13\.0\.1/-partial-13.0.1-funnelcake12/' $d-cck-mozilla12/*.txt
done
 
rsync -a Firefox-14.0.1-build1-funnelcake11_12{-test,}/
for d in `find Firefox-14.0.1-build1-funnelcake11_12 -mindepth 6 -type d`; do
  mv $d ${d/releasetest/release}
done

backupsnip Firefox-14.0.1-build1-funnelcake11_12-test
pushsnip   Firefox-14.0.1-build1-funnelcake11_12-test

Update verify

Automated verify running...

12.0 partial

On mw32-ix-slave07 stop buildbot slave and to this in terminal via VNC

# windows
cd /e/builds/moz2_slave

# all platforms
mkdir bug758795; cd $_
hg clone http://hg.mozilla.org/build/tools scripts
hg -R scripts update -C -r FIREFOX_14_0_1_RELEASE_RUNTIME

cd scripts/release/updates/
wget -O- https://bugzilla.mozilla.org/attachment.cgi?id=642483 | patch -p3

bash verify.sh -c mozRelease-firefox-win32-758795.cfg 2>&1 |tee uv.log; echo $?

Running ...