Releases/Firefox 3.6.13/BuildNotes

From MozillaWiki
Jump to: navigation, search

Build Engineers

bhearsum
Tracking bug

Signed-off Revision(s)

Build1: 5d2cbf9c9669

Tags

Build # Tag Changeset
1 FIREFOX_3_6_13_BUILD1 FIREFOX_3_6_13_RELEASE af7a49ca89b2

Build data

Build # Type Build ID Build machine Time to build
1 Linux 20101122045408 moz2-linux-slave14 1 hrs, 22 mins, 30 secs
Mac 20101122044323 bm-xserve17 1 hrs, 37 mins, 28 secs
Windows 20101122053531 mw32-ix-slave06 1 hrs, 41 mins, 46 secs
2 Linux 20101130065329 moz2-linux-slave15 1 hrs, 17 mins, 27 secs
Mac 20101130064313 bm-xserve17 1 hrs, 35 mins, 58 secs
Windows 20101130064308 mw32-ix-slave06 1 hrs, 46 mins, 5 secs
3 Linux 20101203074139 mv-moz2-linux-ix-slave03 37 mins, 16 secs
Mac 20101203074432 bm-xserve17 1 hrs, 38 mins, 20 secs
Windows 20101203075014 mw32-ix-slave06 1 hrs, 57 mins, 32 secs

Build 1

Done on pm03. Setup notes:

  • No clobberer necessary because this is the first 0.8.0 release on this branch.
  • landed config updates, including l10n-changesets
  • tagged buildbot-configs, buildbotcustom, compare-locales, and tools with FIREFOX_3_6_12_BUILD1 and FIREFOX_3_6_12_RELEASE
  • updated /tools/buildbotcustom/buildbotcustom, /builds/buildbot/configs, and /builds/buildbot/builder_master/tools to latest
  • reconfiged master
  • Left tree open, 0.8.0 automation doesn't require that it closes
  • kicked off automation with:
buildbot sendchange --username=bhearsum --master=localhost:9010 --branch=releases/mozilla-1.9.2 --revision=FIREFOX_3_6_13_RELEASE -p release_config:mozilla/release-firefox-mozilla-1.9.2.py gogogo

Tag

'pl' locale failed when 'hg out' got a bad response from the HG server. (FIXME we should be retrying when any hg operation fails).

Fixed manually by running the following on mv-moz2-linux-ix-slave11, as cltbld:

cd /builds/slave/release-mozilla-1.9.2-tag/build/pl
hg push --new-branch -e 'ssh -l ffxbld -i ~/.ssh/ffxbld_dsa' ssh://hg.mozilla.org/releases/l10n-mozilla-1.9.2/pl

Manually started source, xulrunner_source, _build and _xulrunner_build for all platforms, filling in the form as follows:

  • Branch: releases/mozilla-1.9.2
  • Revision: FIREFOX_3_6_13_RELEASE
  • Property 1 Name: release_config
  • Property 1 Value: mozilla/release-firefox-mozilla-1.9.2.py

Source / XULRunner Source

No Problems

Build

No problems. Removed en-US.xpi from stage

L10n Repack

Triggered manually, see Tag section for form details.

Failed a few times because the scripts were trying to use staging-only variables. Landed a few bustage fixes before it got working again. Patches are in bug 613944.

Lots of individual repacks failed when uploading. Found them by searching for "ssh_exchange_identification: Connection closed by remote host" in the logs. To fix, did the following for each failed locale (on the machine that tried to build it):

export UPLOAD_USER=ffxbld
export MOZ_OBJDIR=obj-l10n
export UPLOAD_SSH_KEY="~/.ssh/ffxbld_dsa"
export UPLOAD_HOST=stage.mozilla.org
export POST_UPLOAD_CMD="post_upload.py -p firefox -n 1 -v 3.6.13 --release-to-candidates-dir"
export UPLOAD_TO_TEMP=1
export MOZ_PKG_PRETTYNAMES=1
cd /builds/moz2_slave/release-mozilla-1.9.2-$platform_repack_$n/build/mozilla-1.9.2/obj-l10n/browser/locales
export AB_CD=$locale
make l10n-upload-$locale

Did the following on stage to make sure we had all of the locales:

cd /home/ftp/pub/firefox/nightly/3.6.13-candidates/build1
ls unsigned/win32 | grep -v xpi | wc -l
ls mac | grep -v xpi | wc -l
ls linux-i686 | grep -v xpi | wc -l

Output for each should be 76, which it was.

Partner Repack

No problems. Signing happened at the same time as other builds.

XULRunner Build

No problems

Unittests

  • Mac: All green
  • Linux: All green
  • Win32: All green except for mochitests, which ran all test successfully and then hung at shutdown. Retriggered it, went green.

Signing

Removed en-US.xpi from candidates directory. No problems, followed CombinedSigning doc.

Got "Signature verification succeeded" mail.

L10n Verify

Many string changes across many platforms/locales. All seemed legitimate, didn't dig further.

Updates

3.6* -> 3.6.13

Generation
  • First attempt failed when the Linux 'id' MAR failed to download. Dug in a bit more and found that all locales I had recovered manually were missing MARs because I forgot to set MOZ_MAKE_COMPLETE_MAR=1 when uploading. To fix
  • re-did the l10n uploads w/ MOZ_MAKE_COMPLETE_MAR
  • resigned everything.
  • clobbered updates builddir
  • restarted updates from the beginning
  • No other issues
Update Verify
  • Linux: PASS
  • Windows: PASS
  • Mac: Complained that there was no partial update for 3.6.4's 'ca' locale. Manually checking of the link did not reproduce, likely a fluke. Re-run update verify for confirmation.
    • No issues on the re-run

Socorro product data updates

Added '3.6.13' and '3.6.14pre' to crash-stats with this admin page.

Update Bouncer

Used "Force Build" to start the "release-mozilla-1.9.2-bouncer_submitter" builder.

  • First attempt failed with the error: "Cannot retrieve credentials file: BuildSlaves.py"
    • Noticed that tuxedoUsername/tuxedoPassword were missing from BuildSlaves.py on pm01/pm03.
  • After adding the entries I used "Force Build" again. No problems this time.

Stage EU Ballot Builds

  • Using the EU Ballot builds generated by the build machines
  • No signing necessary as these were signed earlier
  • Kev looked over them before staging

Once Kev verified them, ran the following to get them ready for the mirror push:

# ffxbld@stage
cd /home/ftp/pub/firefox/nightly/3.6.13-candidates/build1
mv -v partner-repacks/euballot/win32 win32-EUballot

Ideally, the partner repack scripts or signing would upload this to this location from the git go. Filed bug 614227 to track this.

Push to Beta Channel

As cltbld@aus2-staging:

~/bin/backupsnip 20101122-Firefox-3.6.13-beta
~/bin/pushsnip 20101122-Firefox-3.6.13-beta

Build 2

Done on pm03. Setup notes:

  • Set a clobber for pm03, 'Any release', 'Any builder' with clobberer.
    • Noticed that after clobbering, all of the release builders showed up in the table. Filed bug 615192 on the issue.
  • Tagged buildbot-configs, buildbotcustom, and tools with FIREFOX_3_6_13_RELEASE and FIREFOX_3_6_13_BUILD2. compare-locales will be tagged by the tagging step this time.
  • Updated /tools/buildbotcustom/buildbotcustom, /builds/buildbot/buildbot-configs, and /builds/buildbot/builder_master/tools to latest
  • reconfiged master
  • Ran release_sanity.py with --dryrun:
PYTHONPATH=.:tools/lib/python python tools/buildbot-helpers/release_sanity.py -u bhearsum -V 3.6.13 --branch mozilla-1.9.2 --build-number 2 --dryrun localhost:9010
  • After it passed, ran it for real:
PYTHONPATH=.:tools/lib/python python tools/buildbot-helpers/release_sanity.py -u bhearsum -V 3.6.13 --branch mozilla-1.9.2 --build-number 2 localhost:9010

Tag

  • Cancelled the first run because it merged other changesets, and had revision set wrong.
  • Re-ran release_sanity.py, revision was set correctly this time. No further issues.

Source/XULRunner Source

No problems

Build

No problems

L10n Repack

No problems

Partner Repack

No problems

XULRunner Build

No problems.

Sign

Removed en-US.xpi from candidates directory.

Followed instructions on CombinedSigning page.

No problems.

L10n Verify

Many string changes across many platforms/locales. All seemed legitimate, didn't dig further.

Update Generation

First attempt failed with:

Traceback (most recent call last):
  File "/builds/slave/release-mozilla-1.9.2-updates/tools/release/generate-candidate-build-updates.py", line 199, in <module>
    sys.exit(main())
  File "/builds/slave/release-mozilla-1.9.2-updates/tools/release/generate-candidate-build-updates.py", line 194, in main
    options.verbose)
  File "/builds/slave/release-mozilla-1.9.2-updates/tools/release/generate-candidate-build-updates.py", line 55, in createSnippets
    verbose)
TypeError: getShippedLocales() takes at most 6 arguments (7 given)

Tracked in bug 615668. After landing the patch, backed out update verify changes, retagged build/tools with FIREFOX_3_6_13_RELEASE and FIREFOX_3_6_13_BUILD2, clobbered the updates builder, and used "Force Build" to re-start updates.

No further problems.

Update Verify

  • All platforms: PASS

Stage EU Ballot Builds

  • Using the EU Ballot builds generated by the build machines
  • No signing necessary as these were signed earlier
  • Kev looked over them before staging

Once Kev verified them, ran the following to get them ready for the mirror push:

# ffxbld@stage
cd /home/ftp/pub/firefox/nightly/3.6.13-candidates/build2
mv -v partner-repacks/euballot/win32 win32-EUballot

Ideally, the partner repack scripts or signing would upload this to this location from the git go. Filed bug 614227 to track this.

Push to beta channel

~/bin/backupsnip 20101130-Firefox-3.6.13-beta
~/bin/pushsnip 20101130-Firefox-3.6.13-beta

ashughes reported that he was not receiving updates for 3.6.13build1 on Mac. After digging into it I discovered that this was because the 3.6.13 Mac builds use "Darwin_x86-gcc3-u-ppc-i386" instead of "Darwin_Universal-gcc3" in the update URL. To workaround the issue for now, I created a symlink in the live snippet directory:

# cltbld@aus2-staging
cd /opt/aus2/incoming/3/Firefox/3.6.13
ln -s Darwin_x86-gcc3-u-ppc-i386 Darwin_Universal-gcc3

EDIT: Also added symlink for Darwin_ppc-gcc3-u-ppc-i386 (NThomas 17:42, 12 December 2010 (PST))

The long term fix is getting patcher tools to support the new OS names, tracked in bug 583671, which needs to be fixed before 3.6.14.

At LegNeato's request, we completely unthrottled the 3.6 beta channel. Tracked in bug 615902.

Build 3

Done on pm03. Setup notes:

  • Set a clobber for pm03, 'Any release', 'Any builder' with clobberer.
  • Tagged buildbot-configs, buildbotcustom, and tools with FIREFOX_3_6_13_RELEASE and FIREFOX_3_6_13_BUILD3.
  • Updated /tools/buildbotcustom/buildbotcustom, /builds/buildbot/buildbot-configs, and /builds/buildbot/builder_master/tools to latest
  • reconfiged master
  • Ran release_sanity.py with --dryrun:
PYTHONPATH=.:tools/lib/python python tools/buildbot-helpers/release_sanity.py -u bhearsum -V 3.6.13 --branch mozilla-1.9.2 --build-number 3 --dryrun localhost:9010
  • After it passed, ran it for real:
PYTHONPATH=.:tools/lib/python python tools/buildbot-helpers/release_sanity.py -u bhearsum -V 3.6.13 --branch mozilla-1.9.2 --build-number 3 localhost:9010

Tag

First attempt failed because of bug 614946. Re-run the second release_sanity.py command to get things going. No other problems.

Source / XULRunner Source

No problems

Builds

No problems

L10n Repacks

Partner Repacks

Linux partner repacks failed to upload with the following error:

rsync: mkdir "/pub/mozilla.org/firefox/nightly/3.6.13-candidates/build3/unsigned/partner-repacks" failed: No such file or directory (2)
rsync error: error in file IO (code 11) at main.c(577) [Receiver=3.0.7]
rsync: connection unexpectedly closed (8 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at /SourceCache/rsync/rsync-30/rsync/io.c(359)
<p>To fix, did the following manually on bm-xserve09:

cd /builds/slave/release-mozilla-1.9.2-linux_partner_repack/partner-repacks/scripts/repacked_builds/3.6.13/build3
rsync -av -e "ssh -oIdentityFile=~/.ssh/ffxbld_dsa" . ffxbld@stage-old.mozilla.org:/pub/mozilla.org/firefox/nightly/3.6.13-candidates/build3/unsigned/partner-repacks

Filed bug 616512 on the issue.

coop: On release day, we discovered that the partner repacks for linux and mac were missing. I determined that this is because as part of our signing process, our initial rsync of linux and mac builds misses the partner repacks because all our partner repacks live under the unsigned/ dir with the win32 builds.

I ran the following commands on the staging machine to recover from this:

cd ~/signing-work/firefox-3.6.13/signed-build3/partner-repacks
rsync -nav --exclude '*win32*' ../../unsigned-build3/unsigned/partner-repacks / .
cd ~/signing-work/firefox-3.6.13
PRODUCT=firefox
VERSION=3.6.13
BUILD=3
TAG=FIREFOX_3_6_13_RELEASE
REPO='releases/mozilla-1.9.2'
EMAIL=coop@mozilla.com
KEYDIR=d:/2010-keys
make df verify-gpg-passphrase create-sigs verify-signatures verify-asc fake-upload PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
# Verified that only partner repacks for linux/mac are changed (and SUM files)
make postsign PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL}

Then on stage, I did the following:

# ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/3.6.13-candidates/build3/
# EUballot builds have already been synced at this point, and are win32-only anyway
rm -rf partner-repacks/euballot/win32

XULRunner Builds

No problems

Signing

Removed en-US.xpi from candidates directory

To speed up signing, ran the following prior to all repacks being completed:

PRODUCT=firefox
VERSION=3.6.13
BUILD=3
TAG=FIREFOX_3_6_13_RELEASE
REPO='releases/mozilla-1.9.2'
EMAIL=bhearsum@mozilla.com
KEYDIR=d:/2010-keys
cd ~/hg-tools
hg pull && hg up -r ${TAG}
mkdir ~/signing-work/${PRODUCT}-${VERSION}
cd ~/signing-work/${PRODUCT}-${VERSION}
cp ~/hg-tools/release/signing/* .
make df verify-gpg-passphrase setup verify-signcode-passphrase PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
# download target run multiple times to get things as they complete
make download PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1

Once all the l10n and partner repacks finished, ran the following in the same window to do the signing:

# Need to download one more time to make sure we've got everything
make download PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
make stubs sign2 stage verify fake-upload PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1

And then to upload:

make postsign PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL}

Got "Signature Verification Succeeded" mail later.

L10n Verify

Updates

First attempt failed because the updates directory hadn't been clobbered. Filed bug 616581 on the issue. To fix, did the following:

  • Clobbered /builds/slave/release-mozilla-1.9.2-updates on mv-moz2-linux-ix-slave11 by hand
  • Backed out update verify changes
  • Used "Force Build" to restart the updates builder.

Update Verify

  • Linux: PASS
  • Windows: PASS
  • Mac: Complained that there was no partial update for 3.6b2's 'tr' locale. Manually checking of the link did not reproduce, likely a fluke. Re-run update verify for confirmation.
    • No-issues in the re-run.

Push to Beta Channel

~/bin/backupsnip 20101203-Firefox-3.6.13-beta
~/bin/pushsnip 20101203-Firefox-3.6.13-beta

3.0.19 -> 3.6.13 Major Update

Generation

On mv-moz2-linux-ix-slave03

mkdir -p /builds/slave/3.0.19-3.6.13-major-update/snippets
cd !$
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -d patcher-configs mozilla/tools/patcher-configs
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R9 -d patcher mozilla/tools/patcher
cd patcher
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R9 -d MozBuild mozilla/tools/release/MozBuild

export CVSROOT=:ext:cltbld@cvs.mozilla.org:/cvsroot
# build tools
./patcher2.pl --build-tools --app=firefox --tools-rev=UPDATE_PACKAGING_R9 --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_build-tools.log
# download mars
./patcher2.pl --download --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_download.log

# FIXME - patcher needs to see that the MARs that it thinks are partials
#             are there or else it will not attempt to generate patchinfo
cd temp/firefox
ln -s 3.6.13 3.0.19-3.6.13
cd ../..
# Create partial patches and snippets
./patcher2.pl --create-patches --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox-create-patches.log

Quick Verification

cd /builds/slave/3.0.19-3.6.13-major-update/snippets/patcher/temp/firefox/3.0.19-3.6.13/

# releasetest == beta
find aus2.test -type d -iregex '.*/releasetest$' | perl -nle '$a = $_; $a =~ s/releasetest/beta/; $a =~ s/aus2\.test/aus2.beta/; system("diff -r -u $_ $a");'
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/releasetest/; $a =~ s/aus2\.beta/aus2.test/; system("diff -r -u $_ $a");'

# beta == release
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/aus2\.beta/aus2/; $a =~ s/beta/release/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/beta/; $a =~ s/aus2/aus2.beta/; system("diff -r -u $_ $a");'

Push to AUS

cd /builds/slave/3.0.19-3.6.13-major-update/snippets/patcher/temp/firefox/3.0.19-3.6.13
rsync -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20101208-Firefox-3.0.19-3.6.13-MU-test
rsync -av aus2.beta/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20101208-Firefox-3.0.19-3.6.13-MU-beta
rsync -av aus2/      cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20101208-Firefox-3.0.19-3.6.13-MU

Push test snippets

As cltbld@aus2-staging:

cd /opt/aus2/snippets/staging
~/bin/backupsnip 20101208-Firefox-3.0.19-3.6.13-MU-test
~/bin/pushsnip 20101208-Firefox-3.0.19-3.6.13-MU-test

Verify

mv-moz2-linux-ix-slave11, mw32-ix-slave08, moz2-darwin9-slave14

mkdir -p /e/builds/moz2_slave/verify/firefox-3019-3613-major/
cd /e/builds/moz2_slave/verify/firefox-3019-3613-major/
hg clone http://hg.mozilla.org/build/tools

cd tools/release/updates
platform=win32   # or mac
./verify.sh -c moz19-firefox-$platform-major.cfg 2>&1 | tee $platform.log

Results

PASS, all exceptions noted in the 3.0.19->3.6 Results, as well as:

Only in source/firefox/searchplugins: rambler.xml

Stage EU Ballot Builds

  • Using the EU Ballot builds generated by the build machines
  • No signing necessary as these were signed earlier
  • Kev looked over them before staging

Once Kev verified them, ran the following to get them ready for the mirror push:

# ffxbld@stage
cd /home/ftp/pub/firefox/nightly/3.6.13-candidates/build3
mv -v partner-repacks/euballot/win32 win32-EUballot

Ideally, the partner repack scripts or signing would upload this to this location from the git go. Filed bug 614227 to track this.

Virus Scan

# cltbld@stage
clamdscan /mnt/netapp/stage/archive.mozilla.org/pub/firefox/nightly/3.6.13-candidates/build3/
----------- SCAN SUMMARY -----------
Infected files: 0
Time: 6606.432 sec (110 m 6 s)

Final checks before push

# ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/3.6.13-candidates/build3/
find . ! -user ffxbld
find . ! -group firefox
find . -type f ! -perm 644
find . -mindepth 1 -type d ! -perm 755 ! -name 'contrib*'
find . -maxdepth 1 -type d ! -perm 2775  -name 'contrib*'
  • contrib directories/files not owned by ffxbld, that's OK
  • partner builds not 644/755, OK because we aren't shipping them. FIXME anyways though
  • contrib/solaris_pkgadd and solaris_tarball were 755 (g+s), which is OK

Push to Mirrors

rsync -av --exclude=partner-repacks --exclude="*reporter*" --exclude=*.log --exclude=*.txt --exclude=*unsigned* /pub/mozilla.org/firefox/nightly/3.6.13-candidates/build3/ /pub/mozilla.org/firefox/releases/3.6.13/
# cltbld@stage
vim -n /pub/mozilla.org/zz/rsyncd-mozilla-current.exclude
# replace 3.6.12 with 3.6.13

Final Verification

Did not get run.

Push release channel snippets

# cltbld@aus2-staging
~/bin/backupsnip 20101203-Firefox-3.6.13
~/bin/pushsnip 20101203-Firefox-3.6.13

Push 3.0.19 -> 3.6.13 MU snippets

# cltbld@aus2-staging
~/bin/backupsnip 20101208-Firefox-3.0.19-3.6.13-MU
~/bin/pushsnip 20101208-Firefox-3.0.19-3.6.13-MU
~/bin/pushsnip 20101208-Firefox-3.0.19-3.6.13-MU-beta

Push EU Ballot build

Manually added Bouncer entry. Tomcat filed the bug to get the websites updated. bug 618061

Release

Update symlinks

#ffxbld@stage
cd /home/ftp/pub/firefox/releases
rm latest-3.6 && ln -s 3.6.13 latest-3.6

Sent the following mail to metrics@mozilla.com:

Firefox,firefox,firefox,3.6.13,3.6
Firefox,firefox,firefox,3.6.14pre,3.6

XULRunner Signing

#!/bin/bash
set -x
set -e

PRODUCT=xulrunner
VERSION=1.9.2.13
BUILD=3
TAG=FIREFOX_3_6_13_RELEASE
REPO="releases/mozilla-1.9.2"
EMAIL=bhearsum@mozilla.com
KEYDIR=d:/2010-keys

cd ~/hg-tools
hg pull
hg up -r ${TAG}

mkdir -p ~/signing-work/${PRODUCT}-${VERSION}
cd ~/signing-work/${PRODUCT}-${VERSION}
cp ~/hg-tools/release/signing/* .

make setup PRODUCT=${PRODUCT} VERSION=${VERSION} \
  BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
rsync -av -e "ssh -i /home/cltsign/.ssh/xrbld_dsa" \
  --exclude=*.txt --exclude=*-symbols.zip \
  xrbld@stage.mozilla.org:/home/ftp/pub/${PRODUCT}/nightly/${VERSION}-candidates/build${BUILD}/ \
 unsigned-build${BUILD}
rsync -av --exclude=unsigned unsigned-build${BUILD}/ \
 signed-build${BUILD}/
rsync -av unsigned-build${BUILD}/unsigned/ \
 signed-build${BUILD}/

make checksum-files create-sigs stage verify-sigs \
  PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} \
  REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1

rmdir signed-build${BUILD}/contrib{,-localized}

rsync -av -e "ssh -i /home/cltsign/.ssh/xrbld_dsa" \
  signed-build${BUILD}/ \
  xrbld@stage.mozilla.org:/home/ftp/pub/${PRODUCT}/nightly/${VERSION}-candidates/build${BUILD}/

Push XULRunner to mirrors

# xrbld@stage
rsync -av --exclude=*.txt --exclude=*unsigned*  --exclude=*-symbols.zip \
  /pub/mozilla.org/xulrunner/nightly/1.9.2.13-candidates/build3/ \
  /pub/mozilla.org/xulrunner/releases/1.9.2.13/

Edited cltbld@stage:/pub/mozilla.org/zz/rsyncd-mozilla-releases.exclude and added

- xulrunner/releases/1.9.2.12

Update XULRunner entries on wiki

Updated the links in these two wiki pages for the version change: