Releases/Firefox 3.6b1/BuildNotes

NB: Text that is italicized indicates tasks still to be done, or information that needs to be filled in.

Build Engineers

nthomas, bhearsum

Main tracking bug: bug 518496, WinCE updates: bug 518408

Signed-off Revision(s)

Build 1: http://hg.mozilla.org/releases/mozilla-1.9.2/rev/f789bb32468d
Build 2: http://hg.mozilla.org/releases/mozilla-1.9.2/rev/8217722e6af4
Build 3: http://hg.mozilla.org/releases/mozilla-1.9.2/rev/3b49c063bb42

Tags

On mozilla-central:

Build #	Tag	Changeset
1	GECKO192b1_20091014_RELBRANCH	f789bb32468d
	FIREFOX_3_6b1_BUILD1	c60105da01a0
2	FIREFOX_3_6b1_BUILD2	8217722e6af4
3	FIREFOX_3_6b1_BUILD3	3b49c063bb42

Build data

Build 1:

Platform	Build ID	Build machine
Win32	20091014103305	moz2-win32-slave41
Mac	20091014092847	moz2-darwin9-slave11
Linux	20091014092232	moz2-linux-slave41
WinCE	20091014091718	moz2-win32-slave09

Build2:

Platform	Build ID	Build machine
Win32	20091019095728	moz2-win32-slave40
Mac	20091019084525	moz2-darwin9-slave06
Linux	20091019084521	moz2-linux-slave18
WinCE	20091019085003	moz2-win32-slave23

Build3:

Platform	Build ID	Build machine
Win32	20091029171059	moz2-win32-slave17
Mac	20091029151354	moz2-darwin9-slave10
Linux	20091029152254	moz2-linux-slave02
WinCE	20091029152627	moz2-win32-slave23

Notes

This release will be done on p-m.

Build 1

• Clean up previous releases build dirs. Worth checking every slave connected to p-m, since they've moved around a bit. Used cssh for linux/mac, RDP as Administrator on win32.
  • Linux: moz2-linux-slave01,02,05-16,18-21,26-28,41,42; rm -rf /builds/slave/{tag,source,linux_build,linux_repack,updates,linux_update_verify,final_verification}/*
  • Mac: bm-xserve11,12,16,18,19; moz2-darwin9-slave05-07,09-12; rm -rf /builds/slave/{macosx_build,macosx_repack,l10n_verification, macosx_update_verify}/*
  • Win32: moz2-win32-slave01,02,05-20,22-31,35-43; rm -rf /e/builds/moz2_slave/{win32_build,win32_repack,win32_update_verify,wince_build}/*
• Got review on the release config patch, landed, reconfig p-m.
• Axel has already provided shipped-locales and l10n-changesets_mozilla-1.9.2 (bug 519821), but may land further updates
• Closed the mozilla-1.9.2 tree
• Sendchange was

buildbot sendchange --username=bhearsum --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6b1 build1" ShakeAndBakeBaby

Tag

• No issues
• Landed Version bump after tagging finished
• Manually tagged tools, buildbot-configs, and buildbotcustom with FIREFOX_3_6b1_RELEASE.
• Re-opened the mozilla-1.9.2 tree

Source

No problems

Build/Repack

• Scheduler didn't fire linux, mac, or win32 builds off properly FIXME; wince fired fine
  • Used 'force build' to kick off remaining en-US builds, no problems with them
  • Kicked l10n builds like this:

# on bhearsum's laptop:
mkdir -p ~/tmp/3.6b1-l10n
cd ~/tmp/3.6b1-l10n
hg clone http://hg.mozilla.org/users/bhearsum_mozilla.com/tools
hg clone http://hg.mozilla.org/build/buildbotcustom
hg clone http://hg.mozilla.org/build/buildbot
cd tools/buildbot-helpers
wget http://people.mozilla.org/~bhearsum/misc/shipped-locales-linux
wget http://people.mozilla.org/~bhearsum/misc/shipped-locales-macosx
wget http://people.mozilla.org/~bhearsum/misc/shipped-locales-win32
# after linux en-US finished:
PYTHONPATH=../../:../../buildbot python force_release_l10n.py -m http://production-master.build.mozilla.org:8010 -v -t FIREFOX_3_6b1_RELEASE -s shipped-locales-linux -n bhearsum
# after mac en-US finished
PYTHONPATH=../../:../../buildbot python force_release_l10n.py -m http://production-master.build.mozilla.org:8010 -v -t FIREFOX_3_6b1_RELEASE -s shipped-locales-macosx -n bhearsum
# after win32 en-US finished
PYTHONPATH=../../:../../buildbot python force_release_l10n.py -m http://production-master.build.mozilla.org:8010 -v -t FIREFOX_3_6b1_RELEASE -s shipped-locales-win32 -n bhearsum

• moz2-win32-slave06 had an error pulling en-US source for l10n, which blew out building lt pt-BR sl sq sv-SE zh-CN zh-TW. Updated (local copy of) shipped-locales-win32 and kicked new builds.
• manual verification that all l10n builds present

# stage
cd /home/ftp/pub/firefox/nightly/3.6b1-candidates/build1
# each should be 53 = 52 l10n builds + en-US
for p in linux-i686 mac unsigned/win32; do
  echo `ls $p | grep -v xpi | wc -l` $p
done

• Removed en-US.xpi from the 3.6b1-candidates/build1/ directory on stage

Sign

• instructions
• set CONCURRENCY=1 to work around bug 521080
• the FtpPoller was still looking at the 3.6a2-candidates/build2/ directory subsequent steps were forced manually

L10nVerify

This will run, but is of no use to us because it's the first localized release. That said the diff between 3.6b1 en-US and random locales look sane.

FIXME Hit an error in unmounting the 3.6b1 en-US dmg (the first one we process)

installing firefox-3.6b1-build1/mac/en-US/Firefox 3.6 Beta 1.dmg
hdiutil: couldn't unmount "disk1" - error 49153
rm: mnt/ : Read-only file system
rm: mnt/.background/background.png: Read-only file system
....

It seems harmless - couldn't reproduce on my Mac doing typical user operations, and l10n repacks worked fine. Something to follow up on before the next release.

Generate updates and push betatest snippets

The automation generated updates for win32/mac/linux-i686 as normal. It failed to bump moz192-firefox-wince.cfg, but that's OK because we can't do an automated check anyway, and failure in bump_verify_configs_4 can be ignored. FIXME We should disable this bump and the wince_update_verify builder.

Did hit the backupsnip problem - bug 521139. Verified 20091014-1-pre-20091014-Firefox-3.6b1-test.tar.bz2 is a valid backup by unpacking and diffing against production snippets.

WinCE updates required some manual work. Logged on to the slave that generated the updates (moz2-linux-slave20), and

cd /builds/slave/updates/build/patcher-configs

Constructed a patch, updating the wince-arm buildID for the real value. Applied that patch locally to moz192-branch-patcher2.cfg. Then

cd /builds/slave/updates/build
perl patcher2.pl --download --app=firefox --brand=Firefox --config=patcher-configs/moz192-branch-patcher2.cfg
perl patcher2.pl --create-patches --partial-patchlist-file=patchlist.cfg --app=firefox --brand=Firefox --config=patcher-configs/moz192-branch-patcher2.cfg
find temp/firefox/3.6a2-3.6b1/ftp/firefox/nightly/3.6b1-candidates/build1 -type f -exec chmod 644 {} \;
find temp/firefox/3.6a2-3.6b1/ftp/firefox/nightly/3.6b1-candidates/build1 -type d -exec chmod 755 {} \;
cd /builds/slave/updates/build/temp/firefox/3.6a2-3.6b1/ftp/firefox/nightly/3.6b1-candidates/build1
rsync -av -e 'ssh -oIdentityFile=~/.ssh/ffxbld_dsa' --exclude=*complete.mar update ffxbld@stage.mozilla.org:/pub/mozilla.org/firefox/nightly/3.6b1-candidates/build1/
cd /builds/slave/updates/build/temp/firefox/3.6a2-3.6b1
rsync -av aus2/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20091014-Firefox-3.6b1-from-3.6a2
rsync -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20091014-Firefox-3.6b1-from-3.6a2-test

Pushed all updates live

# cltbld@aus2-staging
~/bin/pushsnip 20091014-Firefox-3.6b1-test
~/bin/pushsnip 20091014-Firefox-3.6b1-from-3.6a2-test

Update verify

The update verify bumper doesn't handle the branding switch (from unofficial/Namoroka to official/Firefox).

Landed this patch in the build/tools repo after updates bumps the verify configs. The FtpPoller fail means we don't have to clean up verify runs with wonky configs.

Got the expected diffs in removed-files for partial updates on linux/mac, particularly on mac which now combines components/*.xpt like the other platforms did. We exclude removed-files from the partial mar and I think that diff is harmless. On windows the partial log will has lots of

EXECUTE PATCH c
EXECUTE PATCH c
EXECUTE PATCH d

This was fixed by bug 508746 between a1 and b1, so will show up b1->next.release update verify.

The usual chk file differences on win32, now three of them with the addition of nssdbm3.chk.

Respin declared

Build 2

• Clean up previous releases build dirs, just the slaves we used for build 1. Used cssh for linux/mac, RDP as Administrator on win32.
  • Linux: moz2-linux-slave[1,2,5-10,20,41]; rm -rf /builds/slave/{tag,source,linux_build,linux_repack,updates,linux_update_verify,final_verification}/*
  • Mac: bm-xserve[11,12]; moz2-darwin9-slave[5-7,9-12]; rm -rf /builds/slave/{macosx_build,macosx_repack,l10n_verification, macosx_update_verify}/*
  • Win32: moz2-win32-slave[1,2,5-10,41]; rm -rf /e/builds/moz2_slave/{win32_build,win32_repack,win32_update_verify,wince_build}/*
• Updated release automation config, got review, landed and reconfiged pm
• Closed the mozilla-1.9.2 tree
• Sendchange is

buildbot sendchange --username=bhearsum --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6b1 build2" ShakeAndBakeBaby

Tag

• No issues
• Manually tagged tools, buildbot-configs, and buildbotcustom with FIREFOX_3_6b1_RELEASE and FIREFOX_3_6b1_BUILD2.
• Re-opened the mozilla-1.9.2 tree
• First time through, screwed up and forgot to reconfig before kicking automation. This ended up creating a new relbranch and moving the _BUILD1 release tag to it. To recover, did the following:
• Get the tags back into the right place:

hg clone http://hg.mozilla.org/release/mozilla-1.9.2
cd mozilla-1.9.2
hg tag -r c60105da01a0 -f FIREFOX_3_6b1_RELEASE
hg tag -r c60105da01a0 -f FIREFOX_3_6b1_BUILD1
hg push ssh://hg.mozilla.org/releases/mozilla-1.9.2

• Cleaned up slaves (because source, and builds had already started)
  • tag dir on linux-slave41, source on linux-slave11, linux_build on linux-slave08, macosx_build on darwin9-slave09, win32_build on win32-slave16, wince_build on win32-slave43.
• Did a 'buildbot reconfig"
• Sendchange was

buildbot sendchange --username=bhearsum --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6b1 build2-2" ShakeAndBakeBaby

Source

No problems

Build/Repack

• Scheduler didn't fire linux, mac, or win32 builds off properly FIXME; wince fired fine
  • Used 'force build' to kick off remaining en-US builds
  • Kicked l10n builds like this:

# on bhearsum's laptop:
mkdir -p ~/tmp/3.6b1-l10n
cd ~/tmp/3.6b1-l10n
hg clone http://hg.mozilla.org/users/bhearsum_mozilla.com/tools
hg clone http://hg.mozilla.org/build/buildbotcustom
hg clone http://hg.mozilla.org/build/buildbot
cd tools/buildbot-helpers
wget http://people.mozilla.org/~bhearsum/misc/shipped-locales-linux
wget http://people.mozilla.org/~bhearsum/misc/shipped-locales-macosx
wget http://people.mozilla.org/~bhearsum/misc/shipped-locales-win32
# after linux en-US finished:
PYTHONPATH=../../:../../buildbot python force_release_l10n.py -m http://production-master.build.mozilla.org:8010 -v -t FIREFOX_3_6b1_RELEASE -s shipped-locales-linux -n bhearsum
# after mac en-US finished
PYTHONPATH=../../:../../buildbot python force_release_l10n.py -m http://production-master.build.mozilla.org:8010 -v -t FIREFOX_3_6b1_RELEASE -s shipped-locales-macosx -n bhearsum
# after win32 en-US finished
PYTHONPATH=../../:../../buildbot python force_release_l10n.py -m http://production-master.build.mozilla.org:8010 -v -t FIREFOX_3_6b1_RELEASE -s shipped-locales-win32 -n bhearsum

• First try of linux repacks failed out because the slaves weren't cleaned up (look's like they weren't cleaned after they were inadvertently kicked as part of 3.5.4). Had to clean up slaves and restart them.
• manual verification that all l10n builds present

# stage
cd /home/ftp/pub/firefox/nightly/3.6b1-candidates/build2
# each should be 53 = 52 l10n builds + en-US
for p in linux-i686 mac unsigned/win32; do
  echo `ls $p | grep -v xpi | wc -l` $p
done

• Removed en-US.xpi from the 3.6b1-candidates/build1/ directory on stage

Sign

• instructions
• was supposed to set CONCURRENCY=1 to work around bug 521080, but didn't. everything worked fine still

L10nVerify

Didn't bother.

Generate updates and push betatest snippets

Got a slave that hadn't connected to cvs.m.o before and it failed to checkout patcher configs. FIXED Made sure that all slaves had accepted signature (moz2-linux-slave18 and pretty much everything above 40).

The automation generated updates for win32/mac/linux-i686 as normal. It failed to bump moz192-firefox-wince.cfg, but that's OK because we can't do an automated check anyway, and failure in bump_verify_configs_4 can be ignored. FIXME We should disable this bump and the wince_update_verify builder.

Would have hit the backupsnip problem - bug 521139. Interrupted the updates run when it started backupsnip and ran it manually. Note that the job keeps running on aus2-staging anyway so you have to clean up.

WinCE updates required some manual work. Logged on to the slave that generated the updates (moz2-linux-slave19), and

cd /builds/slave/updates/build/patcher-configs

Constructed a patch, and applied that locally to moz192-branch-patcher2.cfg. Then

cd /builds/slave/updates/build
perl patcher2.pl --download --app=firefox --brand=Firefox --config=patcher-configs/moz192-branch-patcher2.cfg
perl patcher2.pl --create-patches --partial-patchlist-file=patchlist.cfg --app=firefox --brand=Firefox --config=patcher-configs/moz192-branch-patcher2.cfg
find temp/firefox/3.6a2-3.6b1/ftp/firefox/nightly/3.6b1-candidates/build2 -type f -exec chmod 644 {} \;
find temp/firefox/3.6a2-3.6b1/ftp/firefox/nightly/3.6b1-candidates/build2 -type d -exec chmod 755 {} \;
cd /builds/slave/updates/build/temp/firefox/3.6a2-3.6b1/ftp/firefox/nightly/3.6b1-candidates/build2
rsync -av -e 'ssh -oIdentityFile=~/.ssh/ffxbld_dsa' --exclude=*complete.mar update ffxbld@stage.mozilla.org:/pub/mozilla.org/firefox/nightly/3.6b1-candidates/build2/
cd /builds/slave/updates/build/temp/firefox/3.6a2-3.6b1
rsync -av aus2/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20091019-Firefox-3.6b1-from-3.6a2
rsync -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20091019-Firefox-3.6b1-from-3.6a2-test

Pushed all updates live

# cltbld@aus2-staging
~/bin/backupsnip 20091019-Firefox-3.6b1-test
~/bin/pushsnip   20091019-Firefox-3.6b1-test
~/bin/pushsnip   20091019-Firefox-3.6b1-from-3.6a2-test

Update verify

As above, the update verify bumper doesn't handle the branding switch (from unofficial/Namoroka to official/Firefox).

Landed this patch in the build/tools repo after updates bumps the verify configs. The FtpPoller fail means we don't have to clean up verify runs with wonky configs.

Got the expected diffs in removed-files for partial updates on linux/mac, particularly on mac which now combines components/*.xpt like the other platforms did. We exclude removed-files from the partial mar and I think that diff is harmless. On windows the partial log will has lots of

EXECUTE PATCH c
EXECUTE PATCH c
EXECUTE PATCH d

This was fixed by bug 508746 between a1 and b1, so will show up b1->next.release update verify.

The usual chk file differences on win32, now there are three of them with the addition of nssdbm3.chk. Noticed that they don't cause the run to fail though, because diff says

Files source/bin/freebl3.chk and target/bin/freebl3.chk differ

instead of

Binary files source/bin/freebl3.chk and target/bin/freebl3.chk differ

and we only WARN on the former. Possibly fallout from the MozillaBuild upgrade. FIXME??

Update Bouncer

This will be slightly usual because we may be shipping WinCE too. These extra entries were created

Product				OS	Path
Firefox-3.6b1			wince	/firefox/releases/3.6b1/wince-arm/en-US/firefox-3.6b1.cab
Firefox-3.6b1-Complete		wince	/firefox/releases/3.6b1/update/wince-arm/en-US/firefox-3.6b1.complete.mar
Firefox-3.6b1-Partial-3.6a2	wince	/firefox/releases/3.6b1/update/wince-arm/en-US/firefox-3.6a2-3.6b1.partial.mar

Note that all the other partials use the Firefox-3.6b1-Partial-3.6a1 product; Firefox-3.6b1-Partial-3.6a2 is a one-off.

RESPIN DECLARED

Build 3

• Clean up previous releases build dirs, just the slaves we used for build 1. Used cssh for linux/mac, RDP as Administrator on win32.
  • Linux: moz2-linux-slave[01,02,05-10,11,18,41,42]; rm -rf /builds/slave/{tag,source,linux_build,linux_repack,updates,linux_update_verify,final_verification}/*
  • Mac: bm-xserve[07,11]; moz2-darwin9-slave[2,5-7,9-10]; rm -rf /builds/slave/{macosx_build,macosx_repack,l10n_verification, macosx_update_verify}/*
  • Win32: moz2-win32-slave[1,2,5-10,16,23,38,40,43]; rm -rf /e/builds/moz2_slave/{win32_build,win32_repack,win32_update_verify,wince_build}/*
• Updated release automation config, got review, landed and reconfiged pm
• Closed the mozilla-1.9.2 tree
• Sendchange is

buildbot sendchange --username=nthomas --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6b1 build3" ShakeAndBakeBaby

Tag

• No issues
• Manually tagged tools, buildbot-configs, and buildbotcustom with FIREFOX_3_6b1_RELEASE and FIREFOX_3_6b1_BUILD3.
• Re-opened the mozilla-1.9.2 tree

Source

• No problems

Build/Repack

• Schedulers fired all four en-US builds and l10n this time
• moz2-win32-slave05 ran out of disk just prior to win32_repack kicking off, and then got lots of repack jobs. The build/tools repo pull failed and 15 repacks stopped when clean_old_builds could not run. Retrigger the affected locales (af ar bn-BD cs es-ES et eu gl id it mk nb-NO nl pa-IN ru)

# on nthomas's laptop:
mkdir -p ~/tmp/3.6b1-l10n
cd ~/tmp/3.6b1-l10n
hg clone http://hg.mozilla.org/build/tools
hg clone http://hg.mozilla.org/build/buildbotcustom
hg clone http://hg.mozilla.org/build/buildbot
cd tools/buildbot-helpers
# create shipped-locales-win32, one line for each <locale> like this "<locale> win32"
PYTHONPATH=../../:../../buildbot python force_release_l10n.py -m http://production-master.build.mozilla.org:8010 -v -t FIREFOX_3_6b1_RELEASE -s shipped-locales-win32 -n nthomas

• Manual verification that all l10n builds present

# stage
cd /home/ftp/pub/firefox/nightly/3.6b1-candidates/build3
# each should be 53 = 52 l10n builds + en-US
for p in linux-i686 mac unsigned/win32; do
  echo `ls $p | grep -v xpi | wc -l` $p
done

• Removed en-US.xpi from the 3.6b1-candidates/build1/ directory on stage

Sign

• instructions
• set CONCURRENCY=1 to work around bug 521080
• FtpPoller looking at 3.6b1-candidates/build2/ so no further builders triggered automatically

L10nVerify

• Didn't bother

Generate updates and push betatest snippets

Once forced, the automation generated updates for win32/mac/linux-i686 as normal. It failed to bump moz192-firefox-wince.cfg, but that's OK because we can't do an automated check anyway, and failure in bump_verify_configs_4 can be ignored. FIXME We should disable this bump and the wince_update_verify builder.

Locally modified ~/bin/pushsnip to make the tar command verbose verbose, and pushsnip ran to completion in 75 minutes. Don't yet know if this was sufficient or something else changed behind the curtain.

Since we're not shipping WinCE we don't bother generating update snippets for it like we did for the previous two builds.

Update verify

As above, the update verify bumper doesn't handle the branding switch (from unofficial/Namoroka to official/Firefox). Landed this patch in the build/tools repo after updates bumps the verify configs. The FtpPoller fail means we don't have to clean up verify runs with wonky configs.

Same results as build2.

Update Bouncer

Already done for build2, but WinCE config removed as it's unneeded.

Final checks before push

cd /pub/mozilla.org/firefox/nightly/3.6b1-candidates/build3/
find . ! -user ffxbld
find . ! -group firefox
find . -type f ! -perm 644
find . -type d -mindepth 1 ! -perm 755 ! -name 'contrib*'
find . -type d -maxdepth 1 ! -perm 2775  -name 'contrib*'

Should only return Solaris builds (different user).

Push to mirrors

(done by joduinn)

# ffxbld@stage
rsync -av --exclude=*.log --exclude=*.txt --exclude=*wince-arm* --exclude=*unsigned* /pub/mozilla.org/firefox/nightly/3.6b1-candidates/build3/ /pub/mozilla.org/firefox/releases/3.6b1/
...
sent 4587781816 bytes  received 14058 bytes  5618855.94 bytes/sec
total size is 4587169157  speedup is 1.00

We don't need to put this release in the mozilla-current module.

(/done by joduinn)

Final Verification

All green

Push updates to release channel

Start doing backupsnip a few hours at least before the release time

# cltbld@aus2-staging
time ~/bin/backupsnip 20091019-Firefox-3.6b1
Running /bin/tar cfvj /opt/aus2/snippets/backup/20091030-3-pre-20091019-Firefox-3.6b1.tar.bz2 .
real    71m41.733s
user    0m56.061s
sys     1m44.338s

Should have been 20091029-Firefox-3.6b1 so tidy up the backup file's name

mv -v /opt/aus2/snippets/backup/20091030-3-pre-200910{1,2}9-Firefox-3.6b1.tar.bz2

After go from drivers

time ~/bin/pushsnip 20091029-Firefox-3.6b1

Release

No symlink to update until final comes along.

XULRunner build

Created the builds by going to the buildbot waterfall for mozilla-1.9.2 and forcing builds for linux (not x64), mac and windows xulrunner. Set the branch to 'releases/mozilla-1.9.2' and revision to 'FIREFOX_3_6b1_RELEASE'.

When they finished, as xrbld@stage moved /pub/mozilla.org/xulrunner/nightly/latest-mozilla-1.9.2/*1.9.2b1.* to ..../nightly/1.9.2b1-candidates/build3/. Contacted mfinkle and dtownsend for smoketest QA.

# cltbld@stage
VERSION=1.9.2b1
FX_VERSION=3.6b1
BUILD=3

mkdir -p ~/xulrunner-${VERSION}/batch1/stage-unsigned/{runtimes,source,sdk}
cd ~/xulrunner-${VERSION}/batch1/stage-unsigned
rsync -av --exclude=*sdk* --exclude=*info* \
  /home/ftp/pub/xulrunner/nightly/${VERSION}-candidates/build${BUILD}/ runtimes/
rsync -av --include=*sdk* --exclude=* \
  /home/ftp/pub/xulrunner/nightly/${VERSION}-candidates/build${BUILD}/ sdk/
cp -pv /pub/mozilla.org/firefox/releases/${FX_VERSION}/source/firefox-${FX_VERSION}.source.tar.bz2 \
  source/xulrunner-${VERSION}.source.tar.bz2
cd ~/xulrunner-${VERSION}/batch1
rsync -av stage-unsigned/ stage-signed/

Then create detached signatures per usual process (with PRODUCT=xulrunner, VERSION=1.9.2b1, don't need BUILD, TAG, or KEYDIR), skip the win32 signing section, do pgp, verify and upload. Back on stage:

cd ~/xulrunner-${VERSION}/batch1/stage-signed
cp /pub/mozilla.org/firefox/releases/${FX_VERSION}/KEY .
~/bin/checksum-files .
chown -R cltbld:xulrunner .
find . -type f -exec chmod -v 644 {} \;
find . -type d -exec chmod -v 755 {} \;
cd ~/xulrunner-${VERSION}/batch1
rsync -nav stage-signed/* /home/ftp/pub/xulrunner/releases/${VERSION}/

If all is fine then

rsync -av stage-signed/* /home/ftp/pub/xulrunner/releases/${VERSION}/

For this first 1.9.2 XULRunner relase, this page was created

• Release Notes

For subsequent 1.9.2 releases we'll update all the links for the version change.

Once we hit 3.6 final we'll add 1.9.2 to

• Gecko_SDK