Notes About Releasing

Please update the Notes Template and the Release:Primer for future releases (bug fixes, changes to automation) as needed

Bugs hit

Enter any bugs pre-existing or newly discovered and filed during the release:

bug 737809 - release sanity should check if the current release is runnable on the current master
bug 737812 - post_upload.py shouldn't override release files

Build Engineers

nthomas - Tracking bug: bug 735383

Signed-off Revision(s)

Build 1: 74655b354316

L10N changesets

Instructions on how to get them

Firefox: l10n-changesets
Fennec: empty

Build data

Firefox

Build #	Type	Build ID	Build machine	Time to build
1	Linux	20120321033733	mv-moz2-linux-ix-slave11	1 hrs, 10 mins, 38 secs
	Linux64		linux64-ix-slave05	1 hrs, 8 mins, 36 secs
	Mac		moz2-darwin10-slave17	3 hrs, 25 mins, 48 secs
	Windows		mw32-ix-slave12	3 hrs, 23 mins, 7 secs

Fennec

Build #	Type	Build ID	Build machine	Time to build
1	android-xul	20120321033734	mv-moz2-linux-ix-slave19	53 mins, 27 secs

Notes

Build 1

Preparing to start Automation

Set clobbers for any master, mozilla-beta, any builder
Reserve slaves

[cltbld@buildbot-master08 master]$ pwd
/builds/buildbot/build1/master
[cltbld@buildbot-master08 master]$ cat reserved_slaves_bm08-build1 
8

For build1 only, create a symlink for your $version (eg 11.0b4, 10.0.2esr, 10.0.2, 3.6.27) (until bug 725839 is resolved)

# ffxbld@stage
version=12.0b2
cd /pub/mozilla.org/firefox/nightly/
mkdir ../candidates/$version-candidates
ln -s ../candidates/$version-candidates $version-candidates

Update l10n changesets for desktop (mobile json was unchanged as {})

curl -sL https://l10n-stage-sj.mozilla.org/shipping/l10n-changesets?ms=fx12_beta_b2 > l10n-changesets_mozilla-beta

Double landed the automation configs
Tagged buildbot-configs, buildbotcustom, & tools with build & release tags.

hg tag {FENNEC,FIREFOX}_12_0b2_{RELEASE,BUILD1}

updated and reconfigured bm08 (actually bm07, as it turns out)
ran release sanity in test mode:

cd /builds/buildbot/build1/master
source ../bin/activate
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u nthomas -pfirefox,fennec -V 12.0b2 --branch mozilla-beta --build-number 1 -c release-firefox-mozilla-beta.py -c release-fennec-mozilla-beta.py --dryrun localhost:9001

Returning mozconfig errors:

INFO : Comparing fennec mozconfigs to nightly mozconfigs...
ERROR : found in mobile/android/config/mozconfigs/android/release but not in mobile/android/config/mozconfigs/android/nightly: ac_add_options --with-android-sdk="/tools/android-sdk-r15/platforms/android-14"
WARNING : ac_add_options --with-android-sdk="/tools/android-sdk-r13/platforms/android-13" not in android ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --enable-update-channel=${MOZ_UPDATE_CHANNEL}']!
ERROR : found in mobile/android/config/mozconfigs/android/nightly but not in mobile/android/config/mozconfigs/android/release: ac_add_options --with-android-sdk="/tools/android-sdk-r13/platforms/android-13"
ERROR : found in mobile/android/config/mozconfigs/android/release but not in mobile/android/config/mozconfigs/android/nightly: ac_add_options --with-android-tools="/tools/android-sdk-r15/tools"
WARNING : ac_add_options --with-android-tools="/tools/android-sdk-r13/tools" not in android ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --enable-update-channel=${MOZ_UPDATE_CHANNEL}']!
ERROR : found in mobile/android/config/mozconfigs/android/nightly but not in mobile/android/config/mozconfigs/android/release: ac_add_options --with-android-tools="/tools/android-sdk-r13/tools"
ERROR : Error verifying mozconfigs

INFO : Comparing firefox mozconfigs to nightly mozconfigs...
WARNING : ac_add_options --enable-signmar not in macosx64 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --with-macbundlename-prefix=Firefox', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --disable-install-strip']!
ERROR : found in browser/config/mozconfigs/macosx-universal/nightly but not in browser/config/mozconfigs/macosx-universal/release: ac_add_options --enable-signmar
WARNING : ac_add_options --enable-js-diagnostics not in macosx64 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --with-macbundlename-prefix=Firefox', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --disable-install-strip']!
ERROR : found in browser/config/mozconfigs/macosx-universal/nightly but not in browser/config/mozconfigs/macosx-universal/release: ac_add_options --enable-js-diagnostics
WARNING : ac_add_options --enable-signmar not in win32 ['ac_add_options --enable-update-channel=nightly', '. $topsrcdir/configs/mozilla2/win32/include/choose-make-flags', 'mk_add_options MOZ_MAKE_FLAGS=-j1']!
ERROR : found in browser/config/mozconfigs/win32/nightly but not in browser/config/mozconfigs/win32/release: ac_add_options --enable-signmar
WARNING : ac_add_options --enable-js-diagnostics not in win32 ['ac_add_options --enable-update-channel=nightly', '. $topsrcdir/configs/mozilla2/win32/include/choose-make-flags', 'mk_add_options MOZ_MAKE_FLAGS=-j1']!
ERROR : found in browser/config/mozconfigs/win32/nightly but not in browser/config/mozconfigs/win32/release: ac_add_options --enable-js-diagnostics
WARNING : ac_add_options --enable-signmar not in linux64 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --with-ccache=/usr/bin/ccache', 'export MOZILLA_OFFICIAL=1', 'export MOZ_TELEMETRY_REPORTING=1', "mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) @MOZ_OBJDIR@/_profile/pgo/profileserver.py 10'"]!
ERROR : found in browser/config/mozconfigs/linux64/nightly but not in browser/config/mozconfigs/linux64/release: ac_add_options --enable-signmar
WARNING : ac_add_options --enable-js-diagnostics not in linux64 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --with-ccache=/usr/bin/ccache', 'export MOZILLA_OFFICIAL=1', 'export MOZ_TELEMETRY_REPORTING=1', "mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) @MOZ_OBJDIR@/_profile/pgo/profileserver.py 10'"]!
ERROR : found in browser/config/mozconfigs/linux64/nightly but not in browser/config/mozconfigs/linux64/release: ac_add_options --enable-js-diagnostics
ERROR : found in browser/config/mozconfigs/linux64/release but not in browser/config/mozconfigs/linux64/nightly: ac_add_options --enable-warnings-as-errors
WARNING : ac_add_options --enable-warnings-as-errors not in linux64 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --with-ccache=/usr/bin/ccache', 'export MOZILLA_OFFICIAL=1', 'export MOZ_TELEMETRY_REPORTING=1', "mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) @MOZ_OBJDIR@/_profile/pgo/profileserver.py 10'"]!
ERROR : found in browser/config/mozconfigs/linux64/nightly but not in browser/config/mozconfigs/linux64/release: ac_add_options --enable-warnings-as-errors
WARNING : ac_add_options --enable-signmar not in linux ['ac_add_options --enable-update-channel=nightly', 'CXX=$REAL_CXX', 'CXX="ccache $REAL_CXX"', 'CC="ccache $REAL_CC"', 'mk_add_options PROFILE_GEN_SCRIPT=@TOPSRCDIR@/build/profile_pageloader.pl', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --with-ccache=/usr/bin/ccache', 'export MOZILLA_OFFICIAL=1', 'export MOZ_TELEMETRY_REPORTING=1', "mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) @MOZ_OBJDIR@/_profile/pgo/profileserver.py 10'"]!
ERROR : found in browser/config/mozconfigs/linux32/nightly but not in browser/config/mozconfigs/linux32/release: ac_add_options --enable-signmar
WARNING : ac_add_options --enable-js-diagnostics not in linux ['ac_add_options --enable-update-channel=nightly', 'CXX=$REAL_CXX', 'CXX="ccache $REAL_CXX"', 'CC="ccache $REAL_CC"', 'mk_add_options PROFILE_GEN_SCRIPT=@TOPSRCDIR@/build/profile_pageloader.pl', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --with-ccache=/usr/bin/ccache', 'export MOZILLA_OFFICIAL=1', 'export MOZ_TELEMETRY_REPORTING=1', "mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) @MOZ_OBJDIR@/_profile/pgo/profileserver.py 10'"]!
ERROR : found in browser/config/mozconfigs/linux32/nightly but not in browser/config/mozconfigs/linux32/release: ac_add_options --enable-js-diagnostics
ERROR : found in browser/config/mozconfigs/linux32/release but not in browser/config/mozconfigs/linux32/nightly: ac_add_options --enable-warnings-as-errors
WARNING : ac_add_options --enable-warnings-as-errors not in linux ['ac_add_options --enable-update-channel=nightly', 'CXX=$REAL_CXX', 'CXX="ccache $REAL_CXX"', 'CC="ccache $REAL_CC"', 'mk_add_options PROFILE_GEN_SCRIPT=@TOPSRCDIR@/build/profile_pageloader.pl', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --with-ccache=/usr/bin/ccache', 'export MOZILLA_OFFICIAL=1', 'export MOZ_TELEMETRY_REPORTING=1', "mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) @MOZ_OBJDIR@/_profile/pgo/profileserver.py 10'"]!
ERROR : found in browser/config/mozconfigs/linux32/nightly but not in browser/config/mozconfigs/linux32/release: ac_add_options --enable-warnings-as-errors
ERROR : Error verifying mozconfigs

which is all bug 737631.

start automation with -b to bypass failing checks

PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u nthomas -pfirefox,fennec -V 12.0b2 --branch mozilla-beta --build-number 1 -c release-firefox-mozilla-beta.py -c release-fennec-mozilla-beta.py -b localhost:9001

E-mail Metrics

Mailed "metrics-alerts < AT > mozilla < PERIOD > org"

Firefox,firefox,firefox,12.0b2,12.0
Fennec,firefox-mobile,fennec,12.0b2,12.0

Tag

No problems

Bouncer Submitter

Didn't run

Source

Used FIREFOX_12_0b1_RELEASE code tag

Build

en-US built green, but used FIREFOX_12_0b1_RELEASE code. This was because I had updated the repos and reconfiged buildbot-master07 when the release would run on buildbot-master08.

All the l10n repacks blew up, or were canceled while pending.

Build 1 (for real)

Preparing to start Automation

Recovering from the foobar

spot checks of en-US and a few l10n repos showed tagging did work properly. Creating the tags in the right place.
the source and builds are worthless and got uploaded to the wrong place, so fix up 12.0b1 so we get the right partials

# ffxbld@stage
cd /pub/mozilla.org/firefox/candidates/12.0b1-candidates
rsync -a build2/ build2.dirty/
rsync -av --exclude=contrib* /pub/mozilla.org/firefox/releases/12.0b1/ build2/
# output: http://pastebin.mozilla.org/1529366
rm -v build2/jsshell*
for f in build2/*info.txt; do
   echo 20120314195616 > $f
done
# this still leaves */en-US/{*tests.zip,*crashreporter-symbols.zip,*.txt,*checksums wrong :-(

can't recover mobile/candidates/12.0b1-candidates/build2 or xulrunner equivalent, left README's
redo build1 with releaseConfig['skip_tag'] = 1 added to the release configs, retagged buildbot-configs
fix skip_tag so that it has the required properties to pass checkconfig
update bm-08, checkconfig, reconfig
release sanity dry run as above, same mozconfig errors
start automation as above
had forgotten to clobber, so cancel jobs, clobber Any/mozilla-beta/Any, and then restart again

Tag

skipped

Bouncer Submitter

passed on the first run, failed on the post-clobber one. Locations look fine in bouncer

Source

no problems

Build

Firefox

no problems

Firefox repacks

no problems

Fennec

android-xul ok

Fennec repacks

N/A

Unittests / Talos

Look to see that they ran
Document any oranges (per platform) for unittests - if possible, try to confirm it's known/random
Make sure there's no red/failures that we need to have a dev look at

XULRunner

Source

no problems

Build

no problems

Signing

Follow instructions in XULRunner Signing (auth required).

Edited ~/sign_xulrunner_beta.sh with 12.0b1 values and ran it.

Firefox Signing

N/A, done at build time

Android Signing

Instructions in MobileSigning. No problems

L10N verify

no problems

Checksums

no problems

Updates

Failed to pull buildIDs from info.txt files for 12.0b1, because they'd been recreated incorrectly
- Fixed those files by hand, not sure how they got into that state
- 12.0b2 info files are OK, so 12.0b3 won't be affected.
fourth updates job worked

Update verify

no problems

Reset reserved slaves

Set to 0

Check permissions / AV scan

no problems

Push to internal mirrors

no problems

Final verification

no problems

Run backupsnip

Ran

~/bin/backupsnip Firefox-12.0b2-build1

Release Day

Push to beta

~/bin/pushsnip Firefox-12.0b2-build1

Took 30 mins

Publish Fennec to the Android Market

Push the files

Use this script and run it as ffxbld@stage (please push you changes prior to using it)

Android Market

IMPORTANT - Make sure to follow instructions!

download multi apk (e.g. multi/fennec-12.0b2.multi.android-arm.apk)
visit https://market.android.com/publish
choose "Firefox Beta" or "Firefox"
select the "APK Files" tab and choose "Upload APK"
- choose the multi apk that you downloaded and hit "upload"
  - NOTE: the progress bar does not seem to show progress (at least on Google Chrome)
- when it finishes uploading, verify VersionCode is the build date (almost anyway, the market reports an hour later than our buildID)
- hit "Save"
activate new apk (You should see an Error message since both apk's are active)
deactivate old apk (You should not see the Error message anymore)
hit "Save"
If this is a beta 1, go to "Product Details" tab -> Recent Changes, change the url to http://www.mozilla.com/en-US/mobile/{VERSION}beta/releasenotes/
- and hit "Save"

Push XULRunner to Mirrors

# xrbld@stage
rsync  -av --exclude=*.txt --exclude=*unsigned* --exclude=*-symbols.zip \
 --exclude=*.checksums --exclude=jsshell* --exclude=logs \
 /pub/mozilla.org/xulrunner/nightly/12.0b2-candidates/build1/ \
 /pub/mozilla.org/xulrunner/releases/12.0b2/

Build #	Branch, Tags	Changeset
1	MOBILE120_2012032021_RELBRANCH, FENNEC_12_0b2_BUILD1, FENNEC_12_0b2_RELEASE	ff5b5a7cfbd8
1	GECKO120_2012032021_RELBRANCH, FIREFOX_12_0b2_BUILD1 FIREFOX_12_0b2_RELEASE	097049b2438f

Releases/Firefox 12.0b2/BuildNotes

Contents

Notes About Releasing

Bugs hit

Build Engineers

Signed-off Revision(s)

L10N changesets

Tags

Build data

Firefox

Fennec

Notes

Build 1

Preparing to start Automation

E-mail Metrics

Tag

Bouncer Submitter

Source

Build

Build 1 (for real)

Preparing to start Automation

Tag

Bouncer Submitter

Source

Build

Firefox

Firefox repacks

Fennec

Fennec repacks

Unittests / Talos

XULRunner

Source

Build

Signing

Firefox Signing

Android Signing

L10N verify

Checksums

Updates

Update verify

Reset reserved slaves

Check permissions / AV scan

Push to internal mirrors

Final verification

Run backupsnip

Release Day

Push to beta

Publish Fennec to the Android Market

Push the files

Android Market

Push XULRunner to Mirrors

Navigation menu

Search