Notes About Releasing

Please update the Notes Template and the Release:Primer for future releases (bug fixes, changes to automation) as needed

Bugs hit

Enter any bugs pre-existing or newly discovered and filed during the release:

• bug 732516 - ran out of disk on win32 l10n verify builder, patch landed but not merged to production
• bug 733871 - pushsnip failed to sync PHX

Build Engineers

John Ford, Hal Wine - Tracking bug: bug 729422

Signed-off Revision(s)

Build 1: 7b1df36b517c

L10N changesets

Instructions on how to get them

• Firefox: link to the l10n-changesets
• Fennec: link to the json

Tags

Build #	Branch, Tags	Changeset
	MOBILE110_2012030517_RELBRANCH, FENNEC_11_0b6_BUILD1 FENNEC_11_0b6_RELEASE	377c06cd1151
	GECKO110_2012030517_RELBRANCH, FIREFOX_11_0b6_BUILD1 FIREFOX_11_0b6_RELEASE	d1f6bc5e555a

Build data

Firefox

Build #	Type	Build ID	Build machine	Time to build
1	Linux	20120305181207	mv-moz2-linux-ix-slave10	1 hrs, 6 mins, 54 secs
	Linux64		moz2-linux64-slave06	4 hrs, 5 mins, 9 secs
	Mac		moz2-darwin10-slave41	3 hrs, 14 mins, 32 secs
	Windows		mw32-ix-slave03	3 hrs, 27 mins, 46 secs

Fennec

Build #	Type	Build ID	Build machine	Time to build
1	android-xul	20120305175836	mv-moz2-linux-ix-slave08	51 mins, 45 secs

Notes

Build 1

Syncing repos

(eg: mozilla-beta -> mozilla-release for new Firefox releases)

Diff mozconfigs

If this is a *.0 beta -> release. May also need to update the whitelist of nightly vs release differences for release sanity.

Preparing to start Automation

• Set clobbers for any master, mozilla-beta and any builder
• Reserve slaves

[cltbld@buildbot-master08 master]$ pwd
/builds/buildbot/build1/master
[cltbld@buildbot-master08 master]$ echo 8 > reserved_slaves_bm08-build1

• For build1 only, create a symlink for your $version (eg 11.0b4, 10.0.2esr, 10.0.2, 3.6.27) (until bug 725839 is resolved)

# ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/
mkdir ../candidates/11.0b6-candidates
ln -s ../candidates/11.0b6-candidates 11.0b6-candidates

• Update l10n changesets for desktop, mobile
  • firefox: [1]
  • fennec: [2]
• Land automation configs (are you or buildduty going to reconfig? great, merge. otherwise double-land only your configs)
• Tag buildbot-configs, buildbotcustom, & tools with build & release tags

hg tag -f FIREFOX_11_0b6_RELEASE FIREFOX_11_0b6_BUILD1 FENNEC_11_0b6_RELEASE FENNEC_11_0b6_BUILD1 -r 06aec21cda52

• update and reconfigure the master
• start automation

cd /builds/buildbot/build1/master
source ../bin/activate
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u jhford \
 -pfirefox,fennec -V 11.0b6 --branch mozilla-beta --build-number 1 \
 -c release-firefox-mozilla-beta.py -c release-fennec-mozilla-beta.py \
 --dryrun localhost:9001

PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u jhford  -pfirefox,fennec -V 11.0b6 --branch mozilla-beta --build-number 1  -c release-firefox-mozilla-beta.py -c release-fennec-mozilla-beta.py  localhost:9001

E-mail Metrics

Use the address "metrics-alerts < AT > mozilla < PERIOD > org"

For 9.0b2 it would be

Firefox,firefox,firefox,11.0b6,11.0
Fennec,firefox-mobile,fennec,11.0b6,11.0

Edit rsync exclude files

Edit /pub/mozilla.org/zz/rsyncd-mozilla-releases.exclude on stage so that it excludes the current release. This prevents external mirrors from picking up the new release prematurely.

For rapid release betas, this is not necessary as there's already an exclude that matches beta releases.

Tag

no problems

Bouncer Submitter

no problems

Source

no problems

Start autosign

NOTE: use new signcode keys from d:/2011-keys

PRODUCT=firefox
VERSION=11.0b6
BUILD=1
TAG=FIREFOX_11_0b6_RELEASE
REPO='releases/mozilla-beta'
EMAIL=release@mozilla.com
KEYDIR=d:/2011-keys
RELCONFIG=mozilla/release-firefox-mozilla-beta.py

cd ${HOME}/hg-tools/
hg pull && hg up -r ${TAG}


mkdir -p ~/signing-work/${PRODUCT}-${VERSION}/lib/python
cd ~/signing-work/${PRODUCT}-${VERSION}
cp ~/hg-tools/release/signing/* .
rsync -av ~/hg-tools/lib/python/ ./lib/python/
make autosign PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} RELCONFIG=${RELCONFIG}

This wasn't needed

Build

Firefox

Firefox repacks

no problems

Fennec

Fennec repacks

no problems

Unittests / Talos

1. Look to see that they ran
2. Document any oranges (per platform) for unittests - if possible, try to confirm it's known/random
3. Make sure there's no red/failures that we need to have a dev look at

XULRunner

Source

failed once from an unknown cause timeout, retriggered and succeeded

Build

no problem

Signing

no problems

Firefox Signing

Android Signing

no problems

L10N verify

all but windows worked, windows failed due to not enough disk space, was retriggered and ended up passing

Checksums

no problems

Updates

no problems

Update verify

no problems

Reset reserved slaves

This is also a good time to QUIT SIGNCODE on keymaster if you've still got it running.

Check permissions / AV scan

It is supposed to be triggered automatically after updates are done. File a bug if it doesn't.

Push to internal mirrors

This is done by automation for rapid release betas.

Going to Beta

Run backupsnip

NOTE: Remember to do this at least an hour ahead of the expected "go to beta" email.

Push to beta

Before Release Day

Assuming this is not a chemspill.

Run backupsnip for release snippets

#cltbld@aus2-staging
~/bin/backupsnip Firefox-11.0b6-build1

Final verification

Release Day

~/bin/pushsnip Firefox-11.0b6-build1

Received the message:

Updating PHX:
Running time /usr/bin/rsync -e "ssh -i /home/cltbld/.ssh/auspush" -PaO /opt/aus2/snippets/staging/Firefox-11.0b6-build1/ ffxbld@dp-ausstage01.phx.mozilla.com:/opt/aus2/incoming/3
You don't exist, go away!
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(601) [sender=3.0.7]

Publish Fennec to the Android Market

Push the files

• Use this script and run it as ffxbld@stage (please push you changes prior to using it)

Android Market

IMPORTANT - Make sure to follow instructions!

• download multi apk (e.g. multi/fennec-10.0b6.multi.android-arm.apk)
• visit https://market.android.com/publish
• choose "Firefox Beta" or "Firefox"
• select the "APK Files" tab and choose "Upload APK"
  • choose the multi apk that you downloaded and hit "upload"
    • NOTE: the progress bar does not seem to show progress (at least on Google Chrome)
  • when it finishes uploading, verify VersionCode is the build date (almost anyway, the market reports an hour later than our buildID)
  • hit "Save"
• activate new apk (You should see an Error message since both apk's are active)
• deactivate old apk (You should not see the Error message anymore)
• hit "Save"
• If this is a beta 1, go to "Product Details" tab -> Recent Changes, change the url to http://www.mozilla.com/en-US/mobile/{VERSION}beta/releasenotes/
  • and hit "Save"

Push XULRunner to Mirrors

# xrbld@stage
rsync  -av --exclude=*.txt --exclude=*unsigned* --exclude=*-symbols.zip \
 --exclude=*.checksums --exclude=jsshell* \
 /pub/mozilla.org/xulrunner/nightly/11.0b6-candidates/build1/ \
 /pub/mozilla.org/xulrunner/releases/11.0b6/