Build 1

Checklist

~~Ship the l10n milestone ^DESKTOP ^MOBILE ^RELEASE ^BETA~~

Land patches and reconfig^DESKTOP ^MOBILE ^RELEASE ^BETA ^ESR

Start the automation ^DESKTOP ^MOBILE ^RELEASE ^BETA ^ESR

Sign Android builds ^MOBILE ^RELEASE ^BETA

Push to mirrors ^DESKTOP ^RELEASE

Publish Fennec ^MOBILE ^RELEASE ^BETA NB: Push armv6 apk as well, like we do for betas

Run pushsnip ^DESKTOP ^RELEASE ^BETA ^ESR

Update bouncer links ^DESKTOP ^BETA ^RELEASE

Update XULRunner wiki pages ^DESKTOP ^RELEASE

Post-release tasks ^DESKTOP ^RELEASE ^BETA ^ESR

~~Get the Browser Choice website updated ^DESKTOP ^RELEASE~~

bugs hit

bug 826825 Remove mn, sw during beta-release migration for Firefox 18
bug 827224 No multi locale builds for Firefox 18.0
bug 827226 Android signing should exit if downloads fail
bug 830722 xulrunner src and md5checksums are missing in FF 18.0

Notes

Tracking bug 810325 release engineer kmoir

Release was submitted using release-runner

bug 826825 Remove mn, sw during beta-release migration for Firefox 18

A bug wasn't opened to remove the mn and sw locales for the release, so I removed them manually in the changesets submitted to the release runner. However, they still existed in https://hg.mozilla.org/releases/mozilla-release/file/3a7b66445659/browser/locales/shipped-locales so relman had to fix this and issue a new changeset to us. Once this was releasesd, bhearsum retagged our repos and I reran the tagging builds that failed.

kmoir, jhopkins: so, we all need to "force build" our "source", "build", "bouncer submitter", and "xulrunner build" builders (if you have them) because buildbot schedulers are dumb. Firefox (source, build:win32,macosx64,linux,linux64, bouncer submitter, xulrunner: win32,macosx64,linux,linux64) Fennec(source, build, android, android-armv6)

xulrunner_macosx64 had a compile error, bhearsum identfied it as a known intermittent issue and reran the build

Because of rebuilding tag instead of re-doing sendchanges, all repacks are failing because "script_repo_revision" isn't set. These can be fixed by forcing the failed builders and setting a "script_repo_revision" property to the release tag (eg, FIREFOX_18_0_RELEASE). bhearsum did this for all failed repacks, except for Linux and Android.

Linux repacks failed with "cannot create executable" during configure. This is because the gcc path was incorrect. The mozconfigs were updated for official betas recently for mock slaves in bug 798361, but we missed the esr's -release dir's:

Catlee fixed this http://hg.mozilla.org/build/buildbot-configs/rev/e8b7870b1d96

The android repacks were failing because bug 814191 wasn't landed. I didn't realize that this had to be landed before the beta. I landed this patch, catlee landed the buildbot configs and retagged them and restarted the failed repacks.

catlee re-tagged buildbot-configs with:

 FENNEC_18_0_RELEASE
 FENNEC_18_0_BUILD1
 FIREFOX_18_0_RELEASE
 FIREFOX_18_0_BUILD1

catlee re-triggered:

Firefox 18 32/64-bit linux repacks
Fennec 18 repacks

When signing the Android builds, this message appeared

5:18:53    ERROR - Unable to download android:multi unsigned apk!
15:18:53    ERROR - Unable to download android-armv6:multi unsigned apk!
15:18:53    ERROR - Downloaded 29 of 31 unsigned apks successfully.
15:18:53     INFO - Signed 29 of 29 apks successfully.
15:18:53     INFO - Copying logs to upload dir...
15:18:53     INFO - mkdir: /home/cltsign/signing-work/release/upload/logs

Don't recall that from before, not sure if this is normal.

No, it's not, nthomas opened bug 827224

Fennec multilocale

This was caused by a missing --package-multi option in the worst possible place.

Happily, mozharness allows you to run actions separately, and we don't clobber release directories by default.

Looked at the logs; we used bld-centos6-hp-007 and 016 for android-armv6 and android, respectively.
Disabled in slavealloc temporarily
Backed up objdir

# armv6 cltbld@bld-centos6-hp-007
cd /builds/slave/rel-m-rel-andrd-armv6-bld/build
tar czvf obj-firefox.tar.gz obj-firefox

# armv7 cltbld@bld-centos6-hp-016
cd /builds/slave/rel-m-rel-andrd-bld/build
tar czvf obj-firefox.tar.gz obj-firefox

Tried to package-multi (from log, but with actions changed to --package-multi --summary)

mock_mozilla -v -r mozilla-centos6-i386 --cwd /builds/slave/rel-m-rel-andrd-armv6-bld --unpriv --shell '/usr/bin/env MOZ_OBJDIR="obj-firefox" PATH="/tools/buildbot/bin:/usr/local/bin:/bin:/usr/bin" SYMBOL_SERVER_USER="ffxbld" DISPLAY=":2" CCACHE_UMASK="002" SHIP_LICENSED_FONTS="1" MOZ_SYMBOLS_EXTRA_BUILDID="android-armv6" POST_SYMBOL_UPLOAD_CMD="/usr/local/bin/post-symbol-upload.py" SYMBOL_SERVER_PATH="/mnt/netapp/breakpad/symbols_mob/" MOZILLA_REV="FENNEC_18_0_RELEASE" JAVA_HOME="/tools/jdk6" HG_SHARE_BASE_DIR="/builds/hg-shared" LC_ALL="C" SYMBOL_SERVER_HOST="symbols1.dmz.phx1.mozilla.com" TINDERBOX_OUTPUT="1" COMM_REV="FENNEC_18_0_RELEASE" CCACHE_DIR="/builds/ccache" CCACHE_COMPRESS="1" MOZ_UPDATE_CHANNEL="release" SYMBOL_SERVER_SSH_KEY="/home/mock_mozilla/.ssh/ffxbld_dsa" JARSIGNER="/builds/slave/rel-m-rel-andrd-armv6-bld/tools/release/signing/mozpass.py" MOZ_CRASHREPORTER_NO_REPORT="1" python mozharness/scripts/multil10n.py --config-file multi_locale/release_mozilla-release_android-armv6.json --merge-locales --tag-override=FENNEC_18_0_RELEASE --package-multi --summary'

This failed out because of a missing 'aapt' in the mock env. Poked around a bit, figured I needed to recreate the mock environment for Android.

Recreated mock env (copy-pasted from previous log)

# cltbld@bld-centos6-hp-007 and 016
sh -c 'rm -f /builds/mock_mozilla/mozilla-centos6-i386/buildroot.lock; mock_mozilla -v -r mozilla-centos6-i386 --orphanskill'
mock_mozilla -v -r mozilla-centos6-i386 --init
mock_mozilla -r mozilla-centos6-i386 --copyin /home/cltbld/.ssh /home/mock_mozilla/.ssh
mock_mozilla -v -r mozilla-centos6-i386 --cwd / --shell '/usr/bin/env  chown -R mock_mozilla /home/mock_mozilla/.ssh'
mock_mozilla -r mozilla-centos6-i386 --copyin /home/cltbld/.android /builds/.android
mock_mozilla -v -r mozilla-centos6-i386 --cwd / --shell '/usr/bin/env  chown -R mock_mozilla /builds/.android'
mock_mozilla -r mozilla-centos6-i386 --copyin /home/cltbld/.mozpass.cfg /builds/.mozpass.cfg
mock_mozilla -v -r mozilla-centos6-i386 --cwd / --shell '/usr/bin/env  chown -R mock_mozilla /builds/.mozpass.cfg'
mock_mozilla -r mozilla-centos6-i386 --copyin /home/cltbld/.hgrc /builds/.hgrc
mock_mozilla -v -r mozilla-centos6-i386 --cwd / --shell '/usr/bin/env  chown -R mock_mozilla /builds/.hgrc'
mock_mozilla -v -r mozilla-centos6-i386 --cwd / --unpriv --shell '/usr/bin/env  mkdir -p /builds/slave/rel-m-rel-andrd-bld/build' 
mock_mozilla -v -r mozilla-centos6-i386 --install autoconf213 mozilla-python27-mercurial ccache android-sdk15 android-sdk16 android-ndk5 android-ndk8 zip java-1.6.0-openjdk-devel zlib-devel glibc-static openssh-clients mpfr wget

Packaged multi

# android-armv6 cltbld@bld-centos6-hp-007
mock_mozilla -v -r mozilla-centos6-i386 --cwd /builds/slave/rel-m-rel-andrd-armv6-bld --unpriv --shell '/usr/bin/env MOZ_OBJDIR="obj-firefox" PATH="/tools/buildbot/bin:/usr/local/bin:/bin:/usr/bin" SYMBOL_SERVER_USER="ffxbld" DISPLAY=":2" CCACHE_UMASK="002" SHIP_LICENSED_FONTS="1" MOZ_SYMBOLS_EXTRA_BUILDID="android-armv6" POST_SYMBOL_UPLOAD_CMD="/usr/local/bin/post-symbol-upload.py" SYMBOL_SERVER_PATH="/mnt/netapp/breakpad/symbols_mob/" MOZILLA_REV="FENNEC_18_0_RELEASE" JAVA_HOME="/tools/jdk6" HG_SHARE_BASE_DIR="/builds/hg-shared" LC_ALL="C" SYMBOL_SERVER_HOST="symbols1.dmz.phx1.mozilla.com" TINDERBOX_OUTPUT="1" COMM_REV="FENNEC_18_0_RELEASE" CCACHE_DIR="/builds/ccache" CCACHE_COMPRESS="1" MOZ_UPDATE_CHANNEL="release" SYMBOL_SERVER_SSH_KEY="/home/mock_mozilla/.ssh/ffxbld_dsa" JARSIGNER="/builds/slave/rel-m-rel-andrd-armv6-bld/tools/release/signing/mozpass.py" MOZ_CRASHREPORTER_NO_REPORT="1" python mozharness/scripts/multil10n.py --config-file multi_locale/release_mozilla-release_android-armv6.json --merge-locales --tag-override=FENNEC_18_0_RELEASE --package-multi --summary'

# android cltbld@bld-centos6-hp-016
mock_mozilla -v -r mozilla-centos6-i386 --cwd /builds/slave/rel-m-rel-andrd-bld --unpriv --shell '/usr/bin/env HG_SHARE_BASE_DIR="/builds/hg-shared" JARSIGNER="/builds/slave/rel-m-rel-andrd-bld/tools/release/signing/mozpass.py" SHIP_LICENSED_FONTS="1" MOZ_CRASHREPORTER_NO_REPORT="1" SYMBOL_SERVER_HOST="symbols1.dmz.phx1.mozilla.com" CCACHE_DIR="/builds/ccache" POST_SYMBOL_UPLOAD_CMD="/usr/local/bin/post-symbol-upload.py" MOZILLA_REV="FENNEC_18_0_RELEASE" SYMBOL_SERVER_SSH_KEY="/home/mock_mozilla/.ssh/ffxbld_dsa" PATH="/tools/jdk6/bin:/opt/local/bin:/tools/python/bin:/tools/buildbot/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/" MOZ_UPDATE_CHANNEL="release" TINDERBOX_OUTPUT="1" CCACHE_COMPRESS="1" COMM_REV="FENNEC_18_0_RELEASE" MOZ_OBJDIR="obj-firefox" SYMBOL_SERVER_PATH="/mnt/netapp/breakpad/symbols_mob/" LC_ALL="C" SYMBOL_SERVER_USER="ffxbld" JAVA_HOME="/tools/jdk6" DISPLAY=":2" CCACHE_UMASK="002" python mozharness/scripts/multil10n.py --config-file multi_locale/release_mozilla-release_android.json --merge-locales --tag-override=FENNEC_18_0_RELEASE --package-multi --summary'

Uploaded multi

# android-armv6 cltbld@bld-centos6-hp-007
mock_mozilla -v -r mozilla-centos6-i386 --cwd /builds/slave/rel-m-rel-andrd-armv6-bld/build/obj-firefox --unpriv --shell '/usr/bin/env MOZ_OBJDIR="obj-firefox" PATH="/tools/buildbot/bin:/usr/local/bin:/bin:/usr/bin" SYMBOL_SERVER_USER="ffxbld" DISPLAY=":2" CCACHE_UMASK="002" SHIP_LICENSED_FONTS="1" MOZ_SYMBOLS_EXTRA_BUILDID="android-armv6" POST_SYMBOL_UPLOAD_CMD="/usr/local/bin/post-symbol-upload.py" SYMBOL_SERVER_PATH="/mnt/netapp/breakpad/symbols_mob/" UPLOAD_USER="ffxbld" MOZILLA_REV="FENNEC_18_0_RELEASE" JAVA_HOME="/tools/jdk6" HG_SHARE_BASE_DIR="/builds/hg-shared" SYMBOL_SERVER_HOST="symbols1.dmz.phx1.mozilla.com" UPLOAD_SSH_KEY="~/.ssh/ffxbld_dsa" TINDERBOX_OUTPUT="1" COMM_REV="FENNEC_18_0_RELEASE" MOZ_CRASHREPORTER_NO_REPORT="1" LC_ALL="C" UPLOAD_TO_TEMP="1" MOZ_UPDATE_CHANNEL="release" SYMBOL_SERVER_SSH_KEY="/home/mock_mozilla/.ssh/ffxbld_dsa" UPLOAD_EXTRA_FILES="android-armv6_info.txt android-armv6_info.txt" UPLOAD_HOST="stage.mozilla.org" POST_UPLOAD_CMD="post_upload.py -p mobile -n 1 -v 18.0 --builddir android-armv6/multi --release-to-mobile-candidates-dir --nightly-dir=candidates" CCACHE_DIR="/builds/ccache" CCACHE_COMPRESS="1" python '"'"'/builds/slave/rel-m-rel-andrd-armv6-bld/tools/buildfarm/utils/retry.py'"'"' -s 1 -r 5 -t 3660 make upload AB_CD=multi'

# android cltbld@bld-centos6-hp-016
mock_mozilla -v -r mozilla-centos6-i386 --cwd /builds/slave/rel-m-rel-andrd-bld/build/obj-firefox --unpriv --shell '/usr/bin/env MOZ_OBJDIR="obj-firefox" PATH="/tools/jdk6/bin:/opt/local/bin:/tools/python/bin:/tools/buildbot/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/" SYMBOL_SERVER_USER="ffxbld" DISPLAY=":2" CCACHE_UMASK="002" SHIP_LICENSED_FONTS="1" UPLOAD_USER="ffxbld" POST_SYMBOL_UPLOAD_CMD="/usr/local/bin/post-symbol-upload.py" SYMBOL_SERVER_PATH="/mnt/netapp/breakpad/symbols_mob/" MOZILLA_REV="FENNEC_18_0_RELEASE" JAVA_HOME="/tools/jdk6" HG_SHARE_BASE_DIR="/builds/hg-shared" SYMBOL_SERVER_HOST="symbols1.dmz.phx1.mozilla.com" UPLOAD_SSH_KEY="~/.ssh/ffxbld_dsa" TINDERBOX_OUTPUT="1" COMM_REV="FENNEC_18_0_RELEASE" MOZ_CRASHREPORTER_NO_REPORT="1" LC_ALL="C" UPLOAD_TO_TEMP="1" MOZ_UPDATE_CHANNEL="release" SYMBOL_SERVER_SSH_KEY="/home/mock_mozilla/.ssh/ffxbld_dsa" UPLOAD_EXTRA_FILES="android_info.txt android_info.txt" UPLOAD_HOST="stage.mozilla.org" POST_UPLOAD_CMD="post_upload.py -p mobile -n 1 -v 18.0 --builddir android/multi --release-to-mobile-candidates-dir --nightly-dir=candidates" CCACHE_DIR="/builds/ccache" CCACHE_COMPRESS="1" python '"'"'/builds/slave/rel-m-rel-andrd-bld/tools/buildfarm/utils/retry.py'"'"' -s 1 -r 5 -t 3660 make upload AB_CD=multi'

Re-signed multi. This skipped all single locales, but re-signed en-US as well as multi.

# cltsign@signing1
cd signing-work
python2.6 mozharness/scripts/sign_android.py --config-file signing/android_mozilla-release.py --locale multi

Reenabled slaves in slavealloc
Rebooted slaves
Pinged ioanachiorean in #build and sent r-d email.

Run euballot bouncer submitter

QA found that the euballot bouncer links weren't working. After some investigation we discovered that this was because the euballot bouncer entries were never added. That builder never ran because of the problems with tagging, and we forgot to trigger it. Used "force build" to start the builder and fix the problem.

had to run release-mozilla-release-xulrunner_push_to_mirrors

manually as it wasn't invoked during release automation

bug 830722 xulrunner src and md5checksums are missing in FF 18.0

ran xulrunner source build, revealed that the xulrunner mozconfigs pointed to the old compiler path. Fixed this and reran release-mozilla-release-xulrunner_source build and md5 builds, then push to mirrors.

Releases/Firefox 18.0/BuildNotes

Contents

Build 1

Checklist

bugs hit

Notes

Fennec multilocale

Run euballot bouncer submitter

had to run release-mozilla-release-xulrunner_push_to_mirrors

Navigation menu

Personal tools

Namespaces

Variants

Views

More

Search

Navigation

How to Contribute

MozillaWiki

Around Mozilla

Tools