Releases/Firefox 12.0b1/BuildNotes

From MozillaWiki
Jump to: navigation, search

Notes About Releasing

Please update the Notes Template and the Release:Primer for future releases (bug fixes, changes to automation) as needed

Bugs hit

Enter any bugs pre-existing or newly discovered and filed during the release:

  • bug 723386 - reserved slaves is broken for win32, macosx64
  • linux64 repack 4 timed out cloning mozilla-beta
  • signing hung for win32 build
  • clobbering win32 build didn't work (clobbered rel-beta-w32-bld, but build dir is rel-m-beta-w32-bld)
  • mozconfig check warnings
  • source package on build2 failed because of no signing token, probably due to [1] which changed between build1, build2
  • bug 737554 - hgtool not used for release builds
  • bug 737555 - hgtool not used for source builder
  • bug 737556 - email sent for "[release] Fennec 11.0b5 build1: All signed builds now available" in addition to 12.0b1

Build Engineers

catlee - Tracking bug: bug 734996

Signed-off Revision(s)

Build 1: 5fba4153cb57

Build 2: 06883319e069

L10N changesets

Instructions on how to get them

  • Firefox: [2]
  • Fennec: empty

Tags

Build # Branch, Tags Changeset
1 MOBILE120_2012031407_RELBRANCH, FENNEC_12_0b1_BUILD1, FENNEC_12_0b1_RELEASE f4e3303ee1d5
GECKO120_2012031407_RELBRANCH, FIREFOX_12_0b1_BUILD1, FIREFOX_12_0b1_RELEASE 249ecd7beaf3
2 MOBILE120_2012031419_RELBRANCH, FENNEC_12_0b1_BUILD2, FENNEC_12_0b1_RELEASE 780458b51f02
GECKO120_2012031419_RELBRANCH, FIREFOX_12_0b1_BUILD2, FIREFOX_12_0b1_RELEASE 4027017bbaba


Build data

Firefox

Build # Type Build ID Build machine Time to build
1 Linux 20120314075752 mv-moz2-linux-ix-slave11 1 hrs, 8 mins, 29 secs
Linux64 moz2-linux64-slave01 2 hrs, 56 mins, 2 secs
Mac moz2-darwin10-slave41 3 hrs, 30 mins, 10 secs
Windows mw32-ix-slave20 3 hrs, 16 mins, 33 secs
2 Linux 20120314195616 mv-moz2-linux-ix-slave11 1 hrs, 9 mins, 47 secs
Linux64 linux64-ix-slave05 1 hrs, 7 mins, 12 secs
Mac moz2-darwin10-slave17 3 hrs, 27 mins, 36 secs
Windows mw32-ix-slave14 3 hrs, 4 mins, 0 secs

Fennec

Build # Type Build ID Build machine Time to build
1 android-xul 20120314073924 mv-moz2-linux-ix-slave19 50 mins, 29 secs
2 android-xul 20120314194604 mv-moz2-linux-ix-slave19 1 hrs, 3 mins, 11 secs

Notes

Build 1

Preparing to start Automation

  • Set clobbers for the appropriate masters. Doing this 24-48 hours in advance should speedup the build.
    • clobberer - hit clobber on all masters, mozilla-beta
  • set reserved slaves:
[cltbld@buildbot-master08 master]$ pwd
/builds/buildbot/build1/master
[cltbld@buildbot-master08 master]$ cat reserved_slaves_bm08-build1 
6
  • For build1 only, create a symlink for your $version (eg 11.0b4, 10.0.2esr, 10.0.2, 3.6.27) (until bug 725839 is resolved)
# ffxbld@stage
[ffxbld@surf nightly]$ cd /pub/mozilla.org/firefox/nightly/
[ffxbld@surf nightly]$ mkdir ../candidates/12.0b1-candidates
[ffxbld@surf nightly]$ ln -s ../candidates/12.0b1-candidates 12.0b1-candidates
  • Update l10n changesets for desktop, mobile
wget -O l10n-changesets_mozilla-beta https://l10n-stage-sj.mozilla.org/shipping/l10n-changesets\?ms\=fx12_beta_b1
    • fennec l10n changesets are empty
  • Ran release_sanity locally. Complained about
2012-03-14 01:19:04,675 : ERROR : cannot find l10n locale km in repo https://hg.mozilla.org/releases/l10n/mozilla-beta/km/file/2563487d567b

 replaced km with 7fbecf420e17 in l10n-changesets

Also lots of mozconfig differences:

  • All desktop platforms
    • --enable-signmar in nightly but not in release
      • We don't need to build signmar for releases, needs to be added to whitelist FIXME
    • --enable-js-diagnostics in nightly but not in release
      • ASSUMING that we don't want to build betas with this FIXME
  • Linux & Linux64
    • --enable-warnings-as-errors in nightly but not in release
      • ASSUMING that we don't want to build betas with this FIXME
  • Landed automation configs
  • Tagged buildbot-configs, buildbotcustom, & tools with build & release tags.
hg tag -f FIREFOX_12_0b1_RELEASE FIREFOX_12_0b1_BUILD1 FENNEC_12_0b1_RELEASE FENNEC_12_0b1_BUILD1
  • updated and reconfigured bm08
  • ran release sanity:
cd /builds/buildbot/build1/master
source ../bin/activate
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u bhearsum  -pfirefox,fennec -V 12.0b1 --branch mozilla-beta --build-number 1  -c release-firefox-mozilla-beta.py -c release-fennec-mozilla-beta.py --dryrun localhost:9001
  • Lukas landed mobile release mozconfigs on beta, and we changed the revision to build with. Landed and tagged new release configs.
  • Updated and reconfigured bm08
  • ran release sanity in test mode:
cd /builds/buildbot/build1/master
source ../bin/activate
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u catlee -pfirefox,fennec -V 12.0b1 --branch mozilla-beta --build-number 1 -c release-firefox-mozilla-beta.py -c release-fennec-mozilla-beta.py --dryrun localhost:9001

lots of warnings about mozconfigs as mentioned above. warned that locale km doesn't match the dashboard; this is expected.

  • ran release sanity for realz. need to add the -b flag to skip checks
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u catlee -pfirefox,fennec -V 12.0b1 --branch mozilla-beta --build-number 1 -c release-firefox-mozilla-beta.py -c release-fennec-mozilla-beta.py -b localhost:9001

E-mail Metrics

Mailed "metrics-alerts < AT > mozilla < PERIOD > org" 

Firefox,firefox,firefox,12.0b1,12.0
Fennec,firefox-mobile,fennec,12.0b1,12.0

Tag

No issues

Source

No issues

Build

Firefox

linux,linux64,win32,macosx64 OK

Firefox repacks

Linux OK

Linux64 failed repack 4. retriggered.

Fennec

Fennec repacks

Unittests / Talos

  1. Look to see that they ran
  2. Document any oranges (per platform) for unittests - if possible, try to confirm it's known/random
  3. Make sure there's no red/failures that we need to have a dev look at

XULRunner

Source

No issues

Build

linux,linux64,win32,macosx64 OK

Build 1 interrupted here while desktop repacks were running

Build 2

Preparing to start Automation

  • Set clobbers for the appropriate masters. Doing this 24-48 hours in advance should speedup the build.
    • clobberer - hit clobber on all masters, mozilla-beta
  • reserved slaves still set
  • Landed automation configs
  • Tagged buildbot-configs, buildbotcustom, & tools with build & release tags.
hg tag -f FIREFOX_12_0b1_RELEASE FIREFOX_12_0b1_BUILD2 FENNEC_12_0b1_RELEASE FENNEC_12_0b1_BUILD2
  • updated and reconfigured bm08
  • ran release sanity:
cd /builds/buildbot/build1/master
source ../bin/activate
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u catlee  -pfirefox,fennec -V 12.0b1 --branch mozilla-beta --build-number 2  -c release-firefox-mozilla-beta.py -c release-fennec-mozilla-beta.py --dryrun localhost:9001

errors/warnings: 

2012-03-14 19:36:23,748 : ERROR : found in mobile/android/config/mozconfigs/android/release but not in mobile/android/config/mozconfigs/android/nightly: ac_add_options --with-android-sdk="/tools/android-sdk-r15/platforms/android-14"
2012-03-14 19:36:23,748 : WARNING : ac_add_options --with-android-sdk="/tools/android-sdk-r13/platforms/android-13" not in android ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --enable-update-channel=${MOZ_UPDATE_CHANNEL}']!
2012-03-14 19:36:23,748 : ERROR : found in mobile/android/config/mozconfigs/android/nightly but not in mobile/android/config/mozconfigs/android/release: ac_add_options --with-android-sdk="/tools/android-sdk-r13/platforms/android-13"
2012-03-14 19:36:23,748 : ERROR : found in mobile/android/config/mozconfigs/android/release but not in mobile/android/config/mozconfigs/android/nightly: ac_add_options --with-android-tools="/tools/android-sdk-r15/tools"
2012-03-14 19:36:23,748 : WARNING : ac_add_options --with-android-tools="/tools/android-sdk-r13/tools" not in android ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --enable-update-channel=${MOZ_UPDATE_CHANNEL}']!
2012-03-14 19:36:23,748 : ERROR : found in mobile/android/config/mozconfigs/android/nightly but not in mobile/android/config/mozconfigs/android/release: ac_add_options --with-android-tools="/tools/android-sdk-r13/tools"

2012-03-14 19:37:31,822 : WARNING : ac_add_options --enable-signmar not in macosx64 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --with-macbundlename-prefix=Firefox', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --disable-install-strip']!
2012-03-14 19:37:31,822 : ERROR : found in browser/config/mozconfigs/macosx-universal/nightly but not in browser/config/mozconfigs/macosx-universal/release: ac_add_options --enable-signmar
2012-03-14 19:37:31,822 : WARNING : ac_add_options --enable-js-diagnostics not in macosx64 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --with-macbundlename-prefix=Firefox', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --disable-install-strip']!
2012-03-14 19:37:31,823 : ERROR : found in browser/config/mozconfigs/macosx-universal/nightly but not in browser/config/mozconfigs/macosx-universal/release: ac_add_options --enable-js-diagnostics
2012-03-14 19:37:45,067 : WARNING : ac_add_options --enable-signmar not in win32 ['ac_add_options --enable-update-channel=nightly', '. $topsrcdir/configs/mozilla2/win32/include/choose-make-flags', 'mk_add_options MOZ_MAKE_FLAGS=-j1']!
2012-03-14 19:37:45,067 : ERROR : found in browser/config/mozconfigs/win32/nightly but not in browser/config/mozconfigs/win32/release: ac_add_options --enable-signmar
2012-03-14 19:37:45,067 : WARNING : ac_add_options --enable-js-diagnostics not in win32 ['ac_add_options --enable-update-channel=nightly', '. $topsrcdir/configs/mozilla2/win32/include/choose-make-flags', 'mk_add_options MOZ_MAKE_FLAGS=-j1']!
2012-03-14 19:37:45,068 : ERROR : found in browser/config/mozconfigs/win32/nightly but not in browser/config/mozconfigs/win32/release: ac_add_options --enable-js-diagnostics
2012-03-14 19:37:57,472 : WARNING : ac_add_options --enable-signmar not in linux64 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --with-ccache=/usr/bin/ccache', 'export MOZILLA_OFFICIAL=1', 'export MOZ_TELEMETRY_REPORTING=1', "mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) @MOZ_OBJDIR@/_profile/pgo/profileserver.py 10'"]!
2012-03-14 19:37:57,472 : ERROR : found in browser/config/mozconfigs/linux64/nightly but not in browser/config/mozconfigs/linux64/release: ac_add_options --enable-signmar
2012-03-14 19:37:57,472 : WARNING : ac_add_options --enable-js-diagnostics not in linux64 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --with-ccache=/usr/bin/ccache', 'export MOZILLA_OFFICIAL=1', 'export MOZ_TELEMETRY_REPORTING=1', "mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) @MOZ_OBJDIR@/_profile/pgo/profileserver.py 10'"]!
2012-03-14 19:37:57,472 : ERROR : found in browser/config/mozconfigs/linux64/nightly but not in browser/config/mozconfigs/linux64/release: ac_add_options --enable-js-diagnostics
2012-03-14 19:37:57,472 : ERROR : found in browser/config/mozconfigs/linux64/release but not in browser/config/mozconfigs/linux64/nightly: ac_add_options --enable-warnings-as-errors
2012-03-14 19:37:57,473 : WARNING : ac_add_options --enable-warnings-as-errors not in linux64 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --with-ccache=/usr/bin/ccache', 'export MOZILLA_OFFICIAL=1', 'export MOZ_TELEMETRY_REPORTING=1', "mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) @MOZ_OBJDIR@/_profile/pgo/profileserver.py 10'"]!
2012-03-14 19:37:57,473 : ERROR : found in browser/config/mozconfigs/linux64/nightly but not in browser/config/mozconfigs/linux64/release: ac_add_options --enable-warnings-as-errors
2012-03-14 19:38:11,649 : WARNING : ac_add_options --enable-signmar not in linux ['ac_add_options --enable-update-channel=nightly', 'CXX=$REAL_CXX', 'CXX="ccache $REAL_CXX"', 'CC="ccache $REAL_CC"', 'mk_add_options PROFILE_GEN_SCRIPT=@TOPSRCDIR@/build/profile_pageloader.pl', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --with-ccache=/usr/bin/ccache', 'export MOZILLA_OFFICIAL=1', 'export MOZ_TELEMETRY_REPORTING=1', "mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) @MOZ_OBJDIR@/_profile/pgo/profileserver.py 10'"]!
2012-03-14 19:38:11,649 : ERROR : found in browser/config/mozconfigs/linux32/nightly but not in browser/config/mozconfigs/linux32/release: ac_add_options --enable-signmar
2012-03-14 19:38:11,649 : WARNING : ac_add_options --enable-js-diagnostics not in linux ['ac_add_options --enable-update-channel=nightly', 'CXX=$REAL_CXX', 'CXX="ccache $REAL_CXX"', 'CC="ccache $REAL_CC"', 'mk_add_options PROFILE_GEN_SCRIPT=@TOPSRCDIR@/build/profile_pageloader.pl', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --with-ccache=/usr/bin/ccache', 'export MOZILLA_OFFICIAL=1', 'export MOZ_TELEMETRY_REPORTING=1', "mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) @MOZ_OBJDIR@/_profile/pgo/profileserver.py 10'"]!
2012-03-14 19:38:11,650 : ERROR : found in browser/config/mozconfigs/linux32/nightly but not in browser/config/mozconfigs/linux32/release: ac_add_options --enable-js-diagnostics
2012-03-14 19:38:11,650 : ERROR : found in browser/config/mozconfigs/linux32/release but not in browser/config/mozconfigs/linux32/nightly: ac_add_options --enable-warnings-as-errors
2012-03-14 19:38:11,650 : WARNING : ac_add_options --enable-warnings-as-errors not in linux ['ac_add_options --enable-update-channel=nightly', 'CXX=$REAL_CXX', 'CXX="ccache $REAL_CXX"', 'CC="ccache $REAL_CC"', 'mk_add_options PROFILE_GEN_SCRIPT=@TOPSRCDIR@/build/profile_pageloader.pl', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --with-ccache=/usr/bin/ccache', 'export MOZILLA_OFFICIAL=1', 'export MOZ_TELEMETRY_REPORTING=1', "mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) @MOZ_OBJDIR@/_profile/pgo/profileserver.py 10'"]!
2012-03-14 19:38:11,650 : ERROR : found in browser/config/mozconfigs/linux32/nightly but not in browser/config/mozconfigs/linux32/release: ac_add_options --enable-warnings-as-errors

2012-03-14 19:38:54,155 : ERROR :       locale km revisions not matching: 1c91bb9c9696 (config) vs. 2563487d567b (dashboard)

all these look ok, so re-running with -b and without --dryrun: 

PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u catlee  -pfirefox,fennec -V 12.0b1 --branch mozilla-beta --build-number 2  -c release-firefox-mozilla-beta.py -c release-fennec-mozilla-beta.py -b localhost:9001

Tag

No problems

Source

Failed in make_source-package because it tried to sign files before having a nonce. Fixed by landing [3] and re-building the source builder.

Build

Firefox

linux,linux64,macosx64 OK

win32 build timed out trying to sign. re-triggered. 2nd build worked

Firefox repacks

linux,linux64,macosx64,win32 OK

Fennec

android-xul OK

Unittests / Talos

  1. Look to see that they ran
  2. Document any oranges (per platform) for unittests - if possible, try to confirm it's known/random
  3. Make sure there's no red/failures that we need to have a dev look at

XULRunner

Source

No issues

Build

linux,linux64,macosx64,win32 OK

Signing

Followed instructions in XULRunner Signing (aut required).

Edited ~/sign_xulrunner_beta.sh with 12.0b1 values and ran it

Firefox Signing

Done by our friendly machine overlords \o/

Android Signing

Instructions in MobileSigning

Completed without issue

L10N verify

No problems

Checksums

No problems

Updates

No problems

Update verify

No problems

Reset reserved slaves

Reset reserved slaves to 0

Check permissions / AV scan

No problems

Push to internal mirrors

No problems

Going to Beta

Run backupsnip

NOTE: Remember to do this at least an hour ahead of the expected "go to beta" email. 

bash-3.2$ time ~/bin/backupsnip Firefox-12.0b1-build2
real    36m16.617s
user    0m3.919s
sys     0m18.637s

Final verification

No issues

Release Day

Push beta snippets

time ~/bin/pushsnip Firefox-12.0b1-build2/
real    39m29.436s
user    0m4.929s
sys     0m24.646s

Publish Fennec to the Android Market

Push the files
  • Use this script and run it as ffxbld@stage (please push you changes prior to using it)
Android Market

IMPORTANT - Make sure to follow instructions!

  • download multi apk [4]
  • visit https://market.android.com/publish
  • choose "Firefox Beta"
  • select the "APK Files" tab and choose "Upload APK"
    • choose the multi apk that you downloaded and hit "upload"
      • NOTE: the progress bar does not seem to show progress (at least on Google Chrome)
    • when it finishes uploading, verify VersionCode is the build date (almost anyway, the market reports an hour later than our buildID)
    • hit "Save"
  • activate new apk (You should see an Error message since both apk's are active)
  • deactivate old apk (You should not see the Error message anymore)
  • hit "Save"
  • If this is a beta 1, go to "Product Details" tab -> Recent Changes, change the url to http://www.mozilla.com/en-US/mobile/{VERSION}beta/releasenotes/
    • and hit "Save"

Push XULRunner to Mirrors

# xrbld@stage
rsync  -av --exclude=*.txt --exclude=*unsigned* --exclude=*-symbols.zip \
 --exclude=*.checksums --exclude=jsshell* \
 /pub/mozilla.org/xulrunner/nightly/12.0b1-candidates/build2/ \
 /pub/mozilla.org/xulrunner/releases/12.0b1/
Retrieved from "https://wiki.mozilla.org/index.php?title=Releases/Firefox_12.0b1/BuildNotes&oldid=409549"