Releases/Thunderbird 14.0b2/BuildNotes

From MozillaWiki
Jump to: navigation, search

Bugs hit

  • The master was listening to port 8001 so I stopped/started the master.
    • It burned some tagging jobs that were not running when I looked
    • I clobbered and re-triggered
    • this might have happened because I tried using "buildbot-wrangler" first (Ctrl + C) it due to lack of output and then I used release_sanity.py to start the build
  • [Next release will hit it] bug 762256 - bm34 has local changes
  • [Next release will hit it] bug 725839 - create symlinks
  • [Next release will hit it] bug 757798 - Re-enable hgtool for client.py cloning of mozilla-* in Thunderbird releases
Got revision 271b0e7e0728
abort: unknown revision 'THUNDERBIRD_14_0b2_RELEASE'!
Time: real 7.330 secs (user 6.400+0.000 sys 0.120+0.000)
Executing command: ['python', '../tools/buildfarm/utils/hgtool.py', 'http://hg.mozilla.org/releases/mozilla-beta', '/builds/slave/tb-rel-comm-beta-lnx64-bld/build/mozilla']
Executing command: ['hg', 'update', '-r', 'THUNDERBIRD_14_0b2_RELEASE', '-R', '/builds/slave/tb-rel-comm-beta-lnx64-bld/build/mozilla', '--verbose', '--time']
  • * [Next release will hit it] bug 764160 - mozconfigs errors on release_sanity.py
[cltbld@buildbot-master34 build1]$ cd /builds/buildbot/build1/master
[cltbld@buildbot-master34 master]$ source ../bin/activate
(build1)[cltbld@buildbot-master34 master]$ PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u armenzg -V 14.0b2 \
>    --branch comm-beta --build-number 1 \
>    --release-config release-thunderbird-comm-beta.py --products thunderbird  \
>    --dryrun -l localhost:9001
2012-06-12 13:33:11,544 : INFO : Branch comm-beta is enabled on master for thunderbird
2012-06-12 13:33:12,040 : INFO : Comparing thunderbird mozconfigs to nightly mozconfigs...
2012-06-12 13:33:13,097 : WARNING : ac_add_options --with-ccache not in macosx64 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --with-macbundlename-prefix=Firefox', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --enable-signmar', 'ac_add_options --enable-js-diagnostics', 'ac_add_options --disable-install-strip']!
2012-06-12 13:33:13,097 : ERROR : found in mail/config/mozconfigs/macosx-universal/nightly but not in mail/config/mozconfigs/macosx-universal/release: ac_add_options --with-ccache
2012-06-12 13:33:13,097 : ERROR : found in mail/config/mozconfigs/macosx-universal/release but not in mail/config/mozconfigs/macosx-universal/nightly: mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print --comm-rev=${COMM_REV} --mozilla-rev=${MOZILLA_REV}"
2012-06-12 13:33:13,097 : WARNING : mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print" not in macosx64 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --with-macbundlename-prefix=Firefox', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --enable-signmar', 'ac_add_options --enable-js-diagnostics', 'ac_add_options --disable-install-strip']!
2012-06-12 13:33:13,097 : ERROR : found in mail/config/mozconfigs/macosx-universal/nightly but not in mail/config/mozconfigs/macosx-universal/release: mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print"
2012-06-12 13:33:13,962 : ERROR : found in mail/config/mozconfigs/win32/release but not in mail/config/mozconfigs/win32/nightly: mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print --comm-rev=${COMM_REV} --mozilla-rev=${MOZILLA_REV}"
2012-06-12 13:33:13,962 : WARNING : mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print" not in win32 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --enable-signmar', 'ac_add_options --enable-js-diagnostics', '. $topsrcdir/configs/mozilla2/win32/include/choose-make-flags', 'mk_add_options MOZ_MAKE_FLAGS=-j1']!
2012-06-12 13:33:13,962 : ERROR : found in mail/config/mozconfigs/win32/nightly but not in mail/config/mozconfigs/win32/release: mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print"
2012-06-12 13:33:14,889 : ERROR : found in mail/config/mozconfigs/linux64/release but not in mail/config/mozconfigs/linux64/nightly: mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print --comm-rev=${COMM_REV} --mozilla-rev=${MOZILLA_REV}"
2012-06-12 13:33:14,889 : WARNING : mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print" not in linux64 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --enable-signmar', 'ac_add_options --enable-js-diagnostics', 'ac_add_options --with-ccache=/usr/bin/ccache', 'export MOZILLA_OFFICIAL=1', 'export MOZ_TELEMETRY_REPORTING=1', "mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) @MOZ_OBJDIR@/_profile/pgo/profileserver.py 10'"]!
2012-06-12 13:33:14,889 : ERROR : found in mail/config/mozconfigs/linux64/nightly but not in mail/config/mozconfigs/linux64/release: mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print"
2012-06-12 13:33:15,666 : ERROR : found in mail/config/mozconfigs/linux32/release but not in mail/config/mozconfigs/linux32/nightly: mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print --comm-rev=${COMM_REV} --mozilla-rev=${MOZILLA_REV}"
2012-06-12 13:33:15,666 : WARNING : mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print" not in linux ['ac_add_options --enable-update-channel=nightly', 'CXX=$REAL_CXX', 'CXX="ccache $REAL_CXX"', 'CC="ccache $REAL_CC"', 'mk_add_options PROFILE_GEN_SCRIPT=@TOPSRCDIR@/build/profile_pageloader.pl', 'ac_add_options --enable-codesighs', 'mk_add_options MOZ_MAKE_FLAGS="-j4"', 'ac_add_options --enable-signmar', 'ac_add_options --enable-js-diagnostics', 'ac_add_options --with-ccache=/usr/bin/ccache', 'export MOZILLA_OFFICIAL=1', 'export MOZ_TELEMETRY_REPORTING=1', "mk_add_options PROFILE_GEN_SCRIPT='$(PYTHON) @MOZ_OBJDIR@/_profile/pgo/profileserver.py 10'"]!
2012-06-12 13:33:15,667 : ERROR : found in mail/config/mozconfigs/linux32/nightly but not in mail/config/mozconfigs/linux32/release: mk_add_options CLIENT_PY_ARGS="--hg-options='--verbose --time' --hgtool=../tools/buildfarm/utils/hgtool.py --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print"
2012-06-12 13:33:15,667 : ERROR : Error verifying mozconfigs
2012-06-12 13:33:15,767 : INFO : Comparing tagged revision http://hg.mozilla.org/build/buildbot-configs/raw-file/THUNDERBIRD_14_0b2_BUILD1/mozilla/release-thunderbird-comm-beta.py to on-disk release-thunderbird-comm-beta.py ...
2012-06-12 13:33:15,825 : INFO : Comparing tagged revision http://hg.mozilla.org/build/buildbot-configs/raw-file/THUNDERBIRD_14_0b2_BUILD1/mozilla/l10n-changesets_thunderbird-beta to on-disk l10n-changesets_thunderbird-beta ...
2012-06-12 13:33:15,825 : INFO : Checking for existence of http://hg.mozilla.org/releases/mozilla-beta/rev/7912483cf789...
2012-06-12 13:33:16,800 : INFO : Got: http://hg.mozilla.org/releases/mozilla-beta/rev/7912483cf789 !
2012-06-12 13:33:16,800 : INFO : Checking for existence of http://hg.mozilla.org/releases/comm-beta/rev/2d4b4998b70b...
2012-06-12 13:33:17,173 : INFO : Got: http://hg.mozilla.org/releases/comm-beta/rev/2d4b4998b70b !
2012-06-12 13:33:17,173 : INFO : command: START
2012-06-12 13:33:17,174 : INFO : command: buildbot --version
2012-06-12 13:33:17,174 : INFO : command: cwd: /builds/buildbot/build1/master
2012-06-12 13:33:17,174 : INFO : command: output:
Buildbot version: 0.8.2-hg-a2045101fe7a-production-0.8
Twisted version: 10.1.0
2012-06-12 13:33:17,409 : INFO : command: END (0.23s elapsed)

2012-06-12 13:33:17,410 : CRITICAL : Tests Failed! Not running sendchange!
2012-06-12 13:33:17,410 : CRITICAL : Failed tests (run with -b to skip) :
2012-06-12 13:33:17,410 : CRITICAL : verify_mozconfig

Build 1

Preparing to start Automation

  • Set clobbers for the appropriate masters. Doing this 24-48 hours in advance should speedup the build.
  • hit "ship it"
  • Reserve slaves
[cltbld@buildbot-master34 master]$ echo "10" > /builds/buildbot/build1/master/reserved_slaves
  • For build1 only, create a symlink for your $version (eg 11.0b4, 10.0.2esr, 10.0.2, 3.6.27) (until bug 725839 is resolved)
# tbirdbld@stage
cd /pub/mozilla.org/thunderbird/nightly/
mkdir ../candidates/14.0b2-candidates
ln -s ../candidates/14.0b2-candidates 14.0b2-candidates
  • Update l10n changesets & hit "ship it"
wget -O mozilla/l10n-changesets_thunderbird-beta  https://l10n.mozilla.org/shipping/l10n-changesets?ms=tb14_beta_b2
  • Land automation configs
  • Tag buildbot-configs, buildbotcustom, & tools with build & release tags.
    • hg tag THUNDERBIRD_14_0b2_{RELEASE,BUILD1}
  • update and reconfigure the master
# this removed all local changes
cd /builds/buildbot/build1
# ('''in bugs hit''') update buildbot-configs manually - bm34 has local changes
cd /builds/buildbot/build1/master
source ../bin/activate
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u armenzg -V 14.0b2 \
   --branch comm-beta --build-number 1 \
   --release-config release-thunderbird-comm-beta.py --products thunderbird  \
   --dryrun -l localhost:9001
  • (in bugs hit): I had to run with "-m" to by-pass mozconfigs:
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u armenzg -V 14.0b2 \
   --branch comm-beta --build-number 1 \
   --release-config release-thunderbird-comm-beta.py --products thunderbird  \
   --dryrun -m -l localhost:9001
  • start automation
cd /builds/buildbot/build1/master
source ../bin/activate
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u armenzg -V 14.0b2 \
   --branch comm-beta --build-number 1 \
   --release-config release-thunderbird-comm-beta.py --products thunderbird  \
   -l localhost:9001
  • (in bugs hit): I had to run with "-m" to by-pass mozconfigs:
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u armenzg -V 14.0b2 \
   --branch comm-beta --build-number 1 \
   --release-config release-thunderbird-comm-beta.py --products thunderbird  \
   -m -l localhost:9001
  • (in bugs hit): The master was listening to port 8001 so I stopped/started the master.
    • It burned some tagging jobs that were not running when I looked
    • I clobbered and re-triggered
    • this might have happened because I tried using "buildbot-wrangler" first (Ctrl + C) it due to lack of output and then I used release_sanity.py to start the build
  • start automation ← monitor progress on buildbot (e.g. release-comm-beta)

Check that the builds are taken after tagging

  • done

Branches glog

  • Check that tagging created the branches and document them

http://hg.mozilla.org/releases/mozilla-beta/graph/96047 http://hg.mozilla.org/releases/comm-beta/graph/11884

E-mail Metrics

Use the address "metrics-alerts < AT > mozilla < PERIOD > org". Note for first-time-releasers: your email will get held for moderator approval - that is expected. If it happens more than once, get help on #metrics.

Thunderbird,thunderbird,thunderbird,14.0b2,14.0

Reset reserved slaves

We're waiting for TB13.0.1 to be triggered today so we will clear the reservation of slaves in that release.

Going to Beta

Run backupsnip

# tbirdbld@aus3-staging
~/bin/backupsnip Thunderbird-14.0b2-build1
real    12m59.386s
user    0m4.128s
sys     0m9.219s

Push to beta

# tbirdbld@aus3-staging
~/bin/pushsnip Thunderbird-14.0b2-build1