Releases/Thunderbird 15.0b1/BuildNotes: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Created page with "Be sure to take notes on manual steps and any problems you encounter. =Checklist= <small> * [[Release:Release_Automation_on_Mercurial:Preparation...")
 
Line 49: Line 49:
  [tbirdbld@upload1.dmz.scl3 nightly]$ ls -l $version-candidates
  [tbirdbld@upload1.dmz.scl3 nightly]$ ls -l $version-candidates
  lrwxrwxrwx 1 tbirdbld thunderbird 31 Jul 17 10:17 15.0b1-candidates -> ../candidates/15.0b1-candidates
  lrwxrwxrwx 1 tbirdbld thunderbird 31 Jul 17 10:17 15.0b1-candidates -> ../candidates/15.0b1-candidates
'''DONE TO HERE'''
* updated & tagged
<i>
* updated & tagged:
    cd /builds/buildbot/build1
    for d in buildbot-configs buildbotcustom tools ; do hg --cwd $d pull --update ; done
    for d in buildbot-configs buildbotcustom tools ; do hg --config ui.username='Hal Wine <hwine@mozilla.com>' --cwd $d tag  THUNDERBIRD_14_0b5_{RELEASE,BUILD1} ; done
    for d in buildbot-configs buildbotcustom tools ; do hg --config ui.username='Hal Wine <hwine@mozilla.com>' --cwd $d push -e 'ssh -l hwine@mozilla.com' ssh://hg.mozilla.org/build/$d ; done
* reconfig
* reconfig
    [cltbld@buildbot-master30 build1]$ source bin/activate
<pre>
    (build1)[cltbld@buildbot-master30 build1]$ make update && make reconfig
[cltbld@buildbot-master30 build1]$ source bin/activate
    (cd buildbotcustom && /usr/bin/hg pull && /usr/bin/hg update -C)
(build1)[cltbld@buildbot-master30 build1]$ make update && make reconfig
    pulling from http://hg.mozilla.org/build/buildbotcustom
(cd buildbotcustom && /usr/bin/hg pull && /usr/bin/hg update -C)
    searching for changes
pulling from http://hg.mozilla.org/build/buildbotcustom
    no changes found
searching for changes
    0 files updated, 0 files merged, 0 files removed, 0 files unresolved
adding changesets
    (cd buildbot-configs && /usr/bin/hg pull && /usr/bin/hg update -C)
adding manifests
    pulling from http://hg.mozilla.org/build/buildbot-configs
adding file changes
    searching for changes
added 4 changesets with 4 changes to 2 files (+1 heads)
    no changes found
(run 'hg heads' to see heads, 'hg merge' to merge)
    0 files updated, 0 files merged, 0 files removed, 0 files unresolved
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
    (cd tools && /usr/bin/hg pull && /usr/bin/hg update -C)
(cd buildbot-configs && /usr/bin/hg pull && /usr/bin/hg update -C)
    pulling from http://hg.mozilla.org/build/tools
pulling from http://hg.mozilla.org/build/buildbot-configs
    searching for changes
searching for changes
    no changes found
adding changesets
    0 files updated, 0 files merged, 0 files removed, 0 files unresolved
adding manifests
    (python buildbot-configs/update-master-json.py tools/buildfarm/maintenance/production-masters.json master/master_config.json)
adding file changes
    writing new master_config.json file to master/master_config.json
added 9 changesets with 9 changes to 8 files (+1 heads)
    cd master && /builds/buildbot/build1/bin/buildbot reconfig $PWD
(run 'hg heads' to see heads, 'hg merge' to merge)
    sending SIGHUP to process 20790
7 files updated, 0 files merged, 0 files removed, 0 files unresolved
    2012-07-11 18:56:36-0700 [-] loading configuration from /builds/buildbot/build1/master/master.cfg
(cd tools && /usr/bin/hg pull && /usr/bin/hg update -C)
    Never saw reconfiguration finish.
pulling from http://hg.mozilla.org/build/tools
searching for changes
adding changesets
adding manifests
adding file changes
added 3 changesets with 3 changes to 2 files
(run 'hg update' to get a working copy)
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
(python buildbot-configs/update-master-json.py tools/buildfarm/maintenance/production-masters.json master/master_config.json)
writing new master_config.json file to master/master_config.json
cd master && /builds/buildbot/build1/bin/buildbot reconfig $PWD
sending SIGHUP to process 20790
2012-07-17 10:49:19-0700 [-] loading configuration from /builds/buildbot/build1/master/master.cfg
Never saw reconfiguration finish.
(build1)[cltbld@buildbot-master30 build1]$ cd master && /builds/buildbot/build1/bin/buildbot reconfig $PWD
sending SIGHUP to process 20790
Never saw reconfiguration finish.
</pre>


* release sanity:
* release sanity:
     PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u hwine
     PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u armenzg \
         -V 14.0b5 --build-number 1 --branch comm-beta \
         -V 15.0b1 --build-number 1 --branch comm-beta \
         --release-config release-thunderbird-comm-beta.py --product thunderbird \
         --release-config release-thunderbird-comm-beta.py --product thunderbird \
         --dryrun localhost:9001
         --dryrun localhost:9001
** errors:
<pre>
2012-07-17 10:50:35,651 : WARNING : if test "$IS_NIGHTLY" != ""; then not in win32 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --enable-signmar', 'ac_add_options --enable-js-diagnostics', '. $topsrcdir/configs/mozilla2/win32/include/choose-make-flags', 'mk_add_options MOZ_MAKE_FLAGS=-j1']!
2012-07-17 10:50:35,651 : ERROR : found in mail/config/mozconfigs/win32/nightly but not in mail/config/mozconfigs/win32/release: if test "$IS_NIGHTLY" != ""; then
2012-07-17 10:50:35,651 : WARNING : ac_add_options --disable-auto-deps not in win32 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --enable-signmar', 'ac_add_options --enable-js-diagnostics', '. $topsrcdir/configs/mozilla2/win32/include/choose-make-flags', 'mk_add_options MOZ_MAKE_FLAGS=-j1']!
2012-07-17 10:50:35,652 : ERROR : found in mail/config/mozconfigs/win32/nightly but not in mail/config/mozconfigs/win32/release: ac_add_options --disable-auto-deps
2012-07-17 10:50:35,652 : WARNING : fi not in win32 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --enable-signmar', 'ac_add_options --enable-js-diagnostics', '. $topsrcdir/configs/mozilla2/win32/include/choose-make-flags', 'mk_add_options MOZ_MAKE_FLAGS=-j1']!
2012-07-17 10:50:35,652 : ERROR : found in mail/config/mozconfigs/win32/nightly but not in mail/config/mozconfigs/win32/release: fi
2012-07-17 10:50:37,865 : ERROR : Error verifying mozconfigs
2012-07-17 10:51:04,515 : ERROR : l10n_changesets and shipped_locales differ on locales: ['el', 'zh-TW', 'ja-JP-mac', 'ta-LK', 'vi', 'ca', 'it', 'cs', 'pa-IN', 'ar', 'bn-BD', 'nb-NO', 'eu', 'et', 'gl', 'es-ES', 'id', 'en-GB', 'ru', 'sr', 'nl', 'sl', 'tr', 'lt', 'rm', 'ro', 'is', 'pl', 'be', 'fr', 'bg', 'sv-SE', 'ast', 'hr', 'de', 'da', 'pt-BR', 'br', 'fi', 'hu', 'ja', 'he', 'hy-AM', 'pt-PT', 'es-AR', 'sq', 'ko', 'nn-NO', 'ga-IE', 'sk', 'zh-CN', 'si', 'gd', 'uk', 'fy-NL']
</pre>
** bhearsum told me I can ignore the mozconfig errors and file a follow up bug (used "-m" to skip it)
** shipped-locales had been migrated only with "en-US" in it
** I bumped the comm-beta revision and re-tagged
'''DONE TO HERE'''
* reset reserved:
* reset reserved:
     [cltbld@buildbot-master30 master]$ pwd -P
     [cltbld@buildbot-master30 master]$ pwd -P

Revision as of 18:24, 17 July 2012

Be sure to take notes on manual steps and any problems you encounter.

Checklist

Notes

tracking bug: bug 772810
buildbot master: 30

Build 1

echo 8 > /builds/buildbot/build1/master/reserved_slaves'"
  • clobbered release-comm-beta
  • created candidates dir:
ssh -l tbirdbld -i ~/.ssh/tbirdbld_dsa stage.mozilla.org
version=15.0b1
product=thunderbird
cd /pub/mozilla.org/$product/nightly/
mkdir ../candidates/$version-candidates
ln -s ../candidates/$version-candidates $version-candidates
[tbirdbld@upload1.dmz.scl3 nightly]$ ls -l $version-candidates
lrwxrwxrwx 1 tbirdbld thunderbird 31 Jul 17 10:17 15.0b1-candidates -> ../candidates/15.0b1-candidates
  • updated & tagged
  • reconfig
[cltbld@buildbot-master30 build1]$ source bin/activate
(build1)[cltbld@buildbot-master30 build1]$ make update && make reconfig
(cd buildbotcustom && /usr/bin/hg pull && /usr/bin/hg update -C)
pulling from http://hg.mozilla.org/build/buildbotcustom
searching for changes
adding changesets
adding manifests
adding file changes
added 4 changesets with 4 changes to 2 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
(cd buildbot-configs && /usr/bin/hg pull && /usr/bin/hg update -C)
pulling from http://hg.mozilla.org/build/buildbot-configs
searching for changes
adding changesets
adding manifests
adding file changes
added 9 changesets with 9 changes to 8 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)
7 files updated, 0 files merged, 0 files removed, 0 files unresolved
(cd tools && /usr/bin/hg pull && /usr/bin/hg update -C)
pulling from http://hg.mozilla.org/build/tools
searching for changes
adding changesets
adding manifests
adding file changes
added 3 changesets with 3 changes to 2 files
(run 'hg update' to get a working copy)
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
(python buildbot-configs/update-master-json.py tools/buildfarm/maintenance/production-masters.json master/master_config.json)
writing new master_config.json file to master/master_config.json
cd master && /builds/buildbot/build1/bin/buildbot reconfig $PWD
sending SIGHUP to process 20790
2012-07-17 10:49:19-0700 [-] loading configuration from /builds/buildbot/build1/master/master.cfg
Never saw reconfiguration finish.
(build1)[cltbld@buildbot-master30 build1]$ cd master && /builds/buildbot/build1/bin/buildbot reconfig $PWD
sending SIGHUP to process 20790
Never saw reconfiguration finish.
  • release sanity:
   PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u armenzg \
       -V 15.0b1 --build-number 1 --branch comm-beta \
       --release-config release-thunderbird-comm-beta.py --product thunderbird \
       --dryrun localhost:9001
    • errors:
2012-07-17 10:50:35,651 : WARNING : if test "$IS_NIGHTLY" != ""; then not in win32 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --enable-signmar', 'ac_add_options --enable-js-diagnostics', '. $topsrcdir/configs/mozilla2/win32/include/choose-make-flags', 'mk_add_options MOZ_MAKE_FLAGS=-j1']!
2012-07-17 10:50:35,651 : ERROR : found in mail/config/mozconfigs/win32/nightly but not in mail/config/mozconfigs/win32/release: if test "$IS_NIGHTLY" != ""; then
2012-07-17 10:50:35,651 : WARNING : ac_add_options --disable-auto-deps not in win32 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --enable-signmar', 'ac_add_options --enable-js-diagnostics', '. $topsrcdir/configs/mozilla2/win32/include/choose-make-flags', 'mk_add_options MOZ_MAKE_FLAGS=-j1']!
2012-07-17 10:50:35,652 : ERROR : found in mail/config/mozconfigs/win32/nightly but not in mail/config/mozconfigs/win32/release: ac_add_options --disable-auto-deps
2012-07-17 10:50:35,652 : WARNING : fi not in win32 ['ac_add_options --enable-update-channel=nightly', 'ac_add_options --enable-signmar', 'ac_add_options --enable-js-diagnostics', '. $topsrcdir/configs/mozilla2/win32/include/choose-make-flags', 'mk_add_options MOZ_MAKE_FLAGS=-j1']!
2012-07-17 10:50:35,652 : ERROR : found in mail/config/mozconfigs/win32/nightly but not in mail/config/mozconfigs/win32/release: fi
2012-07-17 10:50:37,865 : ERROR : Error verifying mozconfigs


2012-07-17 10:51:04,515 : ERROR : l10n_changesets and shipped_locales differ on locales: ['el', 'zh-TW', 'ja-JP-mac', 'ta-LK', 'vi', 'ca', 'it', 'cs', 'pa-IN', 'ar', 'bn-BD', 'nb-NO', 'eu', 'et', 'gl', 'es-ES', 'id', 'en-GB', 'ru', 'sr', 'nl', 'sl', 'tr', 'lt', 'rm', 'ro', 'is', 'pl', 'be', 'fr', 'bg', 'sv-SE', 'ast', 'hr', 'de', 'da', 'pt-BR', 'br', 'fi', 'hu', 'ja', 'he', 'hy-AM', 'pt-PT', 'es-AR', 'sq', 'ko', 'nn-NO', 'ga-IE', 'sk', 'zh-CN', 'si', 'gd', 'uk', 'fy-NL']
    • bhearsum told me I can ignore the mozconfig errors and file a follow up bug (used "-m" to skip it)
    • shipped-locales had been migrated only with "en-US" in it
    • I bumped the comm-beta revision and re-tagged

DONE TO HERE

  • reset reserved:
   [cltbld@buildbot-master30 master]$ pwd -P
   /builds/buildbot/build1/master
   [cltbld@buildbot-master30 master]$ echo 0 >reserved_slaves 
   [cltbld@buildbot-master30 master]$ cat reserved_slaves
   0
  • push snippets:
   $ ssh aus3-staging.mozilla.org
   Last login: Wed Jul 11 13:45:06 2012 from vpn1.dmz.releng.scl3.mozilla.com
   [hwine@dp-ausstage01 ~]$ sudo su - tbirdbld
   [tbirdbld@dp-ausstage01 ~]$ cd /opt/aus2/snippets/staging
   [tbirdbld@dp-ausstage01 staging]$ ~/bin/pushsnip Thunderbird-14.0b5-build1
   ---------------------------------------------------------------------------
   Updating PHX1:

lines skipped

   sent 6419349 bytes  received 232410 bytes  20126.35 bytes/sec
   total size is 5275931  speedup is 0.79
   
   Touching /opt/aus2/incoming/3/Thunderbird
   [tbirdbld@dp-ausstage01 staging]$ exit
  • manually start final verifications (due to bug 773283) via waterfall gui. All passed
  • skipped "ready for releastest" & "ready for release" builders as moot. Manually verified in bouncer sufficient uptake for beta release. (more fallout from uptake monitoring failure)

Problems Encountered

  • buildbot-master30 had never been used for a release build before, so extra setup needed.
    • Makefile had wrong path to hg
    • Missing symlinks for thunderbird configs/l10n from master/ (caught by release-sanity ftw)
  • Final verifications didn't fire bug 773283
    • worked with bhearsum via irc to confirm okay to push to beta channel before resolution
    • twistd.log files saved off