|
|
| Line 1: |
Line 1: |
| = Steps to Try = | | = Steps to Try = |
| '''Login as cltbld with RDP. Make sure your resolution is set to 1280x1024, 24-bit colour''' | | == Rebooting a slave == |
| | Logon to the machine, reboot it. When it comes back up make sure you connect with '''1280x1024x24 resolution/bit depth'''. Start Buildbot by doing the following |
| | * Open an MSYS window |
| | * Type '/d/mozilla-build/python25/scripts/buildbot start /e/builds/moz2_slave' |
| | * Minimize the MSYS window. |
|
| |
|
| == Intermittent Burning (not related to checkins) == | | == Clobbering a slave == |
| This is often caused by a single slave having a problem such as a host key not accepted, out of disk space, hanging procsses, etc.
| | Find the slave/builder combination on https://build.mozilla.org/clobberer/index.php and use the form to trigger a clobber. The next time that slave does the selected build, it will clobber first. |
| * If the problem is obvious (eg, hanging on a host key):
| |
| *# Log into the failing slave as 'cltbld'
| |
| *# Accept the host key/free some space(1)/whatever.
| |
| * If you don't know what the problem is:
| |
| *# Login to the slave as 'cltbld'
| |
| *# Stop the slave by ctrl+c'ing it in the MSYS terminal
| |
| *# Notify someone in RelEng to investigate further.
| |
| *# Note: Unless 3 or more slaves from a platform are failing the tree does not need to be closed.
| |
| | |
| (1) Slaves sometimes run out of disk space because of failed nightly builds that did not clean-up after themselves. Here's a helper script to clean those up (run this in an MSYS terminal):
| |
| for i in `find /e/builds/moz2_slave -maxdepth 1 -iname "*-nightly" -type d`; do find $i -maxdepth 1 -type d -iname build -exec rm -rf {} \;; done
| |
| == Fixing Failing Builds (clobber steps) ==
| |
| '''Login as cltbld with RDP. Make sure your resolution is set to 1280x1024, 24-bit colour'''
| |
| | |
| Clobber the failing slave and force a new build. (Note: forcing a new build will not necessarily cause a build on the failing slave. This is OK.)
| |
| # Find the hostname in the Tinderbox log (look for 'Building on: XXX' where 'XXX' is a hostname)
| |
| # Login to the slave using provided credentials (RDP for Windows).
| |
| ## Go to the builder directory e:\builds\moz2_slave\$builder_name [where $builder_name matches the name of the failing build on Tinderbox]
| |
| ## Delete the subdirectory named 'build'.
| |
| ## Force a new build
| |
| ### Click on the appropriate column title on the Waterfall (eg. 'mozilla-central-win32')
| |
| ### Fill out the Force Build form. 'Branch to build' MUST be specified (branch name is part of the builders name. eg, mozilla-central or actionmonkey).
| |
| ### Click 'Force Build'.
| |
| ## If the next build doesn't go green, contact RelEng.
| |
| | |
| == Builds not happening at all ==
| |
| '''Make sure all slaves are connected'''
| |
| Note: do NOT use the 'Ping Builder' button on the Waterfall, it will break future builds
| |
| # Go to the Buildslave List page.
| |
| # For any slaves that are listed as 'NOT connected' we need to connect the slave:
| |
| ## Login to the slave using provided credentials (RDP for Windows).
| |
| ### Kill any ssh/sh/make/other unix-type processes in the task manager.
| |
| ### If there is not an existing MSYS window start one with the desktop icon.
| |
| ### Start Buildbot with '/d/mozilla-build/python25/scripts/buildbot start /e/builds/moz2_slave'.
| |
| ### Minimize the MSYS window
| |
| | |
| Wait 30 seconds and check the Buildslave List page again.
| |
| If slaves still aren't connected contact RelEng.
| |
| | |
| | |
| '''It is almost always a bad idea to reboot the buildmaster VM. It runs multiple Buildbot masters. Unless it is completely unreachable please do not restart this machine without talking to someone from RelEng first.'''
| |