Build:Win32:Moz2:ITSupport

Steps to Try

Intermittent Burning (not related to checkins)

This is often caused by a single slave having a problem such as a host key not accepted, out of disk space, hanging procsses, etc.

• If the problem is obvious (eg, hanging on a host key):
  1. Log into the failing slave as 'cltbld'
  2. Accept the host key/free some space(1)/whatever.
• If you don't know what the problem is:
  1. Login to the slave as 'cltbld'
  2. Stop the slave by ctrl+c'ing it in the MSYS terminal
  3. Notify someone in RelEng to investigate further.
  4. Note: Unless 3 or more slaves from a platform are failing the tree does not need to be closed.

(1) Slaves sometimes run out of disk space because of failed nightly builds that did not clean-up after themselves. Here's a helper script to clean those up (run this in an MSYS terminal):

for i in `find /e/builds/moz2_slave -maxdepth 1 -iname "*-nightly" -type d`; do find $i -maxdepth 1 -type d -iname build -exec rm -rf {} \;; done

Fixing Failing Builds

Clobber the failing slave and force a new build. (Note: forcing a new build will not necessarily cause a build on the failing slave. This is OK.)

1. Find the hostname in the Tinderbox log (look for 'Building on: XXX' where 'XXX' is a hostname)
2. Login to the slave using provided credentials (RDP for Windows).
   1. Go to the builder directory e:\builds\moz2_slave\$builder_name [where $builder_name matches the name of the failing build on Tinderbox]
   2. Delete the subdirectory named 'build'.
   3. Force a new build
      1. Click on the appropriate column title on the Waterfall (eg. 'mozilla-central-win32')
      2. Fill out the Force Build form. 'Branch to build' MUST be specified (branch name is part of the builders name. eg, mozilla-central or actionmonkey).
      3. Click 'Force Build'.
   4. If the next build doesn't go green, contact RelEng.

Builds not happening at all

Make sure all slaves are connected

Note: do NOT use the 'Ping Builder' button on the Waterfall, it will break future builds

1. Go to the Buildslave List page.
2. For any slaves that are listed as 'NOT connected' we need to connect the slave:
   1. Login to the slave using provided credentials (RDP for Windows).
      1. Kill any ssh/sh/make/other unix-type processes in the task manager.
      2. If there is not an existing MSYS window start one with the desktop icon.
      3. Start Buildbot with '/d/mozilla-build/python25/scripts/buildbot start /e/builds/moz2_slave'.
      4. Minimize the MSYS window

Wait 30 seconds and check the Buildslave List page again. 
If slaves still aren't connected contact RelEng.

It is almost always a bad idea to reboot the buildmaster VM. It runs multiple Buildbot masters. Unless it is completely unreachable please do not restart this machine without talking to someone from RelEng first.