Notes and Caveats

NOTE! If you connect to any of the Windows boxes with a RDP client set to 16-bit color mode, the reftests in /mozilla/modules/libpr0n/test/reftest/ will start failing, until the display is reset back to 24-bit mode (eg, by disconnecting and/or rebooting). The tests require the display to be in 24-bit color mode, but Windows will reset the display to whatever the depth of the last client to connect to it was.

Steps to Try

Please try these steps in order - if the third step (manual clobber) doesn't work contact bhearsum or robcee.

1. Check Slave is Connected/Force a Build

check waterfall at: http://qm-rhel02.mozilla.org:2005/ (mpt-vpn)
see if slave is connected.
if so, click the machine name link, try a "Force Build"
- fill out the name and reason fields, click the button

2. Restarting the Slave

login to machine using provided credentials using Windows Remote Desktop
on windows, ctrl-C in the command window, answer Yes to terminate buildbot process
check the Task Manager to see if there are any stuck sh.exe and make.exe processes
- if so, reboot the machine using whatever means necessary (the stuck sh.exe and make.exe processes can make shutting down tricky. Kill them if you can)
- when machine is rebooted, log back in and open a command prompt
- restart buildbot:
  - cd c:\
  - buildbot start slave (command does not return)
  - minimize the cmd.exe window
check the waterfall at: http://qm-rhel02.mozilla.org:2005/
the slave sometimes takes a couple of minutes to reconnect
if it does, and is necessary, click the machine name link and force a build as above (fill name and reason fields, click the button)

3. Clobbering Manually

Sometimes a machine will need to be "clobbered" (have its build directory removed inside the slave dir)
login to the machine.
stop the slave (as above, ctrl-C in the command window)
check the task manager to make sure there are no errant sh.exe or make.exe files.
- If there are, kill them (End process on sh.exe) or reboot the machine
from the Command Line:
- cd C:\slave
- rmdir /s /q trunk for qm-winxp01
restart the slave
- cd \
- buildbot start slave (command does not return)
- minimize cmd.exe window

Contact Robcee or Bhearsum if you need to go to Step 4

4. Restarting the Farm

In the worst case, the entire buildbot farm needs to be restarted
shutdown each slave as per the instructions: Unittest:Linux:ITSupport Unittest:Mac:ITSupport
shutdown master on qm-rhel02
- cd /build
- buildbot stop master
reboot qm-rhel02 and slave machines if necessary (stuck processes, strange behavior)
restart master on qm-rhel02
- cd /build
- buildbot start master
restart slaves as above
- qm-centos5-01, qm-centos5-02, qm-centos5-04, qm-xserve01, qm-xserve06, qm-win2k3-01, qm-win2k3-02, qm-win2k3-pgo01
verify waterfall at http://qm-rhel02.mozilla.org:2005/ is visible and slaves are connected

Unittest:WinXP:ITSupport

Contents