Unittest:Linux:ITSupport
From MozillaWiki
Contents
Notes and Caveats
Steps to Try
Please try these steps in order - if the third step (manual clobber) doesn't work contact bhearsum or robcee.
1. Check Slave is Connected/Force a Build
- check waterfall at: http://qm-rhel02.mozilla.org:2005/ (mpt-vpn)
- see if slave is connected.
- if so, click the machine name link, try a "Force Build"
- fill out the name and reason fields, click the button
2. Restarting the Slave
- login to machine using VNC
- in the Terminal, type "buildbot stop slave" (pwd should be /builds/)
- reboot VM if necessary (never needed to yet)
- restart the slave
- cd /builds
- verify Xvfb is running in the other Xterm
- if not, enter "Xvfb -screen 0 1280x1024x24 :2 &" in the second Xterm
- ignore any metacity already running on display1.
- if no metacity is running on display2, run:
- DISPLAY=:2 metacity &
- DISPLAY=:2 buildbot start slave
- check the waterfall at: http://qm-rhel02.mozilla.org:2005/
- the slave sometimes takes a couple of minutes to reconnect
- if it does, and is necessary, click the machine name link and force a build as above (fill name and reason fields, click the button)
3. Clobbering Manually
- Sometimes a machine will need to be "clobbered" (have its build directory removed inside the slave dir)
- login to the machine using VNC
- stop the slave
- buildbot stop slave
- rm -rf /builds/slave/trunk_(machine name)
- restart the slave
- cd /builds
- verify Xvfb is running in the other Xterm
- if not, enter "Xvfb -screen 0 1280x1024x24 :2 &" in the second Xterm
- ignore any metacity already running on display1.
- if no metacity is running on display2, run:
- DISPLAY=:2 metacity &
- DISPLAY=:2 buildbot start slave
Contact Robcee or Bhearsum if you need to go to Step 4
4. Restarting the Farm
- In the worst case, the entire buildbot farm needs to be restarted
- shutdown each slave as per the instructions: Unittest:WinXP:ITSupport Unittest:Mac:ITSupport
- shutdown master on qm-rhel02
- cd /build
- buildbot stop master
- reboot qm-rhel02 and slave machines if necessary (stuck processes, strange behavior)
- restart master on qm-rhel02
- cd /build
- buildbot start master
- restart slaves as above
- qm-centos5-01, qm-centos5-02, qm-centos5-04, qm-xserve01, qm-xserve06, qm-win2k3-01, qm-win2k3-02, qm-win2k3-pgo01
- verify waterfall at http://qm-rhel02.mozilla.org:2005/ is visible and slaves are connected