Unittest:Linux:ITSupport

From MozillaWiki
Jump to: navigation, search

Notes and Caveats

Steps to Try

Please try these steps in order - if the third step (manual clobber) doesn't work contact bhearsum or robcee.

1. Check Slave is Connected/Force a Build

  • check waterfall at: http://qm-rhel02.mozilla.org:2005/ (mpt-vpn)
  • see if slave is connected.
  • if so, click the machine name link, try a "Force Build"
    • fill out the name and reason fields, click the button

2. Restarting the Slave

  • login to machine using VNC
  • in the Terminal, type "buildbot stop slave" (pwd should be /builds/)
  • reboot VM if necessary (never needed to yet)
  • restart the slave
    • cd /builds
    • verify Xvfb is running in the other Xterm
      • if not, enter "Xvfb -screen 0 1280x1024x24 :2 &" in the second Xterm
    • ignore any metacity already running on display1.
    • if no metacity is running on display2, run:
      • DISPLAY=:2 metacity &
    • DISPLAY=:2 buildbot start slave
  • check the waterfall at: http://qm-rhel02.mozilla.org:2005/
  • the slave sometimes takes a couple of minutes to reconnect
  • if it does, and is necessary, click the machine name link and force a build as above (fill name and reason fields, click the button)

3. Clobbering Manually

  • Sometimes a machine will need to be "clobbered" (have its build directory removed inside the slave dir)
  • login to the machine using VNC
  • stop the slave
    • buildbot stop slave
    • rm -rf /builds/slave/trunk_(machine name)
  • restart the slave
    • cd /builds
    • verify Xvfb is running in the other Xterm
      • if not, enter "Xvfb -screen 0 1280x1024x24 :2 &" in the second Xterm
    • ignore any metacity already running on display1.
    • if no metacity is running on display2, run:
      • DISPLAY=:2 metacity &
    • DISPLAY=:2 buildbot start slave

Contact Robcee or Bhearsum if you need to go to Step 4

4. Restarting the Farm

  • In the worst case, the entire buildbot farm needs to be restarted
  • shutdown each slave as per the instructions: Unittest:WinXP:ITSupport Unittest:Mac:ITSupport
  • shutdown master on qm-rhel02
    • cd /build
    • buildbot stop master
  • reboot qm-rhel02 and slave machines if necessary (stuck processes, strange behavior)
  • restart master on qm-rhel02
    • cd /build
    • buildbot start master
  • restart slaves as above
    • qm-centos5-01, qm-centos5-02, qm-centos5-04, qm-xserve01, qm-xserve06, qm-win2k3-01, qm-win2k3-02, qm-win2k3-pgo01
  • verify waterfall at http://qm-rhel02.mozilla.org:2005/ is visible and slaves are connected