|
|
| (11 intermediate revisions by 4 users not shown) |
| Line 1: |
Line 1: |
| == Machines ==
| | '''This document is obsolete -- please refer to [[ReleaseEngineering:ITSupport]] for your IT Support needs''' |
| | |
| === Production ===
| |
| | |
| {| class="fullwidth-table"
| |
| | style="background:#efefef" | '''Name'''
| |
| | style="background:#efefef" | '''Platform'''
| |
| | style="background:#efefef" | '''CVS Branch for Config'''
| |
| | style="background:#efefef" | '''Tree'''
| |
| | style="background:#efefef" | '''Support Tier'''*
| |
| |-
| |
| | qm-rhel02 (master)
| |
| | linux
| |
| | -
| |
| | [http://tinderbox.mozilla.org/showbuilds.cgi?tree=Firefox Firefox]
| |
| | 1
| |
| |-
| |
| | qm-centos5-01
| |
| | linux
| |
| | Trunk
| |
| | [http://tinderbox.mozilla.org/showbuilds.cgi?tree=Firefox Firefox]
| |
| | 1
| |
| |-
| |
| | qm-xserve01
| |
| | Mac OS X 10.4
| |
| | Trunk
| |
| | [http://tinderbox.mozilla.org/showbuilds.cgi?tree=Firefox Firefox]
| |
| | 1
| |
| |-
| |
| | qm-winxp01
| |
| | Windows XP
| |
| | Trunk
| |
| | [http://tinderbox.mozilla.org/showbuilds.cgi?tree=Firefox Firefox]
| |
| | 1
| |
| |-
| |
| | qm-win2k3-01 (master)
| |
| | Windows Server 2003
| |
| | Trunk
| |
| | [http://tinderbox.mozilla.org/showbuilds.cgi?tree=Firefox Firefox]
| |
| | 1
| |
| |-
| |
| | qm-leak-centos5-01
| |
| | linux
| |
| | Trunk
| |
| | [http://tinderbox.mozilla.org/showbuilds.cgi?tree=MozillaTest MozillaTest]
| |
| | X
| |
| |-
| |
| | qm-leak-tiger-01
| |
| | Mac OS X 10.4
| |
| | -
| |
| | [http://tinderbox.mozilla.org/showbuilds.cgi?tree=MozillaTest MozillaTest]
| |
| | X
| |
| |-
| |
| | qm-leak-winxp01
| |
| | Windows XP
| |
| | -
| |
| | [http://tinderbox.mozilla.org/showbuilds.cgi?tree=MozillaTest MozillaTest]
| |
| | X
| |
| |-
| |
| | qm-leak-w23k-01
| |
| | Windows Server 2003
| |
| | -
| |
| | [http://tinderbox.mozilla.org/showbuilds.cgi?tree=MozillaTest MozillaTest]
| |
| | X
| |
| |-
| |
| |}
| |
| | |
| - * - [[Build:Farm#Tiers_Explained | tiers explained]]
| |
| | |
| === Staging ===
| |
| master:
| |
| * qm-unittest02
| |
| | |
| slaves:
| |
| * qm-stage-centos5-01
| |
| * qm-stage-osx-01
| |
| * qm-stage-winxp01
| |
| * qm-stage-win2k3-01
| |
| | |
| == list of steps to try ==
| |
| If #1 doesn't work, try #2. If that doesn't work, try #3. If #3 doesn't work, contact bhearsum or robcee.
| |
| | |
| '''NOTE!''' If you connect to any of the Windows boxes with a RDP client set to 16-bit color mode, the reftests in <code>/mozilla/modules/libpr0n/test/reftest/</code> will start failing, until the display is reset back to 24-bit mode (eg, by disconnecting and/or rebooting). The tests require the display to be in 24-bit color mode, but Windows will reset the display to whatever the depth of the last client to connect to it was. See [https://bugzilla.mozilla.org/show_bug.cgi?id=414720 bug 414720] for more history. | |
| | |
| | |
| 1. first check
| |
| * check waterfall at: http://qm-rhel02.mozilla.org:2005/ (mpt-vpn)
| |
| * see if slave is connected.
| |
| * if so, click the machine name link, try a "Force Build"
| |
| ** fill out the name and reason fields, click the button
| |
| | |
| 2. second check - restart slave
| |
| * login to machine using provided credentials
| |
| | |
| 2a. Windows Remote Desktop (qm-winxp01, qm-win2k3-01):
| |
| * on windows, ctrl-C in the command window, answer Yes to terminate buildbot process
| |
| * check the Task Manager to see if there are any stuck sh.exe and make.exe processes
| |
| ** if so, reboot the machine using whatever means necessary (the stuck sh.exe and make.exe processes can make shutting down tricky. Kill them if you can)
| |
| ** when machine is rebooted, log back in and open a command prompt
| |
| ** restart buildbot:
| |
| *** cd c:\
| |
| *** buildbot start slave (command does not return)
| |
| *** minimize the cmd.exe window
| |
| | |
| 2b. Mac OS X VNC (qm-xserve01):
| |
| * in the Terminal window, type "buildbot stop slave"
| |
| ** note: message about not returning is expected on Mac
| |
| * if necessary, reboot machine (if it appears to be responding strangely. I think we've only rebooted it once or twice since incept)
| |
| * in a Terminal window,
| |
| ** cd /builds/
| |
| ** buildbot start slave
| |
| | |
| 2c. Linux VNC (qm-centos5-01):
| |
| * in the Terminal, type "buildbot stop slave"
| |
| * reboot if necessary (never needed to yet)
| |
| * restart buildbot:
| |
| ** cd /build/slave (should already be there)
| |
| ** DISPLAY=:2 buildbot start slave
| |
| | |
| 2d. Verify that the slave is connected
| |
| * check the waterfall at: http://qm-rhel02.mozilla.org:2005/
| |
| * the slave sometimes takes a couple of minutes to reconnect
| |
| * if it does, and is necessary, click the machine name link and force a build as above (fill name and reason fields, click the button)
| |
| | |
| 3. clobbering manually
| |
| * Sometimes a machine will need to be "clobbered" (have its objdir directory removed)
| |
| | |
| 3a. Windows Remote Desktop (qm-win2k3-01, qm-winxp01)
| |
| * login to the machine.
| |
| * stop the slave (as above, ctrl-C in the command window)
| |
| * check the task manager to make sure there are no errant sh.exe or make.exe files.
| |
| ** If there are, kill them (End process on sh.exe) or reboot the machine
| |
| * from the Command Line:
| |
| ** cd C:\slave
| |
| ** rmdir /s /q trunk (if qm-winxp01, or rmdir /s /q trunk_2k3 for qm-win2k3-01)
| |
| * restart the slave
| |
| ** cd \
| |
| ** buildbot start slave (command does not return)
| |
| ** minimize cmd.exe window
| |
| | |
| 3b. Linux VNC (qm-centos5-01)
| |
| * login to the machine
| |
| * stop the slave (as above, buildbot stop slave)
| |
| ** cd /build/slave/trunk_linux/mozilla
| |
| ** rm -rf objdir
| |
| * restart the slave
| |
| ** cd /build
| |
| ** buildbot start slave
| |
| | |
| 3c. Mac OS X VNC (qm-xserve01)
| |
| * login to the machine
| |
| * stop the slave (as above, buildbot stop slave)
| |
| ** cd /builds/slave/trunk_osx/mozilla
| |
| ** rm -rf objdir
| |
| * restart the slave
| |
| ** cd /builds
| |
| ** buildbot start slave
| |
| *** message that buildbot didn't return is expected
| |
| | |
| 4. Restarting the Farm
| |
| * In the worst case, the entire buildbot farm needs to be restarted
| |
| * shutdown each slave as per the instructions above
| |
| ** Ctrl-C in Command window on Windows
| |
| ** buildbot stop slave in Terminal on Mac and Linux
| |
| * shutdown master on qm-rhel02
| |
| ** cd /build
| |
| ** buildbot stop master
| |
| * reboot qm-rhel02 and slave machines if necessary (stuck processes, strange behavior)
| |
| * restart master on qm-rhel02
| |
| ** cd /build
| |
| ** buildbot start master
| |
| * restart slaves as above
| |
| ** qm-centos5-01, qm-xserve01, qm-winxp01, qm-win2k3-01
| |
| * verify waterfall at http://qm-rhel02.mozilla.org:2005/ is visible and slaves are connected
| |