ReleaseEngineering/How To/Fix the RAID config on an HP Machine

From MozillaWiki
Jump to: navigation, search

This class of machine can experience a problem where they lose their RAID config at boot. This problem is tracked is bug 779487. If you see an HP machine at a "Welcome to the Mozilla Corporation Network Installer" screen, it needs to have its RAID config fixed. You can do that with the following step:

  1. Re-open (or create) the host specific bug blocked by bug 779487, and make sure the bug is assigned to "Server Operations: DCOps", adding the current details.
  2. Wait for DCOps to do their thing. Mostly, this occurs with mock-builders, and production has burst coverage via AWS.

DCOps is working this problem closely with HP, and there may be new patches and/or firmware they will want to apply. Letting them do the work ensures the latest information is used.

Urgent Workaround Instructions

If you need the host back in service ASAP (e.g. it's the only mock builder in staging), you can use the following steps to temporarily work around the issue:

  1. Assign the #bug to yourself, and indicate you're going to attempt the workaround.
  2. Open up the management console for the machine (for example, http://bld-centos6-hp-004-mgmt.build.mozilla.org) - if this doesn't work on your local machine, RDP To winadmin to do it.
  3. Click through the "This connection is untrusted" warning and login
  4. Open the Remote Console ("Remote Console" → "Launch" under Java Integrated Remote Console). Click through any warnings/confirmations it throws up.
  5. Reset the machine ("Power Switch" → "Reset")
  6. Once the "HP Smart Array" screen comes up, press F8 until you get into the "Option Rom Configuration for Arrays" screen. If you end up in the main BIOS setup, start again.
  7. Hit ESC once to get to the main menu of this configuration screen.
  8. Choose "Create Logical Drive"
  9. Choose the defaults ("Port 1, Box 1, Bay 1" for the drive, "RAID 0" for the RAID config, "Use one drive as spare" should be unchecked, and "Disable" Maximum Boot partition)
  10. Press F8 to confirm, Enter to continue, then ESC to exit.
  11. The machine should boot to CentOS and reenter the slave pool.
  12. Document that you performed this procedure in the #bug and resolve the bug. (There's no action for DCOps if the machine isn't hanging at the prompt.)
    • If the workaround did not succeed, be sure the #bug is unassigned in component "Server Operations: DCOps"