ReleaseEngineering/worker configuration/2008

From MozillaWiki
Jump to: navigation, search

Overview

As of 2015-11-24 the configuration of 2008 buildbot slaves consists of 4 parts. The initial manual build of the AWS AMI, application of the 2008 PuppetAgain catalog, manual configuration changes, and lastly the EC2 userdata scripts.

Manual configuration changes will eventually be part of PuppetAgain, and as well as portion of the manual AMI creation. These were currently left out do to the priority of getting buildbot builds working in AWS over the individual automation and management pieces.

AMI Creation

Starting Operating System

Due to a deep level issue affecting hg clone and hg bundle times we move away from using a custom AMI built of a Microsoft Deployment Workbench image and starting with an AWS vanilla AMI: Windows_Server-2008-R2_SP1-English-64Bit-Base-2015.09.09 (ami-31620c54).

Installed Applications

  • Visual Studio 2010
    • ISO: SW_DVD5_NTRL_Visual_Studio_2010SP1_MultiLang_FPP_VL_MLF_X17-40329
      • Resulting Version: 10.0.30319.1 RTMRel
  • Visual Studio 2013
    • ISO: en_visual_studio_premium_2013_with_update_3_x86_dvd_4836351
      • Resulting Version:
  • Puppet Agent
  • Other Applications that can be moved into PuppetAgain management
    • Direct X SDK
    • Windows 8 SDK

Other Configuration

  • Rename administrator account to Root
    • Can be moved into the EC2 userdata scripts pre-Puppet run
  • Roll back patching level to 2011-06
    • ADD script
  • Set os_version variable "reg add "HKLM\System\CurrentControlSet\Control\Session Manager\Environment" /v os_version /t REG_SZ /d 2008"

Puppet Configuration =

Manual Configuration/Tweaks

(These tweaks have been ported into Puppet. Keeping the list here for historical reference.)

Mercurial Tweaks

  • File copies into C:\mozilla-build\hg
  • Lines added to the mercurial.ini
    • Under [extensions]
      • bundleclone=C:\mozilla-build\hg\bundleclone.py
      • share =
    • Under [bundleclone]
      • prefers = ec2region=us-west-2, stream=revlogv1
        • This line should be ultimately be managed with the EC2 userdata scripts to determine which location for specific AWS region
    • Under [ui]
      • username = Mozilla Release Engineering <release@mozilla.com>
      • traceback = True
    • Under [hostfingerprints]
      • hg.mozilla.org = af:27:b9:34:47:4e:e5:98:01:f6:83:2b:51:c9:aa:d8:df:fb:1a:27

Runner Configuration

Buildbot Tweaks

  • File copies were needed specifically to have sendchange work. Reference: https://bugzilla.mozilla.org/show_bug.cgi?id=1175701
    • Copy from C:\mozilla-build\buildbotve\Lib\site-packages to C:\mozilla-build\python27\Lib\site-packages:
      • buildbot module
      • twisted module
      • zope module
      • Twisted-10.2.0-py2.7.egg-info
      • zope.interface-3.6.1-py2.7-nspkg.pth

Services

  • Disable Windows Defender Service
  • Disable Puppet Service

Other File Changes

  • SSH keys, API keys, and API tokens need to be copied over to C:\builds and C:\users\cltbld\.ssh
    • This should happen through the Puppet run. However, currently it is not happening in AWS.
  • Cltbld permissions added to:
    • C:\builds\crash-stats-api.token
    • C:\builds\google-oath-api.key
  • Add AWS credentials to C:\Users\cltbld\.boto
  • Update yasm.exe to 1.3 in C:\mozilla-build\

EC2 Service Userdata Scripts

OLD PRE 2015-11 Configuration Notes (Incomplete)

Overview

The building of the 2008 builder slave image uses both the Microsoft Deployment Workbench (MDT) and Puppet configuration software. In the case of AWS instances the Amazon EC2 service is also utilized.

The initial steps is building a base image in MDT. The base image is comprised of large applications that will seldom change, Microsoft SDKs, applications that may aid in deployment, and Windows Server Update Service (WSUS) components.

The second portion of this differs according to datacenter deployment or cloud deployment. Differences are documented below.However, in both cases the image has Puppet installed, and the image is prep to be ran against the PuppetAgain. As well a configurations that Puppet needs to run or at the time made more sense to be done through MDT.

General MDT documentation: https://mana.mozilla.org/wiki/display/SYSADMIN/Microsoft+Deployment+Toolkit+2012+Update+1+General+Documentation

Windows 2008 Build Slave Configuration Managed by Microsoft Deployment Workbench Configuration

Starting Operating System

The process is begun with Windows Server 2008 R2 SP1 iso (en_windows_server_2008_r2_with_sp1_vl_build_x64_dvd_617403.iso). The iso file is mounted, in our case, using Virtual CloneDrive. Once the iso file is mounted it is then imported into the MDT.

Base Configuration Capture (MDT Task Sequence ID SW2008_BASE_001)

The modification here is found under tasks sequence steps -> State Restore -> Custom Tasks -> Mozilla Custom.

Applications

Microsoft Visual Studio 2010

  • This application install varies away from the typical MDT install. The files needed to be extracted from the iso and have the F# portion removed. The extracted files are then copied over local, and the installation command is ran

Microsoft Visual Studio 2013 Update 3

  • A reboot needs to occur between the installation of 2010 and 2013

Microsoft Windows Windows 8 SDK

Microsoft Windows Windows Direct X SDK

Microsoft Net Framework 3.5

Configuration

WSUS Updates


Data Center Deployment Configuration (MDT Task Sequence SW2008_DPLY_001)

The modification here is found under tasks sequence steps -> State Restore -> Custom Tasks -> Mozilla Custom.

Applications

Puppet

  • Package installation
    • This installs version 3.4.3
    • Future updates will be managed through Puppet
  • Kill the Puppet service
    • In order to control when Puppet runs
  • Set Puppet Registry bits
    • "Type"=dword:00000010
    • "Start"=dword:00000003
    • "ErrorControl"=dword:00000001
  • Copy over WGET from "Z:\Applications\WGET" to "C:\Windows\System32"
    • Needed to run the Puppetize vbs to obtain certs
  • Copy over puppet.conf from Z:\Applications\Puppet Puppet\puppet.conf" to "C:\programdata\Puppetlabs\puppet\etc\"
    • A copy of pupept.conf is needed prior to the Pupeptize vbs being ran
    • After this initial copy the file is managed by Puppet
  • Copy over the Pupeptize vbs from "Z:\Scripts\" to "C:\ProgramData\Puppetlabs\puppet\var\"
  • Run Puppetize vbs
  • Run Puppet

Configuration

Rename Administrator to root

  • Copy over security template, root.inf, from "Z:\Files\root"
  • Apply template cmd /c secedit /configure /db %temp%\temp.sdb /cfg C:\root.inf
  • Set registry so that root may continue to auto-login during task sequence execution

Set a os_version "reg add "HKLM\System\CurrentControlSet\Control\Session Manager\Environment" /v os_version /t REG_SZ /d 2008"

    • For Puppet use. Currently there is no facter fact to differentiate between Windows OSes

Disable Server Manager Window "reg add HKCU\Software\Microsoft\ServerManager /t REG_DWORD /v DoNotOpenServerManagerAtLogon /d 1 /f

  • Because the registry setting is HKey current user it is quite difficult to implement through Puppet
    • Needs to be done after reboot post Puppet run


Cloud Deploy Configuration Capture (MDT Task Sequence ID W2008_CLD_002)

The modification here is found under tasks sequence steps -> State Restore -> Custom Tasks -> Mozilla Custom.

Puppet

  • Package installation
    • This installs version 3.4.3
    • Future updates will be managed through Puppet
  • Kill the Puppet service
    • In order to control when Puppet runs
  • Copy over WGET from "Z:\Applications\WGET" to "C:\Windows\System32"
    • Needed to run the Puppetize vbs to obtain certs
  • Copy over puppet.conf from Z:\Applications\Puppet Puppet\puppet.conf" to "C:\programdata\Puppetlabs\puppet\etc\"
    • A copy of pupept.conf is needed prior to the Pupeptize vbs being ran
    • After this initial copy the file is managed by Puppet
  • Copy over the Pupeptize vbs from "Z:\Scripts\" to "C:\ProgramData\Puppetlabs\puppet\var\"
  • Run Puppetize vbs
  • Run Puppet

EC2 config service

  • Installation ""Z:\Applications\AWS EC2config service\Ec2Install.exe" /passive /norestart2config service\"

Citrix PV

  • Copy from "Z:\Applications\Citrix-Win_PV" to "c:\Citrix-Win"
  • "C:\Citrix-Win\upgrade.bat" to install

Configuration

Rename Administrator to root

  • Copy over security template, root.inf, from "Z:\Files\root"
  • Apply template cmd /c secedit /configure /db %temp%\temp.sdb /cfg C:\root.inf
  • Set registry so that root may continue to auto-login during task sequence execution

Set a os_version "reg add "HKLM\System\CurrentControlSet\Control\Session Manager\Environment" /v os_version /t REG_SZ /d 2008"

    • For Puppet use. Currently there is no facter fact to differentiate between Windows OSes
Retrieved from "https://wiki.mozilla.org/index.php?title=ReleaseEngineering/worker_configuration/2008&oldid=1139357"