ReferencePlatforms/Win32: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(→‎Version 10: Twisted patch for process killing)
(Updated ref image with second drive and ssh keys)
Line 1: Line 1:
This procedure will re-create the Windows 2003 based reference image. The current version is '''Version 10'''. Internally, this VM is named '''win2k3sp2-vc8tools-ref-vm'''.
This procedure will re-create the Windows 2003 based reference image. The current version is '''Version 11'''. Internally, this VM is named '''win2k3sp2-vc8tools-ref-vm'''.


= Things Done For You =
= Things Done For You =
Line 272: Line 272:
  patch -p8 < patch
  patch -p8 < patch


= Manual Setup Steps after VMImage =
== Version 11 ==


== Setup extra disk ==
=== Setup extra disk ===
* finish disk setup by
* finish disk setup by
** logging into machine as Administator
** logging into machine as Administator
Line 284: Line 284:




== setup ssh keys ==
=== setup ssh keys ===
* manually copy  ssh keys from another staging / production VM and install into /c/Documents and Settings/cltbld/.ssh.
* manually copy  ssh keys from another staging / production VM and install into /c/Documents and Settings/cltbld/.ssh.
* also copy them to /.ssh in msys
* also copy them to /.ssh in msys
= Manual Setup Steps after VMImage =


== Installing Buildbot ==
== Installing Buildbot ==

Revision as of 21:23, 7 November 2008

This procedure will re-create the Windows 2003 based reference image. The current version is Version 11. Internally, this VM is named win2k3sp2-vc8tools-ref-vm.

Things Done For You

Version 3

Configuration Details

VM Created with ISO: EN_WIN2003_ENTWITHSP1_WIN2003_STANDARDWITHSP1.ISO MD5 sum: 9f584062feca857738ac794a8bc46936 From MSDN CD: Disc 2429.4, November 2005

Includes Downloaded Software Updates

  • Security Update for Windows Server 2003 (KB917734)
  • Security Update for Windows Server 2003 (KB914389)
  • Security Update for Windows Server 2003 (KB917344)
  • Security Update for Windows Server 2003 (KB918439)
  • Security Update for Windows Server 2003 (KB911280)
  • Cumulative Security Update for Internet Explorer for Windows Server 2003 (KB916281)
  • Security Update for Windows Server 2003 (KB917953)
  • Windows Malicious Software Removal Tool - June 2006 (KB890830)
  • Security Update for Windows Server 2003 (KB908531)
  • Cumulative Security Update for Outlook Express for Windows Server 2003 (KB911567)
  • Security Update for Windows Server 2003 (KB911562)
  • Security Update for Windows Server 2003 (KB911927)
  • Security Update for Windows Server 2003 (KB908519)
  • Security Update for Windows Server 2003 (KB912919)
  • Update for Windows Server 2003 (KB910437)
  • Security Update for Windows Server 2003 (KB904706)
  • Security Update for Windows Server 2003 (KB896424)
  • Security Update for Windows Server 2003 (KB900725)
  • Security Update for Windows Server 2003 (KB905414)
  • Security Update for Windows Server 2003 (KB902400)
  • Security Update for Windows Server 2003 (KB899589)
  • Security Update for Windows Server 2003 (KB901017)
  • Security Update for Windows Server 2003 (KB893756)
  • Security Update for Windows Server 2003 (KB896428)
  • Security Update for Windows Server 2003 (KB896422)
  • Security Update for Windows Server 2003 (KB896358)
  • Security Update for Windows Server 2003 (KB899587)
  • Security Update for Windows Server 2003 (KB890046)
  • Security Update for Windows Server 2003 (KB899591)
  • Security Update for Windows Server 2003 (KB899588)
  • Security Update for Windows Server 2003 (KB901214)
  • Update for Windows Server 2003 (KB898715)

Configuration Changes

  • Turned OFF all automatic updates
  • Turned ON firewall; allowed RDP and SSH
  • Turned off all MS networking, leaving only TCP/IP
  • Created cltbld account
  • Turned on RDP; allowed cltbld and Administrator to connect
  • Turned performance settings to forground programs -> max
  • Turned off screensavers
  • Set networking to DHCP (so, will have to assign an IP)
  • Installed VMware tools; expects vmxnet networking

VC8 CONFIGURATION DETAILS

Installed from ISOs: 1cc97c96e3b037e4514b8f0fe6929862 en_vs_2005_pro_cd1.iso 7ac81a9a1c1571ed7e204fc96093e0c7 en_vs_2005_pro_cd2.iso

Includes ONLY the following VC8 components:

  • Visual C++
  • Tools for Redistributing Applications

FURTHER CONFIGURATION DETAILS

  • Add 15 gig d:\ drive
  • install cygwin into d:\cygwin
  • download cygwin packages to d:\packages (from www.cygwin.com; check CVS)
  • download vc8 moztools packages into d:\packages (from MDC; check CVS)
  • install vc8 tools into d:\moztools
  • rename link.exe to cyglink.exe (per MDC instructions)
  • Changed the following environment variables (system wide) in the System control panel:
 -- CYGWINBASE: d:\cygwin
 -- VCVARS: C:\Program Files\Microsoft Visual Studio 8\VC\bin\vcvars32.bat
  • Setup openssh to start as a service by running ssh-host-config
 (http://pigtail.net/LRP/printsrv/cygwin-sshd.html)
  • Check out mofo/release/tinderbox-configs/host/ssh in ~; ln -s .ssh/ to it.
  • Change font on Cygwin window
  • Install Blat:
 c25c74ecb8fe8e670e204e7196679190  blat-2.50/Blat250.source.zip
 66fc3b95fde52fa1887c9d8aab472033  blat-2.50/Blat250.zip
    • unzip Blat250.zip
    • mv -v full/blat.* /cygdrive/d/moztools/bin
    • Configure Blat: blat -install rheet.mozilla.org build@mozilla.org 5
  • Install 7Zip
 1c412567bb5473911a44ce17b5aa02dd  7zip-4.32/7z432.exe
 d55f567a62153bc43a04e9c52f210cdd  7zip-4.32/7z432.tar.bz2
    • Run the installer; install in d:\buildtools\7zip
  • Install UPX:
 852bd691d8abc75b52053465846fba34  upx-1.25/ucl-1.03.tar.gz
 6f20a62999a46a1864652454b3c8a5d8  upx-1.25/upx-1.25-src.tar.gz
 a092272e9e6995764adfcd4fb1e729db  upx-1.25/upx125w.zip
    • unzip upx125w.zip into d:\buildtools\upx
  • Install NSIS:
 32b1a327d3611f01053c3a2677e50d7b  nsis-2.17/nsis-2.17-setup.exe
 a3c068fc1cb613160d7202e1719d63bc  nsis-2.17/nsis-2.17-src.tar.bz2
    • Run the installer; full install; install in d:\buildtools\NSIS
    • Delete desktop icon
  • Set the perms on the installed tools correctly
    • chgrp -R users d:\buildtools
    • chmod -Rv 0755 d:\buildtools

Version 4

Was created from pre-tools installation snapshot of the above Win32 VM, so Cygwin, etc. weren't ever installed (from this VM's perspective).

  • Installed Win2k3 Service Pack 2 for x86 32-bit.
  • Installed post-SP2 updates:
    • Windows Malicious Software Removal Tool - March 2007 (KB890830)
    • Update for Windows Server 2003 Service Pack 2 (KB931836)
    • MSXML 6.0 RTM Security Update (KB927977)
    • MSXML 6.0 RTM Security Update (925673)

Version 5 (August 2007 Update/Refresh)

Configuration Changes

These are all VMware-related changes:

  • Upgrade virtual hardware in VI (ESX 3 virtual machine)
  • Change to LSI logic SCSI controller
  • Upgrade VMware tools to build 38803

Includes Automatic Updates

In Order:

  • Run Windows Updater
    • Install Microsoft Update Active X control
    • Install Microsoft Windows Component Publisher Active X control
    • Click "Update Windows"
  • Update to Windows 2k3 Service Pack 2 (32-bit x86, released 6/12/2007)

Individual Updates:

Windows 2003 Server

  • Security Update for Windows Media Player 6.4 (KB925398)
  • Windows Malicious Software Removal Tool - July 2007 (KB890830)
  • Security Update for Microsoft .NET Framework, Version 2.0 (KB928365)
  • Security Update for Microsoft .NET Framework, Version 1.1 Service Pack 1 (KB933854)
  • Security Update for Windows Server 2003 (KB926122)
  • Security Update for Windows Server 2003 (KB924667)
  • Cumulative Security Update for Internet Explorer 6 for Windows Server 2003 (KB933566)
  • Cumulative Security Update for Outlook Express for Windows Server 2003 (KB929123)
  • Security Update for Windows Server 2003 (KB935840)
  • Security Update for Windows Server 2003 (KB935839)
  • Update for Windows Server 2003 (KB927891)
  • MSXML 6.0 RTM Security Update (KB927977)
  • Security Update for Windows Server 2003 (KB930178)
  • Security Update for Windows Server 2003 (KB931784)
  • Security Update for Windows Server 2003 (KB932168)
  • Security Update for Windows Server 2003 (KB925902)
  • Update for Windows Server 2003 Service Pack 2 (KB931836)

Microsoft CAPICOM

  • Security Update for CAPICOM (KB931906)

Microsoft SQL Server Feature Pack

  • MSXML 6.0 RTM Security Update (925673)

Microsoft Visual Studio 2005

  • Security Update for Microsoft Visual Studio 2005 (KB925674)

to install

  • Visual Studio 2005 Service Pack 1
  • Install MozillaBuild 1.1 into /d/mozilla-build
  • Run sysprep-cd scripts in Win32 directory of the moz-sysprep-cd (stored in mofo repo, public copy)

Version 6 (January 2008 Update/Refresh)

Install Microsoft® Windows® Software Development Kit Update for Windows Vista

http://www.microsoft.com/downloads/details.aspx?familyid=4377F86D-C913-4B5C-B87E-EF72E5B4E065&displaylang=en

  • Make sure you get the Web Install
    • ONLY install the 'Developer Tools' (all of them). Do NOT install Documentation or Samples.
    • Install to 'd:\sdks\v6.0'
    • Install Samples to 'd:\sdks\v6.0\Samples'
  • After installing it, it should show up in Add/Remove Programs as 'Microsoft Windows Software Development Kit for Windows Vista Update (6000.16384.10)'.

Install MozillaBuild 1.2

  • Delete everything in d:\mozilla-build
  • Uninstall previous NSIS (v2.17) installation.
  • Download MozillaBuild 1.2 and install to d:\mozilla-build.
    • MozillaBuild1.2 includes NSIS v2.22

Version 7

Installing the NRPE daemon

  • Download NSClient++ and extract to C:\Program Files\nscp
  • Checkout mozilla/tools/nagios and copy NSC.ini to C:\Program Files\nscp
  • Open up cmd.exe and do the following:
cd C:\Program Files\nscp
nsclient++ /install
nsclient++ /start
  • Added firewall exceptions for port 5666 (tcp+udp) and ICMP ping replies.
    1. Go to Control Panel -> Windows Firewall
    2. Click on the 'Exceptions' tab.
    3. Click 'Add Port'
    4. Name: 'nscp', Port: '5666', (*) TCP
    5. Repeat the above for (*) UDP
    1. Go To Control Panel -> Windows Firewall
    2. Click on the 'Advanced' tab.
    3. Click on ICMP -> Settings
    4. Check the box for 'Allow incoming echo request'
    5. Hit OK, OK.

Version 8

Move to Python 2.5

  • Delete d:\buildbot (which contains python24)
  • Append ';d:\mozilla-build\python25' to system path.
  • Install this registry key:
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\2.5\InstallPath]
@="d:\\mozilla-build\\python25"

Install Mercurial

  • Download Mercurial and install to d:\mercurial.
  • Append ';d:\mercurial' to the system path.

Version 9

Disable JIT Debugger

Per this MSDN entry: 

Under the registry key: 
HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\Current Version\AeDebug

set "Auto"=0
rename the "Debugger" variable to "VDebugger"
create a new string entry "Debugger" with an empty string

Under the registry key: 
HKEY_LOCAL_MACHINE\Software\Microsoft\.NETFramework\DbgJITDebugLaunchSetting

set value to 1 (dumps stack and exits)

Version 10

Fix registry setting

In version 9, we accidentally set 

 HKEY_LOCAL_MACHINE\Software\Microsoft\.NETFramework\DbgJITDebugLaunchSetting

to hexidecimal 10 instead of 1. Not clear if this will fix bug 453260.

Setup environment vars

  • finish setting up environment vars by:
    • logging into machine as Administrator
    • in control panel, system, advanced tab, environment
    • set MOZ_NO_RESET_PATH=1

Downgrade twisted and apply process group patch

To fix a long running saga with win32 machines not recovering when an executable dies, we downgrade Twisted to version 2.5.0 and patch _dumbwin32proc.py so that twisted (and hence buildbot) can kill process groups properly. 

rm -rfv /d/mozilla-build/python25/Lib/site-packages/{twisted/,Twisted-8.0.1-py2.5.egg-info/,zope/} /d/mozilla-build/python25/scripts/twistd*
wget http://tmrc.mit.edu/mirror/twisted/Twisted/2.5/Twisted_NoDocs-2.5.0.win32-py2.5.exe
./Twisted_NoDocs-2.5.0.win32-py2.5.exe
cd /d/mozilla-build/python25/Lib/site-packages/twisted/internet/
wget -O patch "http://twistedmatrix.com/trac/attachment/ticket/2726/_dumbwin32proc.patch?format=raw"
patch -p8 < patch

Version 11

Setup extra disk

  • finish disk setup by
    • logging into machine as Administator
    • right click on MyComputer, and select "Manage"
    • click "DiskManagement" and select "Rescan"
    • see unformatted partition. Assign it drive "E", and format it as FAT32 (not NTFS)
    • save (and possibly reboot?).


setup ssh keys

  • manually copy ssh keys from another staging / production VM and install into /c/Documents and Settings/cltbld/.ssh.
  • also copy them to /.ssh in msys


Manual Setup Steps after VMImage

Installing Buildbot

Because our Buildbot code changes fairly regularly we don't keep a copy of it on the VM. In MSYS window, run: 

cd /c/Documents and Settings/cltbld
cvs -d :pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot co -d buildbot-077 -r BUILDBOT_0_7_7_BRANCH mozilla/tools/buildbot

NOTE:

  • For staging (the previous will most likely not work) use "-d staging-1.9-master:/builds/cvsmirror/cvsroot" instead
  • For people outside of MoCo build network, use "-d :pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot" instead.
cd buildbot-077
python setup.py build
python setup.py install

Accept SSH keys

Manually use ssh to connect to the following machines, and manually accept "yes" the first connection 

# production
$ ssh -i ~/.ssh/ffxbld_dsa ffxbld@aus2-staging.mozilla.org
$ ssh -i ~/.ssh/ffxbld_dsa ffxbld@dm-symbolpush01.mozilla.org
$ ssh -i ~/.ssh/ffxbld_dsa stage.mozilla.org
# staging
$ ssh -i ~/.ssh/ffxbld_dsa staging-stage.build.mozilla.org

Setup Slave

cd /e
mkdir -p builds/moz2_slave

...and either run "buildbot create-slave" or else copy over buildbot.tac and info files from another win32 slave.

Start Slave

$ cd 
$ /d/mozilla-build/python25/scripts/buildbot start /e/builds/moz2_slave

verify setup in staging

  • trigger a build on staging-master, to make sure all works ok

switchover from staging to production

  • stop slave process
  • switchover the .ssh keys to use production keys
  • edit /e/builds/moz2_slave/buildbot.tac to change "staging-master" to "production-master".
  • start slave process
Retrieved from "https://wiki.mozilla.org/index.php?title=ReferencePlatforms/Win32&oldid=115713"