DO NOT USE - PLEASE SEE ReferencePlatforms/Mac-10.5

NEW! Reference Image

~~The things we've setup before, originally on bm-xserve16.~~

We now have one image for xserve and one for minis:

macOSX-10.5.2-xserve-ref-image 
macOSX-10.5.2-mini-ref-v3 will be the current image.  
(image is up-to-date at the colo and 650 Castro 20091020)

This document needs to be updated to reflect the changes. The manual steps outlined below are already done and you should only need to create a buildbot slave in the /builds dir.

Things done for you

Darwin Version

These are the kernel versions we're starting with:

Intel: Darwin bm-xserve16.build.mozilla.org 9.2.0 Darwin Kernel Version 9.2.0: Tue Feb 5 16:13:22 PST 2008; root:xnu-1228.3.13~1/RELEASE_I386 i386

This is 10.5.2, up to date on 28/Feb/2008.

Initial Setup and Account Creation

Change the resolution to 1024 x 768, 60 Hz, Millions of Colours, under System Preferences->Displays (this is changed later)
Change the password for the administrator user to the standard build system root password.
Create a new admin-equiv account for the cltbld user using the standard build password.
Under System Preferences->Sharing->Screen Sharing->Computer Settings change the VNC screen control password to the standard build password. NOTE: your VNC session will drop when you do this. You'll need to reconnect with the new password.
If the hostname is wrong, e.g. unused-*, use sudo scutil --set HostName new.hostname.mozilla.org to fix it. NOTE: you'll need to restart the machine for this to fully take effect.
Under System Preferences->Software Update, turn off automatic update checking.

softwareupdate --schedule off

Using the Workgroup Manager, open the Preferences pane for the cltbld user. Under Software Update, set Manage: to Always, and set the Software Update server to use: as http://bm-xserve01.build.mozilla.org:8088/ - NEED TO FIGURE THIS OUT

defaults write /Library/Preferences/com.apple.SoftwareUpdate CatalogURL http://bm-xserve01.build.mozilla.org:8088/

Ref platform packages

Helpfully, OS X doesn't have CVS out of the box, so retrieve the Mac reference platform packages on another machine:

cvs -d :ext:cltbld@cvs.mozilla.org:/mofo co ref-platforms/mac/chud_4.5.0.dmg
cvs -d :ext:cltbld@cvs.mozilla.org:/mofo co ref-platforms/mac/macports-10.5.tar.bz2
cvs -d :ext:cltbld@cvs.mozilla.org:/mofo co ref-platforms/mac/MacPorts-1.6.0-10.5-Leopard.dmg
cvs -d :ext:cltbld@cvs.mozilla.org:/mofo co ref-platforms/mac/xcode_3.0.dmg

This should yield 4 files with md5sums:

3f02477e0df2a3bc53d02fe0824c0015  chud_4.5.0.dmg
514e818170147c3851820defb6151ee1  macports-10.5.tar.bz2
5cbae915b67f7d06576d544b83303714  MacPorts-1.6.0-10.5-Leopard.dmg
e6ce732e203511c9d3102b027a8d89a8  xcode_3.0.dmg

Then transfer them to the machine you are setting up. (We switch to xcode 3.1 later)

Installing Xcode

Mount the Xcode disk image, and begin installing the XcodeTools.mpkg.
A standard install is fine (we don't need 10.3.9 support or WebObjects).

Now we have cvs.

Installing CHUD

Mount the CHUD disk image, install the CHUD.mpkg

Installing MacPorts

Install the MacPorts package from ref platform version 1.6-10.5; ignore the error about the postflight script

Unpack the macports-10.5 tarball of source packages and move them into place:

cd /opt/local/var/macports
sudo rm -rf *
sudo tar jxf ~/ref-platforms/mac/macports-10.5.tar.bz2

Next, you need to install a series of ports, all of which should already be contained (and the right version) in the macports tarball you just unpacked. If you see the port command hitting the network at all (-->Fetching...) while installing the needed ports, something has gone wrong.

sudo port install sqlite3
sudo port install autoconf213 cvs libidl subversion wget

There's a bug with sqlite3 where it fails to compile, depending on what was built before it, so we build it and it's deps first. The second command builds everything else. Afterwards, running port installed should show only the 21 ports listed below:

 apr @1.2.12_1+darwin_9 (active)
 apr-util @1.2.12_0 (active)
 autoconf213 @2.13_0 (active)
 cvs @1.11.22_0 (active)
 db44 @4.4.20_1 (active)
 expat @2.0.1_0 (active)
 gawk @3.1.6_0 (active)
 gettext @0.17_3 (active)
 glib2 @2.14.6_0+darwin_9 (active)
 libiconv @1.12_0 (active)
 libidl @0.8.10_0 (active)
 ncurses @5.6_0 (active)
 ncursesw @5.6_1 (active)
 neon @0.26.4_0 (active)
 openssl @0.9.8g_0 (active)
 pkgconfig @0.23_0 (active)
 readline @5.2.007_0+darwin_9 (active)
 sqlite3 @3.5.6_0 (active)
 subversion @1.4.6_0 (active)
 wget @1.11_0 (active)
 zlib @1.2.3_1 (active)

ssh (these are missing in my image??)

scp the cltbld and appropriate role keys over directly from a machine that already has them installed. In .ssh, make sure the cltbld keys are called cltbld* and not id_dsa*, then symlink

 ln -s cltbld_dsa id_dsa
 ln -s cltbld_dsa.pub id_dsa.pub

This is temporary to get hold of the standard .profile (we should move that file to public cvs).

ENV setup

cvs -d :ext:cltbld@cvs.mozilla.org:/mofo co release/tinderbox-configs/host
ln -s release/tinderbox-configs/host/mac-osx-profile .profile

...at which point you'll probably want to restart your shell to use the new .profile.

switch to role key

In ~/.ssh,

 rm id_dsa id_dsa.pub
 ln -s ffxbld_dsa id_dsa
 ln -s ffxbld_dsa.pub id_dsa.pub

or substitute ffxbld for whatever key is appropriate for what you will be building.

chown scripts

After you've created your initial /builds/tinderbox directory structure, checkout and build the following tools that are also required for changing permissions while packaging on Macs:

cd ~
cvs -d ffxbld@cvs.mozilla.org:/cvsroot co mozilla/build/macosx/permissions
cd mozilla/build/macosx/permissions
gcc -o chown_revert chown_revert.c
gcc -o chown_root chown_root.c
mkdir ~/bin
cp chown_revert chown_root ~/bin
cd ~/bin
chmod 4755 chown*
sudo chown root chown*
ln -s chown_revert revert_root

Establish initial ssh connections and accept keys

Execute the following simple ssh calls to connect to the ftp server (stage), symbol server (if different from ftp), and update server so that the various pieces of the build can be uploaded automatically by the build scripts when needed:

ssh ffxbld@stage.mozilla.org pwd
ssh ffxbld@aus2-staging.mozilla.org pwd
ssh ffxbld@hg.mozilla.org
ssh ffxbld@dm-symbolpush01.mozilla.org pwd

Mail config

NB: Postfix config is tinderbox client specific, all the buildbot information goes over a socket to the master and is posted to the tinderbox server from there.

You'll need to update the postfix config to make sure that myhostname is set correctly and that large messages can be sent. The tail of /etc/postfix/main.cf should look similar to:

myhostname = bm-xserve16.build.mozilla.org
mailbox_transport = cyrus

# Allow large messages (unlimited size, required for tinderbox)
message_size_limit = 0

You'll then want to restart postfix to pick up the changes.

Also make sure that Mail.app is installed.

ntp config

sudo su -
echo "server ntp1.build.mozilla.org iburst" > /etc/ntp.conf
kill -2 `ps x | grep ntp | grep -v grep | awk '{print $1}'`
exit

Tp2 config

NB: This is deprecated by talos.

Set up the server by doing to Application->Server->Server Preferences. Disable File Sharing, iCal, and iChat. Then go to Web, and uncheck "Enable group Wikis", "Webmail", and "User blogs"; set "Home Page" to "Server Home Page".

To setup Tp2:

cd /builds/tinderbox
cvs -d ffxbld@cvs.mozilla.org:/cvsroot mozilla/tools/performance/pageload
cd /Library/WebServer/Documents
ln -s /builds/tinderbox/mozilla/tools/performance/pageload .

nagios

Copy over the appropriate package from bm-admin01, either nrpe-darwin-i386.tar.gz or nrpe-darwin-powerpc.tar.gz depending on your platform.
Create a nagios user and group

sudo dscl localhost -create /Local/Default/Users/nagios
sudo dscl localhost -create /Local/Default/Users/nagios UserShell /usr/bin/false
sudo dscl localhost -create /Local/Default/Users/nagios UniqueID 510
sudo dscl localhost -create /Local/Default/Groups/nagios
sudo dscl localhost -create /Local/Default/Groups/nagios UniqueID 1010

Unpack the package in /usr/local. Eg,

cd /usr/local && sudo tar xfz ~cltbld/nrpe-darwin-i386.tar.gz

Edit/checkout nrpe.cfg. The one in mozilla/tools/nagios/nrpe-mac.cfg may be useful.
Run enablenrpe to turn it on

sudo /usr/local/nagios/sbin/enablenrpe

Ask IT to setup the suite of checks.

Buildbot

Make directories

 mkdir -p /tools/dist/logs
 mkdir -p /builds/logs

Install Python 2.5.2

Python 2.5 doesn't compile on 10.5, and we'll take the latest version rather than the 2.5.1 that ships with the OS.

mkdir -p /tools/dist
cd /tools/dist
wget http://www.python.org/ftp/python/2.5.2/Python-2.5.2.tar.bz2
tar jxvf Python-2.5.2.tar.bz2
cd Python-2.5.2
./configure --prefix=/tools/python
make
make test
make install

export PYTHONHOME="/tools/python"
export PATH="/tools/python/bin:$PATH"

Installing Zope Interface

cd /tools/dist/
wget http://www.zope.org/Products/ZopeInterface/3.3.0/zope.interface-3.3.0.tar.gz
tar xfvz zope.interface-3.3.0.tar.gz 
cd zope.interface-3.3.0/
python setup.py install --prefix=/tools/zope-interface-3.3.0
cd /tools; ln -s zope-interface-3.3.0/ zope-interface
export PYTHONPATH="/tools/zope-interface/lib/python2.5/site-packages/:$PYTHONPATH"

Installing Twisted

cd /tools/dist/
wget http://tmrc.mit.edu/mirror/twisted/Twisted/2.4/TwistedCore-2.4.0.tar.bz2
wget http://tmrc.mit.edu/mirror/twisted/Twisted/2.4/Twisted-2.4.0.tar.bz2
tar xfvj TwistedCore-2.4.0.tar.bz2 
tar xfvj Twisted-2.4.0.tar.bz2 
cd TwistedCore-2.4.0
python setup.py install --prefix=/tools/twisted-core-2.4.0
pushd /tools/; ln -s twisted-core-2.4.0/ twisted-core; popd
export PYTHONPATH="/tools/twisted-core/lib/python2.5/site-packages:$PYTHONPATH"
cd /tools/dist/Twisted-2.4.0/
python setup.py install --prefix=/tools/twisted-2.4.0
cd /tools; ln -s twisted-2.4.0 twisted
export PYTHONPATH="/tools/twisted/lib/python2.5/site-packages/:$PYTHONPATH"

Final Configuration

Add the appropriate paths to the Buildbot user's .bash_profile

echo 'export PYTHONHOME="/tools/python"' >> ~/.profile
echo 'export PYTHONPATH="/tools/buildbot/lib/python2.5/site-packages:/tools/twisted/lib/python2.5/site-packages:/tools/twisted-corelib/python2.5/site-packages/:/tools/zope-interface/lib/python2.5/site-packages/"' >> ~/.profile
echo 'export PATH="/opt/local/bin:/tools/buildbot/bin:/tools/twisted/bin:/tools/twisted-core/bin:$PYTHONHOME/bin:$PATH"' >> ~/.profile

Old Manual Steps (prior to May 27, 2009)

Update to Xcode 3.1

Check out the XCode disk image from cvs (will need to adjust ~/.ssh/id_dsa)

 cvs -d :ext:cltbld@cvs.mozilla.org:/mofo co -d xcode ref-platforms/mac/xcode_3.1.dmg

Check sha1 sum is right

openssl dgst -sha1 xcode_3.1.dmg; \
  echo 'SHA1(xcode_3.1.dmg)= e90f21f262dec5f1dae983e8fb53a613f2cc72b0'

Mount the Xcode disk image, and begin installing the XcodeTools.mpkg.
A standard install is fine (we don't need 10.3.9 support or WebObjects).

Installing Mercurial 1.2.1

cd /tools/dist
wget http://www.selenic.com/mercurial/release/mercurial-1.2.1.tar.gz 
# SHA512(mercurial-1.2.1.tar.gz)= dcadfc731e5e1afae2cfc3eb1ac46875c26d5f029228ea5f397061c05e0524bdf651e3443d8936ec78f51c8d92a904d89c1a45ca941ee8908a99ad5c075096c2
tar xfvz mercurial-1.2.1.tar.gz
cd mercurial-1.2.1
python setup.py install

Copy fonts from another Mac

cd /Library/Fonts
sudo rsync -av cltbld@bm-xserve16:/Library/Fonts/ ./

Set dock size

defaults write com.apple.dock tilesize -int 49; killall Dock

(until a fix is in for bug 470420)

Symlink for autoconf

Due to calling in RepackFactory to command=['bash', '-c', 'autoconf-2.13'] to fix this we have to add a symlink to autoconf213

sudo ln -s /opt/local/bin/autoconf213 /opt/local/bin/autoconf-2.13

"Install" 7z

We need 7zip for the l10n verification step of releases

sudo scp -p cltbld@bm-xserve16:/usr/local/bin/7z* /usr/local/bin/

Setup for verification on staging

Move ~/.ssh to ~/ssh_prod, and scp staging keys in from a staging box. Patch master config to allow new clone to connect and get jobs.

Setup buildbot start on boot

~~cd /Library/LaunchAgents~~

~~sudo wget --no-check-certificate -Obuildbot.start.slave.plist https://bug428124.bugzilla.mozilla.org/attachment.cgi?id=371801 sudo chown root:wheel buildbot.start.slave.plist~~

Ensure the the correct user and slave directory are set in the script (cltbld and /builds/slave by default)

From VNC:

Make sure the resolution is set to 1280x1024. (this is not in the xserve ref image)
System Prefs -> Accounts -> Login Options
- Set 'Automatic Login' to 'cltbld', enter the password when prompted.

Reboot (after setting up the slave, see following step in this doc). NB: The LaunchAgent will try to start buildbot every 10 minutes, be wary of leaving a working buildbot.tac in place during setup

(See bug 428124 for details.)

Manual setup after cloning

If you used the new ref images:

macOSX-10.5.2-xserve-ref-image 
macOSX-10.5.2-mini-ref-image

Then you should only need to do all the bullet points and sections below

Set the hostname if required (IT probably did this, but see above if not)
Change screen resolution to 1280 x 1024, Millions of colours
Set the background to the light blue Solid Colour (better for slower network connections)
Remove crud from doc, leaving Finder, System Preferences, Terminal, Activity Monitor (not essential, just tidier)

Install Buildbot

cd /tools

Check and remove any old buildbot dirs here

hg clone http://hg.mozilla.org/build/buildbot

To ensure uniformity across the pool, work out what revision of buildbot we are currently using and do

cd /tools/buildbot
hg up -r BUILDBOT_PRODUCTION

then

export PYTHONPATH=.:$PYTHONPATH
python setup.py install --prefix=/tools/buildbot

Setup buildbot slave

Remove any /builds/*slave* left over on the image then

mkdir -p /builds/slave
cd /builds/slave

and either

copy over buildbot.tac and info files from another staging slave and update the slavename and basedir
You can also run "buildbot create-slave" with appropriate arguments to point at the staging-master. Note that we need to set "umask = 002" and "keepAlive = None" in buildbot.tac, and that you'll get quite a different config from our old school files elsewhere in the pool. This is not desirable.

Using a VNC session rather than ssh, start the slave with

buildbot start /builds/slave

Switchover from staging to production

After verifying that all is well in staging (builds ok, unit working etc)

stop slave process
switchover the .ssh keys (ie restore ~/ssh_prod
edit /builds/slave/buildbot.tac to change "staging-master" to "production-master".
clobber all build trees on slave
start slave process

ReferencePlatforms/Mac-10.5.puppet

Contents