User:Bhearsum:Build/Buildserver ref image

WIP

Buildserver Ref Image

This document describes how to create a "Build server" ref image. More info is in bug#385911.

Copy Linux Buildbot ref image

1. VM->Clone
2. Next
3. If possible, use a clean snapshot of a machine, otherwise choose "The current state" and click Next.
4. 'Create a full clone', Next.
5. Choose a name and location and click Next.
6. Finish

Add a 10GB drive

1. VM->Settings
2. Click Add
3. Select 'Hard Disk' and click Next
4. "Create a new virtual disk" and click Next
5. SCSI, click Next
6. Disk Size: 10GB, click Next
7. Choose location, Finish

Boot up the VM and 'mkdir /cvs'. Create a single partition on the new hard drive (/dev/sdb). mkfs -t ext3 -j /dev/sdb1 echo "/dev/sdb1 /cvs ext3 defaults 0 0" >> /etc/fstab mount -a

Install and setup CVSD

Install CVSD:

wget http://ch.tudelft.nl/~arthur/cvsd/cvsd-1.0.13.tar.gz
tar -zvxf cvsd-1.0.13.tar.gz
cd cvsd-1.0.13
./configure
make
make install

Setup CVSD:

mkdir /etc/cvsd
cp /usr/local/etc/cvsd/cvsd.conf /etc/cvsd
cp /usr/local/etc/init.d/cvsd /etc/init.d
chkconfig --add cvsd

Edit /etc/cvsd/cvsd.conf and change the following parameters:

RootJail /cvs
Listen * 2401 
Repos /cvsroot
CvsArgs -u

Run these commands:

cvsd-buildroot /cvs
mkdir -p /cvs/var/lock
adduser cvsd

Install CVS

CVS must be patched to work in this setup.

wget http://ftp.gnu.org/non-gnu/cvs/source/stable/1.11.22/cvs-1.11.22.tar.bz2
wget http://people.mozilla.com/~bhearsum/Build/Misc/cvs.noreadlocks.patch
tar -jvxf cvs-1.11.22.tar.bz2
cd cvs-1.11.22/src
patch -p0 < ../../cvs.noreadlocks.patch
./configure --prefix=/cvs
make
make install

Rsync cvs-mirror.m.o

These instructions taken from How to Create a CVS Mirror:

Create a directory for the repository

mkdir /cvs/cvsroot

Add that directory to your cvsd.conf

echo "/cvsroot" >> /etc/cvsd/cvsd.conf

Create an 'rsync-excludes' file to ignore history, users

echo "CVSROOT/passwd" > /cvs/etc/rsync-excludes
echo "CVSROOT/writers" >> /cvs/etc/rsync-excludes
echo "CVSROOT/history" >> /cvs/etc/rsync-excludes
echo "CVSROOT/history*" >> /cvs/etc/rsync-excludes

Copy this script that will perform an rsync of the Mozilla CVS

/etc/init.d/cvsd stop
rsync -q -az --delete --exclude-from=/cvs/etc/rsync-excludes cvs-mirror.mozilla.org::mozilla /cvs/cvsroot
/etc/init.d/cvsd start

If you want to continually update your mirror you should create a cron job to do so. Otherwise you can run the above script at any time to do it manually.

Create a buildbot master

mkdir -p /buildbot/master
chown -R buildbot /buildbot
su - buildbot
cd /buildbot/master
buildbot create-master .
mv Makefile.sample Makefile

master.cfg:

# -*- python -*-
# ex: set syntax=python:

# This is a sample buildmaster config file. It must be installed as
# 'master.cfg' in your buildmaster's base directory (although the filename
# can be changed with the --basedir option to 'mktap buildbot master').

# It has one job: define a dictionary named BuildmasterConfig. This
# dictionary has a variety of keys to control different aspects of the
# buildmaster. They are documented in docs/config.xhtml .


# This is the dictionary that the buildmaster pays attention to. We also use
# a shorter alias to save typing.
c = BuildmasterConfig = {}


####### PROJECT IDENTITY
c['projectName'] = "VMware Team Buildbot"
c['buildbotURL'] = "[IP ADDRESS HERE]:8810"
c['slavePortnum'] = 9990

####### BUILDSLAVES

# the 'bots' list defines the set of allowable buildslaves. Each element is a
# tuple of bot-name and bot-password. These correspond to values given to the
# buildslave's mktap invocation.
c['bots'] = [("win32", "w1nbu1ld3r"),
             ("linux", "l1nuxbu1ld3r")]

####### STATUS PLUGINS

from buildbot.status import html

c['status'] = []
c['status'].append(html.Waterfall(http_port=8810, allowForce=True))


####### SOURCES

c['sources'] = []


####### SCHEDULERS

c['schedulers'] = []


####### BUILDERS

c['builders'] = []

from mozbuild import *
from buildbot.process import factory
from buildbot.steps.shell import Configure,Compile

s = factory.s

## unix builder

trunk_unix_steps = [
    s(MozillaCheckoutClientMk, workdir=".",
      cvsroot=":pserver:anonymous@10.0.0.1:/cvsroot"),
    s(MozillaClientMkPull, workdir="mozilla"),
    s(Configure,
      workdir="mozilla",
      command=["./configure",
               "--enable-application=browser"]),
    s(Compile, workdir="mozilla"),
]

trunk_unix_builder = {
    'name': "Build Team Default Unix Builder",
    'slavenames': ['linux'],
    'builddir': "trunk-unix",
    'factory': factory.BuildFactory(trunk_unix_steps),
    'category': "default",
}
c['builders'].append(trunk_unix_builder)


## win32 builder

trunk_win32_steps = [
    s(MozillaCheckoutClientMk, workdir=".",
      env=MozillaEnvironments['vc8_express']),
    s(MozillaClientMkPull, workdir="mozilla",
      env=MozillaEnvironments['vc8_express']),
    s(Configure,
      workdir="mozilla",
      env=MozillaEnvironments['vc8_express'],
      command=["bash", "-f", "configure",
               "--enable-application=browser",
              ]),
    s(Compile, workdir="mozilla", env=MozillaEnvironments['vc8_express'])
]

trunk_win32_builder = {
    'name': "Build Team Default Win32 Builder",
    'slavenames': ['win32'],
    'builddir': "trunk-win32",
    'factory': factory.BuildFactory(trunk_win32_steps),
    'category': "default"
}
c['builders'].append(trunk_win32_builder)

## END OF DEFAULT MASTER.CFG

TODO: Have Buildbot start with the machine.