ReleaseEngineering/BuildAPI/Setup Local Virtualenv for BuildAPI

The purpose of this wiki is to show you how to setup a virtual environment for buildapi and run buildapi inside it.
Note: These instructions are currently only for Mac OSX

Installing/Creating a VirtualEnv

1. Install pip¹ and virtualenv by running:

   sudo easy_install pip
   sudo pip install virtualenv
   

2. Create and activate your own virtualenv, with your desired name (ie <foo>):

   virtualenv <foo>
   cd <foo>
   source bin/activate
   

Installing Dependencies for your Virtual Environment

1. Install MySQL:
   • Download the most recent .dmg for your version of Mac (for me that was 'Mac OS X ver. 10.7 (x86, 64-bit), DMG Archive')
   • Make a free account, then Oracle will ask what you're doing with MySQL (just tell them, it's no secret) and, finally download it and install the .dmg file
2. Add MySQL to $PATH by running:

   export PATH=$PATH:/usr/local/mysql/bin

3. Install XCode:
   • Go , launch in the App Store and either sign up for a free account or use your existing credentials (It's free, so no stress).
   • Then open XCode, and go to Preferences>Downloads and install 'Command Line Tools'
     • Note 1: If you are sketched out about having your credit card info attached then follow these instructions.
     • Note 2: This should take care of the issues with not having 'clang' or 'gcc' on your machine
4. Install Mercurial, by running:

   pip install mercurial

5. Install the Google Python Visualization Library (version 1.7.1)
   • Note: Based on 'pip freeze' from buildapi01 supplied by catlee
6. Install Redis
7. Create a pip install file by copy/paste'ing the following snippet into a file called 'requirements.txt'
   • Note: This snippet based on 'pip freeze' from buildapi01 supplied by catlee

   Beaker==1.5.4
   FormEncode==1.2.4
   Mako==0.4.1
   MarkupSafe==0.12
   MySQL-python==1.2.3
   Paste==1.7.5.1
   PasteDeploy==1.5.0
   PasteScript==1.7.3
   Pygments==1.4
   Pylons==1.0
   Routes==1.12.3
   SQLAlchemy==0.7.1
   Tempita==0.5.1
   WebError==0.10.3
   WebHelpers==1.3
   WebOb==1.0.8
   WebTest==1.2.3
   amqplib==0.6.1
   anyjson==0.3.1
   carrot==0.10.7
   decorator==3.3.1
   distribute==0.6.14
   nose==1.0.0
   pytz==2011h
   redis==2.4.5
   simplejson==2.1.6
   wsgiref==0.1.2

8. If you haven't already, activate your virtual environment and move into the <foo> directory by running:

   source <foo>/bin/activate
   cd <foo>
   

   • This step ensures that you are installing dependencies ONLY within the virtual environment
9. Install the requirements by running:

   pip install -r requirements.txt

   • Note: If you run across the error "ImportError: No module named pkg_resources", then type this to fix it²:

     curl http://python-distribute.org/distribute_setup.py | python

10. Clone BuildAPI to your local machine:

    hg clone http://hg.mozilla.org/build/buildapi -r dec9d2ddedb715255db9cfce1a890d6a22bc7cd9

Setup SQlite Databases

1. Run the following commands from your <foo>/buildapi/ directory:

   cd buildapi
   sqlite3 schedulerdb
   .databases
   .exit
   sqlite statusdb
   .databases
   .exit
   

Setup and Start BuildAPI

1. To get started with the pylons project, make sure you are in the <foo>/buildapi/ directory and inside your virtualenv, then simply run:

   python setup.py develop

   • This handles grabbing pylon project dependencies. You will also need to grab and install the google python visualization library from here.
2. Now generate a config file for your project, by running:

    paster make-config buildapi config.ini
   

3. Now edit that config.ini
   • Edit config.ini to use the correct database URLs. SQLite databases take the format
     `sqlite:////absolute/path/to/database_name`.
     For example:

     sqlalchemy.scheduler_db.url = sqlite:////Users/<username>/<foo>/buildapi/schedulerdb
     sqlalchemy.status_db.url = sqlite:////Users/<username>/<foo>/buildapi/statusdb
     

   • Add the following line below "branches = b1, b2":

     branches_url = http://www.example.com

     • Note: Without a branches_url, buildapi will complain at you
     • If instead you'd like a sample of real production branches, use this:

       branches_url = http://hg.mozilla.org/build/tools/raw-file/default/buildfarm/maintenance/production-branches.json

   • Change the port listed under [server:main] to 7000, or some other port that is available, like so:

     port = 7000

     • Note: If you run into the error "socket.error: [Errno 48] Address already in use", then try a new port. This is likely because the port you've chosen really is busy, and is not some wacky error in this case.
4. You should now be all set up now! Running the project locally can be done through:

   paster serve --daemon config.ini

   or, if you'd like to monitor output:

   paster serve -v config.ini

   Which starts running the buildapi on your local machine. To view it, open up http://localhost:7000. Or, replace 7000 with whichever port you changed to.

References

1. http://www.tlswebsolutions.com/mac-os-x-lion-setting-up-django-pip-virtualenv-and-homebrew/
2. http://stackoverflow.com/questions/7446187/no-module-named-pkg-resources