ReleaseEngineering/BuildAPI/Setup Local Virtualenv for BuildAPI

The purpose of this wiki is to show you how to setup a virtual environment for buildapi and run buildapi inside it.
Note: These instructions are currently only for Mac OSX

Installing/Creating a VirtualEnv

1. Install pip¹ and virtualenv by running:

   sudo easy_install pip
   sudo pip install virtualenv
   

2. Create and activate your own virtualenv, with your desired name (ie <foo>):

   virtualenv <foo>
   cd <foo>
   source bin/activate
   

Installing Dependencies for your Virtual Environment

1. Install MySQL:
   • Download the most recent .dmg for your version of Mac (for me that was 'Mac OS X ver. 10.7 (x86, 64-bit), DMG Archive')
   • Make a free account, then Oracle will ask what you're doing with MySQL (just tell them, it's no secret) and, finally download it and install the .dmg file
2. Add MySQL to $PATH by running:

   export PATH=$PATH:/usr/local/mysql/bin

3. Install XCode:
   • Go , launch in the App Store and either sign up for a free account or use your existing credentials (It's free, so no stress).
   • Then open XCode, and go to Preferences>Downloads and install 'Command Line Tools'
     • Note 1: If you are sketched out about having your credit card info attached then follow these instructions.
     • Note 2: This should take care of the issues with not having 'clang' or 'gcc' on your machine
4. Install Mercurial, by running:

   pip install mercurial

5. Install the Google Python Visualization Library (version 1.7.1)
   • Note: Based on 'pip freeze' from buildapi01 supplied by catlee
6. Install Redis
7. Create a pip install file by copy/paste'ing the following snippet into a file called 'requirements.txt'
   • Note: This snippet based on 'pip freeze' from buildapi01 supplied by catlee

   Beaker==1.5.4
   FormEncode==1.2.4
   Mako==0.4.1
   MarkupSafe==0.12
   MySQL-python==1.2.3
   Paste==1.7.5.1
   PasteDeploy==1.5.0
   PasteScript==1.7.3
   Pygments==1.4
   Pylons==1.0
   Routes==1.12.3
   SQLAlchemy==0.7.1
   Tempita==0.5.1
   WebError==0.10.3
   WebHelpers==1.3
   WebOb==1.0.8
   WebTest==1.2.3
   amqplib==0.6.1
   anyjson==0.3.1
   carrot==0.10.7
   decorator==3.3.1
   distribute==0.6.14
   nose==1.0.0
   pytz==2011h
   redis==2.4.5
   simplejson==2.1.6
   wsgiref==0.1.2

8. If you haven't already, activate your virtual environment and move into the <foo> directory by running:

   source <foo>/bin/activate
   cd <foo>
   

   • This step ensures that you are installing dependencies ONLY within the virtual environment
9. Install the requirements by running:

   pip install -r requirements.txt

   • Note: If you run across the error "ImportError: No module named pkg_resources", then type this to fix it²:

     curl http://python-distribute.org/distribute_setup.py | python

10. Clone BuildAPI to your local machine:

    hg clone http://hg.mozilla.org/build/buildapi -r dec9d2ddedb715255db9cfce1a890d6a22bc7cd9

Setup SQlite Databases

1. Run the following commands from your <foo>/buildapi/ directory:

   cd buildapi
   sqlite3 schedulerdb
   .databases
   .exit
   sqlite statusdb
   .databases
   .exit
   

Setup and Start BuildAPI

1. To get started with the pylons project, make sure you are in the <foo>/buildapi/ directory and inside your virtualenv, then simply run:

   python setup.py develop

   • This handles grabbing pylon project dependencies. You will also need to grab and install the google python visualization library from here.
2. Now generate a config file for your project, by running:

    paster make-config buildapi config.ini
   

3. Now edit that config.ini
   • Edit config.ini to use the correct database URLs. SQLite databases take the format
     `sqlite:////absolute/path/to/database_name`.
     For example:

     sqlalchemy.scheduler_db.url = sqlite:////Users/<username>/<foo>/buildapi/schedulerdb
     sqlalchemy.status_db.url = sqlite:////Users/<username>/<foo>/buildapi/statusdb
     

   • Add the following line below "branches = b1, b2":

     branches_url = http://www.example.com

     • Note: Without a branches_url, buildapi will complain at you
     • If instead you'd like a sample of real production branches, use this:

       branches_url = http://hg.mozilla.org/build/tools/raw-file/default/buildfarm/maintenance/production-branches.json

     </il>
     • Change the port listed under [server:main] to 7000, or some other port that is available, like so:

       port = 7000

       • Note: If you run into the error "socket.error: [Errno 48] Address already in use", then try a new port. This is likely because the port you've chosen really is busy, and is not some wacky error in this case.
   • You should now be all set up now! Running the project locally can be done through:

     paster serve --daemon config.ini

     or, if you'd like to monitor output:

     paster serve -v config.ini

     Which starts running the buildapi on your local machine. To view it, open up http://localhost:7000. Or, replace 7000 with whichever port you changed to.

References

1. http://www.tlswebsolutions.com/mac-os-x-lion-setting-up-django-pip-virtualenv-and-homebrew/
2. http://stackoverflow.com/questions/7446187/no-module-named-pkg-resources