ReleaseEngineering/Tupperware: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Deprecation notice)
 
(10 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{Release Engineering How To|Use Tupperware}}
<font color=red>'''This page contains obsolete content. Buildbot and BuildAPI are no longer in use.'''</font>
 
= Overview =
= Overview =
                             _._                                                   
                             _._                                                   
Line 26: Line 27:


Great for keeping an unpolluted development environment, testing patches and enabling new contributors to join the effort by lowering the barriers to entry (being one of the biggest deterrents to potential new contributors).
Great for keeping an unpolluted development environment, testing patches and enabling new contributors to join the effort by lowering the barriers to entry (being one of the biggest deterrents to potential new contributors).
Vagrant is used as a quick and easy way to provision the docker apps and make the setup truly plug n' play. The current setup only has a single Vagrantfile which launches BuildAPI and BuildBot, with their dependency apps RabbitMQ and MySQL.


= Project Requirements =
= Project Requirements =
There is only a single project dependency that you have to worry about
There is only a single project dependency that you have to worry about
* [Vagrant 1.6.3](https://www.vagrantup.com/download-archive/v1.6.3.html)
* [https://www.vagrantup.com/download-archive/v1.6.3.html Vagrant 1.6.3]
 
= Currently Included Apps =
Currently Tupperware is used for only Release Engineering apps, but that is not meant as a preclusion to avoid other apps, so by all means add some!


= Setup =
Currently included Mozilla apps:
Mana documentation:
* [http://hg.mozilla.org/build/buildapi/ BuildAPI]
* [https://mana.mozilla.org/wiki/display/IT/Redis Redis]
* [http://hg.mozilla.org/build/buildbot/ BuildBot]
* [https://mana.mozilla.org/wiki/display/IT/BuildAPI BuildAPI]
* [https://mana.mozilla.org/wiki/display/IT/RelEng+RabbitMQ]


Puppet setup:
Dependency apps currently included:
* [http://hg.mozilla.org/build/puppet/file/default/modules/selfserve_agent selfserve_agent]  
* [https://www.rabbitmq.com/ RabbitMQ]
* [http://hg.mozilla.org/build/puppet-manifests/file/default/modules/buildapi buildapi]
* [https://www.mysql.com/ MySQL] - Forked from [https://github.com/orchardup/docker-mysql/tree/76f9fa1566cfb5dad8b1ff8fe4606d5d3a2b6d72 orchardup/mysql]
* [http://hg.mozilla.org/build/puppet-manifests/file/default/modules/redis redis]


= Getting Started =
= Getting Started =
The source for the buildapi is available here: [http://hg.mozilla.org/build/buildapi buildapi source]
How to run:
* Install [https://www.vagrantup.com/download-archive/v1.6.3.html Vagrant 1.6.3]
* <pre>hg clone https://hg.mozilla.org/build/tupperware/ && cd tupperware && vagrant up</pre>
** (takes >10 minutes the first time)


Before you get started with that, you should setup your MySQL database instances.
Where to see apps:
To do that all you need to do is download, extract and load the sql dumps provided. So, create the databases first
* BuildAPI: http://127.0.0.1:8888/
<pre>
* BuildBot: http://127.0.0.1:8000/
$ mysql -u <user> -p
* RabbitMQ Management: http://127.0.0.1:15672/
<at prompt>
mysql>create database schedulerdb;
mysql>create database statusdb;
mysql>exit
</pre>


Download the sql dumps, then load them into the db with:
= Troubleshooting Tips =
<pre>
== App not visible? ==
mysql schedulerdb -u <user> -p < schedulerdb.sql
ex. Buildbot is not visible at http://127.0.0.1:8000/
mysql statusdb -u <user> -p < statusdb.sql
* If the others are up and Buildbot is not, then run "vagrant ssh", then "docker restart buildbot"
</pre>


'''NOTE''': The files unzipped can account for more than 10GB. Watch out! :)
== Vagrant Error: The guest machine entered an invalid state... ==
'''NOTE2''': The import can take a lot of time.
When running "vagrant up", I am getting a error which states, "The guest machine entered an invalid state while waiting for it to boot. Valid states are 'starting, running'. The machine is in the 'poweroff' state. Please verify everything is configured properly and try again."


Now, to get started with the pylons project, simply run:
There are a few possibilities:
<pre>
* OS X Mavericks may be causing problems with the 64-bit vagrant boxes (https://github.com/mitchellh/vagrant/issues/2157)
python setup.py install
** could be a missing kernel driver, there is a hidden virtualbox GUI window that will give more information on errors
</pre>
* Make sure you are running
In the buildapi/ directory. (And inside your virtualenv, if you're using one).
** [https://www.vagrantup.com/download-archive/v1.6.3.html Vagrant 1.6.3+]
which should handle grabbing pylon project dependencies. You will also need to grab and install the google python visualization library from [http://code.google.com/p/google-visualization-python/ here].
** [https://www.virtualbox.org/ VirtualBox 4.3.6+]
 
* Switching to a 32-bit vagrant box seems to be a temporary fix
Next you will need to generate a config file for your project, do this by running:
<pre>
paster make-config buildapi config.ini
</pre>
 
Now you will need to edit that config.ini to use the correct host(localhost should be fine), and database URLs.
Note: MySQL databases take the format `mysql://[username][:password][@hostname]/database_name`. For example:
sqlalchemy.scheduler_db.url = mysql://username:password@localhost/schedulerdb
sqlalchemy.status_db.url = mysql://username:password@localhost/statusdb
 
You should now be all set up now! Running the project locally can be done through:
<pre>
paster serve --reload --daemon config.ini
</pre>
Which starts running the buildapi on your local machine. To view it, open up http://localhost:5000.
 
= Building a simple controller =
The buildAPI is built on top of the pylons framework, which enforces a strict [http://en.wikipedia.org/wiki/Model-view-controller MVC] stack.
To add a new feature to the buildapi you will typically want to create a new controller and associate it with one or more models and views.
 
To create a simple 'hello world' controller, first use paster to create a template to work with.
<pre>
# in buildapi/
paster controller hello_world
</pre>
This automatically creates a workable template and a functional test case for your new controller.
You can now hack the newly created file under buildapi/controllers to your heart's content.
Models associated with your controller can be created under buildapi/model.
Views are created using Mako python templates. Create a template file under buildapi/templates for your new controller.
 
To associate with a model, simply add an import to your controller; for example:
<pre>
from buildapi.model.<model_name> import <functions to import>
</pre>
 
Lastly, once you're done editing your controller and have a resultset to publish, you probably want to render a page. To do this you can call render('/<template-name>.mako') from your controller body to render a view with your results. To access results from your controller, it is best to store the results in a pylons.tmpl_context object, which will make them available to your mako template.
 
===Simple code sample===
Now, to write a simple controller that prints "Hello World" using an M-V-C stack, we need three files:
* controllers/hello_world.py
* model/hello_world.py
* templates/hello_world.mako
 
Our controller lives in: controllers/hello_world.py
<pre>
import logging
 
from pylons import request, response, session, tmpl_context as c, url
from pylons.controllers.util import abort, redirect
 
from buildapi.lib.base import BaseController, render
from buildapi.model.hello_world import GetMessage
 
log = logging.getLogger(__name__)
 
class HelloController(BaseController):
 
    def index(self):
        # Return a rendered template
        c.message = GetMessage()
        return render('/hello_world.mako')
</pre>
 
Our model lives in: model/hello_world.py
<pre>
def GetMessage():
    return "Hello world!"
</pre>
 
Our view lives in: templates/hello_world.mako
Note that the tmpl_context object is available in the template.
<pre>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<title>Hello World!</title>
</head>
<body>
<h1>${c.message}</h1>
</body>
</html>
</pre>
 
To access this page, we need to set up a route to it. To do this we add to config/routing.py:
<pre>
map.connect('/hello_world', controller='hello_world', action='index')
</pre>
Now restart buildapi, and point your browser to [http://localhost:5000/hello_world http://localhost:5000/hello_world].
 
= Maintenance =
== Updating code ==
 
BuildAPI has two components: BuildAPI and Selfserve Agent.  They're deployed differently
 
=== BuildAPI ===
 
BuildAPI updates are handled like any other webapp, through pushes. See also [[ReleaseEngineering/How_To/Update_BuildAPI]]
 
The steps are roughly as follows, from the base directory of buildapi:
* Push new version of BuildAPI to HG
** Edit setup.py to show the updated version (ie version='0.3.0' becomes version='0.3.1')
* Comment out tag_build in setup.cfg
** This appends tarball name, among other things
* Build a new sdist tarball of BuildAPI, using normal python techniques, and upload it to the puppetagain python repository at releng-puppet2.srv.releng.scl3.mozilla.com (/data/python/packages on the distinguished puppetmaster)
** Tarballs can be seen here: http://puppetagain.pub.build.mozilla.org/data/python/packages/
  python setup.py sdist
  scp dist/buildapi-0.3.1.tar.gz releng-puppet2.srv.releng.scl3.mozilla.com:
  ssh releng-puppet2.srv.releng.scl3.mozilla.com
  sudo buildapi-0.3.1.tar.gz /data/python/packages
  cd /data/python/packages
  sudo chown puppetsync:puppetsync buildapi-0.3.1.tar.gz
  sudo chmod 664 buildapi-0.3.1.tar.gz
* Login to relengwebadm.private.scl3.mozilla.com and become root.  Go to /data/releng-stage/src/buildapi (or /releng/ for production) and have a look at README.txt. Where 0.3.1 is the version you want to install, it boils down to:
  ./update 0.3.1
 
=== Selfserve Agent ===
 
Deploying new functionality may require deploying a new version of the <code>scripts/selfserve_agent.py</code> file on build masters. This process is handled by puppet, but orthogonal to buildapi itself. The steps are roughly:
 
* bump the version of the buildapi repository in "<code>setup.py</code>" and commit (following semantic versioning guidelines, please)
* add a new tag to the buildapi repository for that version, commit & push
* create a python sdist tarball of that version:
** make sure you have a clean checkout
** locally modify (but do not commit) <code>setup.cfg</code> and comment out "<code>tag_build</code>"
** run "python setup.py "<code>sdist</code>"
** updload the file from "<code>dist/</code>"
* [[ReleaseEngineering/How_To/Add_a_Python_Package_to_PuppetAgain|upload to the internal puppetagain python]] repo.
* update the manifests for the new version in [http://hg.mozilla.org/build/puppet/file/default/modules/selfserve_agent/manifests/install.pp puppet]
* wait for puppet to deploy the new agent.
 
== Adding branches ==
buildapi and self-serve pull from [http://hg.mozilla.org/build/tools/raw-file/default/buildfarm/maintenance/production-branches.json] to determine which branches are active and caches the result for a few minutes.
 
== Kicking ==
If things are broken, figure out if it's
* self-serve requests not completing (check the self-serve agents and rabbit connections)
* buildapi HTTP requests not working (check buildapi - [[ReleaseEngineering/How_To/Restart_BuildAPI]])
* builds-4hr.js not being updated (check the [[ReleaseEngineering/BuildAPI#report_crontask|report crontask]] report crontask)
 
=== report crontask ===
 
There's a crontask on [https://mana.mozilla.org/wiki/display/websites/Releng+Cluster relengwebadm] that runs every minute to generate the builds-4hr.js file.  If this crontask gets "hung", it will prevent updates from occuring.  This can happen due to DB connection issues, or bad data in the DB.  In these cases, killing the hung crontask is the appropriate fix.
 
However, the crontask takes about 15 minutes to run on a cold cache.  So if its cache (memcached) has gone cold, then killing it before 15 minutes have elapsed is only delaying the failure.
 
You can also read [https://bugzilla.mozilla.org/show_bug.cgi?id=1005342#c6 this comment].
 
= Staging =
You can visit https://secure-pub-build.allizom.org/buildapi
 
From Dustin:
Buildapi itself doesn't write anything but jobrequests.  For the staging instance, those go to a different DB than for production.  Like the production version, it reads from the production status and scheduler db's.
 
The bit that it's missing is a running selfserve-agent.  If you trigger some action on the staging instance, it pushes a message to the staging rabbitmq virtualhost and waits for a response, but since there's no agent that response never comes.
 
= Development environment =
Running your own instance of BuildAPI is relatively simple. You should:
* Create a virtualenv ([[ReleaseEngineering/BuildAPI/Setup_Local_Virtualenv_for_BuildAPI|details]])
* Install buildapi with its setup.py script
* Create a config. You may base your config on the production one, with at least the following changes:
** Change DEFAULT.email_to to your own email address
** Change server:main.port to 0.0.0.0
** Change server:main.port to something unused
** Comment out app:main.carrot*
** Set app:main.buildapi.cache to nothing
 
= Troubleshooting Tips =
== Host ==
NOTE: buildapi01 is largely unused now and the hosts are managed by webops


==No MySQLdb==
==ProTips==
If after installing you run:
Nothing yet :)
<pre>
paster serve --reload --daemon config.ini
</pre>
and it does not start the server, check paster.log and if you see "ImportError: No module named MySQLdb" then you need to easy_install MySQL-python into the site-packages of the python it's looking for MySQLdb in.

Latest revision as of 07:31, 8 October 2018

This page contains obsolete content. Buildbot and BuildAPI are no longer in use.

Overview

                           _._                                                  
                      _.-``__ -._                                             
                 _.-``.-```  ```-. -._           
             .-`` .-``            ``._  -._                                   
            (    '                    `,      `.     
            |`-._ `-._               _.'  _.--'     
             `-._`-._ `-._       _.-' _.-'_.         
              |  `-._`-._ `-._.-'_.-'_.-'  ,                                    
               \     `-._`-.__.-'_.-'     /                                   
                `-.      `-.__.-'      .-'                                     
                   ``-.            .-                                 
                       ``-.    .-            
                           ``
 _______                                                 
|__   __|                                                
   | |_   _ _ __  _ __   ___ _ ____      ____ _ _ __ ___ 
   | | | | | '_ \| '_ \ / _ \ '__\ \ /\ / / _` | '__/ _ \
   | | |_| | |_) | |_) |  __/ |   \ V  V / (_| | | |  __/
   |_|\__,_| .__/| .__/ \___|_|    \_/\_/ \__,_|_|  \___|
           | |   | |                                     
           |_|   |_|

Mozilla apps in Docker. Portable, Reusable, and Containerized.

Great for keeping an unpolluted development environment, testing patches and enabling new contributors to join the effort by lowering the barriers to entry (being one of the biggest deterrents to potential new contributors).

Vagrant is used as a quick and easy way to provision the docker apps and make the setup truly plug n' play. The current setup only has a single Vagrantfile which launches BuildAPI and BuildBot, with their dependency apps RabbitMQ and MySQL.

Project Requirements

There is only a single project dependency that you have to worry about

Currently Included Apps

Currently Tupperware is used for only Release Engineering apps, but that is not meant as a preclusion to avoid other apps, so by all means add some!

Currently included Mozilla apps:

Dependency apps currently included:

Getting Started

How to run:

  • Install Vagrant 1.6.3
  • hg clone https://hg.mozilla.org/build/tupperware/ && cd tupperware && vagrant up
    • (takes >10 minutes the first time)

Where to see apps:

Troubleshooting Tips

App not visible?

ex. Buildbot is not visible at http://127.0.0.1:8000/

  • If the others are up and Buildbot is not, then run "vagrant ssh", then "docker restart buildbot"

Vagrant Error: The guest machine entered an invalid state...

When running "vagrant up", I am getting a error which states, "The guest machine entered an invalid state while waiting for it to boot. Valid states are 'starting, running'. The machine is in the 'poweroff' state. Please verify everything is configured properly and try again."

There are a few possibilities:

ProTips

Nothing yet :)

Retrieved from "https://wiki.mozilla.org/index.php?title=ReleaseEngineering/Tupperware&oldid=1202049"