Build/Database

Design Considerations

• Code must be suitable for submission to upstream
• We should store as much data about the builds in the database as possible
  • The current exception to this is build logs, which are _not_ stored in the database.
• All active buildbot masters should report into the same database.

Source code

The source code currently resides at http://hg.mozilla.org/users/catlee_mozilla.com/bb_db.

model.py defines all the SQL tables as well as the mapping to python classes.

status.py implements the buildbot status plugin. It's added to the BuildmasterConfig like this:

   import bbdb.status
   reload(bbdb.status)
   c['status'].append(bbdb.status.DBStatus(dburl))

dburl is a string describing how to connect to the database [1]

Schema

The SQL schema is close approximation of buildbot's internal data structures into SQL.

Querying the database

reporter.py contains some examples of querying the database. The basic process is:

• Call model.connect with the appropriate DB URL. A session-maker object is returned. Use this session-maker object to create a session.
• Use the session to query the various tables.
• Close the session when done

   import model
   # Connect to database called 'buildbot' on localhost, using
   # 'buildbot', 'passw0rd' as the username, password
   session_maker = model.connect("mysql://buildbot:passw0rd@localhost/buildbot")
   session = session_maker()
   try:
       builds = session.query(model.Build).filter_by(result=0)
       for build in builds:
           # Do something with build
           print "Build", build.buildnumber, "started at", build.starttime
   finally:
       # Close the session.  This cleans up cached objects, and frees the
       # connection to the database so it can be used by another session
       session.close()

The SQLAlchemy session is used to hold python representations of SQL rows in memory. It's kind of like a cache for these mapped objects.

Alternatively, raw SQL can also be run directly with the session:

   result = session.execute("select * from builds where result=0 limit 10")

Hooks

The DBStatus plugin allows you to pass in a list of subscribers that get regular buildbot status notifications that are augmented with database information.

Recommended Reading

• SQLAlchemy docs: http://www.sqlalchemy.org/docs, especially the section on querying