User:Catlee/Podding

Splitting our buildbot master into "pods" might look something like this:

• A controller process
  • Handles incoming changes (from hg)
  • Creates initial optimized build, debug build entries in the Queue
  • Monitors build progress
    • Creates subsequent unit test, talos test entries in the Queue once builds are available
    • Re-queues builds on certain types of failure

• Buildbot masters
  • No hg polling or sendchanges are done any more
  • Only source of requests is by looking in the Queue. When a request that the master can satisfy is found, it can mark that request as taken.

The Queue

A database with the following tables:

• requests
  • id
  • branch
  • type (optimized build, debug build, etc.)
  • revision
  • file_url (required for talos and packaged unittests. combine this with revision somehow?)
  • depends_on (refers to another request, which must succeed before this request can be processed)