From MozillaWiki
Jump to: navigation, search

This page is an exploration of some ideas of how to clean up our build infrastructure, in particular the set of commands that get run to produce a build, or run a set of tests.


  • module and class dependencies in buildbotcustom has become very complicated
  • testing changes to classes in buildbotcustom isn't feasible outside of a buildbot environment
  • buildbotcustom is opaque to outside contributors
  • Making minor changes that affect only one platform or branch is very complicated in the current setup, requiring a lot of testing across all potentially affected platforms and branches.

Idea 1

Have a new repository, let's call it build/build-scripts. This repository is laid out similarly to how our mozconfigs are organized:


Each directory would contain a config file which would specify the steps to run, as well as things like the mozconfig and helper scripts.

To run a build, the slave would checkout a harness script, along with the build scripts. The harness is then pointed at the config file to execute the required steps.

The "build configuration", which contains things like which revision to build, gets written somewhere (like /tmp?). An environment variable is set to indicate this location ($BUILD_CONFIG). It could be nice to have this configuration written out in multiple formats, e.g. suitable for source'ing in a bash script, suitable for importing in a python script, and config.json suitable for other purposes.

The harness will also create a temporary working directory where scripts are free to save data for use later in the build, e.g. recording changesets, hashes, or package urls.



steps = cleanup, checkout, mozconfig, build, package, upload

MOZ_OBJDIR = obj-firefox
PATH = /usr/local/bin:/usr/bin:/bin

timeout = 1200

timeout = 1200

command = cp %(CONFIG_DIR)s/mozconfig mozilla-central/.mozconfig

command = make -C mozilla-central -f

command = make -C mozilla-central/obj-firefox package



set -e
test -f ${BUILD_CONFIG}.sh && . ${BUILD_CONFIG}.sh

if [ -n $REV ]; then

if [ -d mozilla-central ]; then
    cd mozilla-central
    hg pull $HGURL
    hg update -C -r $REV
    hg clone $HGURL mozilla-central

steps with no 'command' set expect to find a script with the same name of the step in the configuration directory.

To run this build, you would run: -f mozilla-central/linux/depend/config.ini -C build/mozilla-central/depend HGURL= REV=123456

Open Questions

  • We'd lose per-step logging and status as far as buildbot is concerned. Does this matter?
  • How to share common scripts and configs?