User:Ianconnolly/runner

So, following the example of Anhad, I've decided that I should document my own project here to make sure that I've got this straight in my own head and to allow other people to correct any misunderstandings I may have had.

Resources

• Original Bugzilla thread: https://bugzilla.mozilla.org/show_bug.cgi?id=712206
• Puppet code: https://github.com/catlee/puppet/compare/master...runner
• Catlee's Runner: https://github.com/catlee/runner

The Idea

"A lot of slave maintenance is done at the beginning of build and test jobs that could be done instead before even starting buildbot. things like doing clobbers, purging old directories, making sure the tools checkout is up-to-date. in the case where we have spare capacity in the pool this can be a slight win for end-to-end time since builds don't have to waste time cleaning up. in the case where we're maxed out, it's no worse than the current situation." -- catlee

So if I understand this correctly, build and test job configs are cluttered with pre-/post-flight logic that could be pulled out for performance and sanity wins. Catlee has built a framework for running a set of these jobs called Runner. He's also made a start on a runner puppet module for deploying this.

Short-term To-Dos

• Lint that puppet code!
• Template-tise the Puppet code, abstracting out the hard-coded paths
• Make Runner distributable
• Ensure Runner runs before buildbot
• Deploy Runner as is for a quick win and to identify problems.

Long-term To-Dos/Stretch goals

Catlee has a list of stuff that he tentatively wants added to Runner here: https://github.com/catlee/runner/blob/master/README.md

Stuff I Need

• Somewhere to test the puppet deployments: Catlee indicated I should ping whoever's buildduty (currently coop) - Thanks coop!
• SSH access to puppet masters - Thanks dustin!
• Help with slave CA/config problem - Thanks dustin!