ReleaseEngineering/Buildbot Best Practices: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(deleting obsolete page)
 
(8 intermediate revisions by 5 users not shown)
Line 1: Line 1:
'''<big>WIP</big>'''


We've gotten to the point of having many different masters, and more complex masters. It's becoming more and more important to have consistent setup in terms of where configuration files are stored, where supporting code is stored, and other such details. This document is a rulebook and guideline for how to maintain these machines and instances.
== Repositories ==
* [http://hg.mozilla.org/build/buildbot-configs/ buildbot-configs] - This repository contains Buildbot master.cfg and mozconfig files for most of our Buildbot instances.
* [http://hg.mozilla.org/build/buildbotcustom/ buildbotcustom] - This repository contains custom steps, factories, and other Mozilla specific Buildbot code required by our Buildbot masters.
* [http://hg.mozilla.org/build/buildbot/ buildbot] - This repository is an import of upstream Buildbot code, plus some patches that we require.
== Check-in Policies ==
* We track configuration updates to our masters on [[ReleaseEngineering:BuildbotMasterChanges | BuildbotMasterChanges]]. Any change to a production Buildbot Master should be tracked on this page.
* Patches should not be checked in until you are ready to update the master with them. This helps to avoid situations where an urgent fix needs to go in, and a random unrelated patch ends up getting enabled at the same time.
* Production masters should never contain local changes. Even if you are just testing something you should check it in and pull it rather than making the change locally. Having temporary changes in the version control history is useful when debugging things later.
* If you have a patch that affects multiple masters you should update all of the masters when you land your patch.
** This is less strict for staging masters. Eg, you shouldn't update a staging master if someone is currently using it.
* There are no staging repositories for buildbot or buildbotcustom. Because of this you should never check in code to either of these repositories until it is properly reviewed and about to be deployed in production.
** If you want to keep these repositories version controlled while doing development/testing you should create a user repository for them.
== How to land a patch ==
Below is one way to land a patch for a Buildbot master. Your technique may vary, but this will ensure there are no other patches sneaking in with yours, and no local changes. Before doing any of the below, land your patches into the repositories.
* Look for local changes on the master.
cd /tools/buildbotcustom/buildbotcustom
hg diff
cd /builds/buildbot/configs
hg diff
* Check to make sure only your patches are going to be enabled. If more than just your changeset show up in 'hg in' you can use 'hg in -p' to look at a diff of all of the patches that are going to be pulled in. It's not uncommon for other changesets to be there in buildbot-configs, since it holds configs for so many masters. Use your common sense here.
cd /tools/buildbotcustom/buildbotcustom
hg in
# If there are more than just your changeset use 'hg in -p' to look at the diffs
hg in -p
# Same thing for configs
cd /builds/buildbot/configs
hg in
hg in -p
* Now, assuming there are no interfering patches, pull the changes for real
cd /tools/buildbotcustom/buildbotcustom
hg pull && hg up
cd /builds/buildbot/configs
hg pull && hg up
* Run checkconfig to make sure the reconfig/restart will succeed
cd /builds/buildbot/$master
buildbot checkconfig
* If everything looks OK, reconfig or restart the master
buildbot reconfig `pwd`
# OR
# You may need to run stop multiple times before Buildbot recognizes that the process is dead
# You can also check 'ps auxwww | grep buildbot' for it.
buildbot stop `pwd`
buildbot start `pwd`
* Watch the Buildbot Waterfall for problems.

Latest revision as of 21:43, 19 November 2018

Retrieved from "https://wiki.mozilla.org/index.php?title=ReleaseEngineering/Buildbot_Best_Practices&oldid=1204119"