ReleaseEngineering/PuppetAgain

History

Releng once used a puppet infrastructure based on Puppet-0.24.8, and manifests at http://hg.mozilla.org/build/puppet-manifests/. This had a few weaknesses:

• lots of assumptions and fragile dependencies based on bugs in 0.24.8
• very few modules - mostly manifest files, organized per slave type, rather than per service/purpose
• many references to external files which are not as available as the repo itself
• puppet manifests assume some manual ref-image steps; external exact reproduction is extremely difficult

Dustin started work on a new puppet deployment - chronicled at User:Djmitche/New Releng Puppet Infrastructure. That's this puppet.

System Description

Quick info:

• Manifest repository: http://hg.mozilla.org/build/puppet

Puppetmasters

Releng puppet masters are managed by IT (in fact, managed by IT's puppet infrastructure, which can lead to some confusion). There will be as many puppet masters as required, attempting to minimize the need communication across WAN links. The puppet masters do not permit root logins by non-sysadmins, but automatically update from the manifest repository using a crontask. As described below, masters also allow user logins for a limited set of people, who can set up puppet environments.

See ReleaseEngineering/PuppetAgain/Puppetmasters for more.

Puppet Versions

The releng puppet infrastructure will be using the same puppet versions as the rest of Mozilla. This generally tracks the latest puppet release. As IT upgrades, the masters will be upgraded; releng can then upgrade the clients using puppet itself.

Base Images

The base images for this infrastructure are barely-modified OS installs. They have just enough installed that they can connect to a puppet server, get certificates, and puppetize on boot.

See ReleaseEngineering/PuppetAgain/Base Images for more.

Manifests & Modules

The Puppet manifests themselves are documented here. Any new

Settings

The file manifests/settings.pp specifies a global settings for the puppet manifests. It specifies releng values for settings that other sites or users may want to change. This file is currently checked in and used as-is, but if it becomes necessary it can be changed to work like secrets.pp.in. The settings are:

• puppet_notif_email - email address to which errors in running puppet will be sent.

Secrets

The file manifests/secrets.pp.in specifies a template for per-site settings; this must be copied to manfiests/secrets.pp and filled in with appropriate secret values for the site. The current secrets are:

• root_pw_hash - linux md5 password hash for the root password
• builder_username - username for build and test slaves (cltbld for releng)
• builder_pw_hash - linux md5 password hash for the builder user

Stages

Stages need to be defined globally in Puppet manifests, and this is done in manifests/stages.pp. The following stages are available, aside from 'main', the default stage.

• packagesetup - This stage should handle any preliminaries required for package installations, so that subsequent package installations do not need to require them explicitly.

Nodes

manifests/nodes.pp defines all of the nodes the puppet masters recognize. Note that all nodes are defined for all masters.

In anticipation of using an external node classifier (ENC), node definitions should only include classes - do not define any resources within nodes. In general, the included classes should be in the toplevel module.

If the need arises to set host-specific values for classes, we will need to carefully consider whether to set variables in the node scope, or to use parameterized classes, based on the capabilities of an ENC.

Modules

All substantial configuration is done with puppet modules, in the modules directory. Each should have its own page describing both how to use the module, and how the module works.

Infrastructure

These modules are part of the puppet system itself, and provide support to other modules as needed

• ReleaseEngineering/Puppet/Modules/packages - install packages generically
• ReleaseEngineering/Puppet/Modules/puppet - install, upgrade, and run puppet (including custom facts, etc.)
• ReleaseEngineering/Puppet/Modules/toplevel - top-level classes for node types, included from node definitions
• ReleaseEngineering/Puppet/Modules/users - user account management
• ReleaseEngineering/Puppet/Modules/yum-repos - set up yum repositories