From MozillaWiki
Jump to: navigation, search


It's not uncommon to need to synchronize some data into puppet from another authoritative source. That may be a list of users, SSH keys, machine characteristics, etc.

PuppetAgain supports these through the extsync system. The idea is that data is periodically pulled from external sources by the distinguished puppetmaster, processed as needed, and then written into hiera data files. Those files are synchronize to other masters in the organization along with the hiera secrets files. Then the data is available using normal hiera functions.

Since each organization will have different needs for synchronization, the system is highly configurable. First, each kind of synchronization is a different "extsync" with its own name. For example, `ldap_users` or `masters_json`. Each extsync also takes arbitrary named parameters to customize it to the needs of the organization. These are all customized in the organization configuration. For example:

   $puppetmaster_extsyncs = {
       masters_json => {
           masters_json_url => "..",

This will activate only those two extsyncs in this organization, passing them the appropriate configuration parameters. The data that the ldap_users extsync produces will be placed in /etc/hiera/ldap_users.eyaml.

The following describe the existing extsyncs:


This synchronizes slave data from slavealloc.


the URL from which to pull the data (`/slaves` is appended to this URL)


A hiera key named 'slavealloc_slaves' containing a hash by unqualified slavename. Each value in the hash contains the slavealloc-supplied information about the slave, including its environment.


The `slavealloc_environment($fqdn)` function fetches the environment for a buildslave.


This is like slavealloc, but does no actual syncing; add data to `/etc/hiera/fake_slavealloc.yml`.


This sync pulls SSH keys and group membership from an LDAP directory. It works great with Mozilla's LDAP directory, and may also work with others.

It uses the following LDAP attributes:

* sshPublicKey (in the posixAccount class)
* uid (in the posixAccount class)
* member (for groups)


LDAP URL from which to pull the data
Root DN at which to begin searches
DN of the bind user to use for authentication
password for the bind user
groups to synchronize. This is a hash, where the values are lists of groups to combine, and the keys give a name to the resulting combination.

For example:

   users_in_groups =>
       editors => ['operators', 'vouched_contributors']

would combine the operators and vouched_contributors LDAP groups into a list of userids named editors.


a hash of SSH keys by userid, with values being a list of SSH keys
a list of userids accumulated from LDAP groups; in the example above you would use hiera('editors') to get the userids of all operators and vouched_contributors

The former is automatically invoked in ssh::keys, combined with the $extra_user_ssh_keys config, so it is unlikely you will use it directly.

The latter is best used with a backup, e.g.,

   $admin_users = hiera('ldap_admin_users',
       # backup to ensure access in case the sync fails:
       ['arr', 'dmitchell', 'jwatkins'])


There is a "chicken and egg" problem when setting up a new organization with this feature: if $admin_users is based on LDAP groups and keys, those keys will not be synchronized yet. To fix this, use the backup trick above with a small, static $admin_users with keys in $extra_user_ssh_keys, then set up the extsync, and once it is synchronized, switch to using it.