Overview

It's not uncommon to need to synchronize some data into puppet from another authoritative source. That may be a list of users, SSH keys, machine characteristics, etc.

PuppetAgain supports these through the extsync system. The idea is that data is periodically pulled from external sources by the distinguished puppetmaster, processed as needed, and then written into hiera data files. Those files are synchronize to other masters in the organization along with the hiera secrets files. Then the data is available using normal hiera functions.

Since each organization will have different needs for synchronization, the system is highly configurable. First, each kind of synchronization is a different "extsync" with its own name. For example, `ldap_users` or `masters_json`. Each extsync also takes arbitrary named parameters to customize it to the needs of the organization. These are all customized in the organization configuration. For example:

   $puppetmaster_extsyncs = {
       masters_json => {
           masters_json_url => "..",
       },
   }

This will activate only those two extsyncs in this organization, passing them the appropriate configuration parameters. The data that the ldap_users extsync produces will be placed in /etc/hiera/ldap_users.eyaml.

The following describe the existing extsyncs:

slavealloc

This synchronizes slave data from slavealloc.

Parameters:

slavealloc_api_url

the URL from which to pull the data (`/slaves` is appended to this URL)

Result:

A hiera key named 'slavealloc_slaves' containing a hash by unqualified slavename. Each value in the hash contains the slavealloc-supplied information about the slave, including its environment.

Functions:

The `slavealloc_environment($fqdn)` function fetches the environment for a buildslave.

fake_slavealloc

This is like slavealloc, but does no actual syncing; add data to `/etc/hiera/fake_slavealloc.yml`.

moco_ldap

This sync pulls SSH keys and group membership from an LDAP directory. It works great with Mozilla's LDAP directory, and may also work with others.

It uses the following LDAP attributes:

* sshPublicKey (in the posixAccount class)
* uid (in the posixAccount class)
* member (for groups)

Parameters:

moco_ldap_uri

LDAP URL from which to pull the data

moco_ldap_root

Root DN at which to begin searches

moco_ldap_dn

DN of the bind user to use for authentication

moco_ldap_pass

password for the bind user

users_in_groups

groups to synchronize. This is a hash, where the values are lists of groups to combine, and the keys give a name to the resulting combination.

For example:

   users_in_groups =>
       editors => ['operators', 'vouched_contributors']

would combine the operators and vouched_contributors LDAP groups into a list of userids named editors.

Result:

hiera('ssh-keys')

a hash of SSH keys by userid, with values being a list of SSH keys

hiera($result_name)

a list of userids accumulated from LDAP groups; in the example above you would use hiera('editors') to get the userids of all operators and vouched_contributors

The former is automatically invoked in ssh::keys, combined with the $extra_user_ssh_keys config, so it is unlikely you will use it directly.

The latter is best used with a backup, e.g.,

   $admin_users = hiera('ldap_admin_users',
       # backup to ensure access in case the sync fails:
       ['arr', 'dmitchell', 'jwatkins'])

Caution

There is a "chicken and egg" problem when setting up a new organization with this feature: if $admin_users is based on LDAP groups and keys, those keys will not be synchronized yet. To fix this, use the backup trick above with a small, static $admin_users with keys in $extra_user_ssh_keys, then set up the extsync, and once it is synchronized, switch to using it.