ReleaseEngineering/Puppet/Usage: Difference between revisions

Jump to navigation Jump to search

ReleaseEngineering/Puppet/Usage (view source)

Revision as of 23:21, 17 February 2011

2,647 bytes removed ,  17 February 2011
no edit summary
No edit summary
Line 4: Line 4:


== Definitions ==
== Definitions ==
* Type - Puppet documentation talks a lot about this. Each different "type" deals with a different aspect of the system. For example, the "user" type can do most things related to user management (passwords, UID/GID, homedirs, shells, etc). The 'package' type deals with package management (eg, apt, rpm, fink, etc). And so on.
* Type - Puppet documentation talks a lot about this. Each different "type" deals with a different aspect of the system. For example, the "user" type can do most things related to user management (passwords, UID/GID, homedirs, shells, etc). The 'package' type deals with package management (eg, apt, rpm, fink, etc). And so on.


Line 17: Line 18:


=== The Slave-Master Link ===
=== The Slave-Master Link ===
You can find to which puppet master a slave connects to by checking this files' contents:
You can find to which puppet master a slave connects to by checking this file's contents:
  # for linux testers (fedora)
  # for linux testers (fedora)
  ~cltbld/.config/autostart/gnome-terminal.desktop
  ~cltbld/.config/autostart/gnome-terminal.desktop
Line 35: Line 36:
* Basic includes
* Basic includes
* Packages that make changes
* Packages that make changes
* Modules
We are pushing toward organizing everything into modules, although this is not a particularly rapid process at the moment.  Talk to Dustin.


=== Site Files & Basic Includes ===
=== Site Files & Basic Includes ===
Line 63: Line 66:
* The classes [http://hg.mozilla.org/build/puppet-manifests/file/tip/classes/buildslave.pp buildslave.pp] and [http://hg.mozilla.org/build/puppet-manifests/file/tip/classes/staging-buildslave.pp staging-buildslave.pp] include most of the packages (devtools, nagios, mercurial, buildbot, extras, etc) we want.
* The classes [http://hg.mozilla.org/build/puppet-manifests/file/tip/classes/buildslave.pp buildslave.pp] and [http://hg.mozilla.org/build/puppet-manifests/file/tip/classes/staging-buildslave.pp staging-buildslave.pp] include most of the packages (devtools, nagios, mercurial, buildbot, extras, etc) we want.
* The packages can have different sections or "Types" that can be "exec", "user", "package", "file", "service"
* The packages can have different sections or "Types" that can be "exec", "user", "package", "file", "service"
=== Modules ===
Going forward, puppet functionality should be encapsulated into modules.  Modules include the relevant manifests, as well as files, templates, and (with some minor changes to our puppet client configs) even custom facts or types!
Modules should be generic in their purpose, and well-encapsulated.  They should not be specific to one operating system or distro by design, although it's OK to omit implementations we do not need (for example, it's OK for a module providing resources only used by build slaves to error out if it's used on a Fedora system - if and when we start building on Fedora, we'll need to extend the implementation).
A module should be self-contained and have a well-documented and commented interface.  If it depends on any other modules, that should also be highlighted in the comments.


== Puppet Files ==
== Puppet Files ==
Line 74: Line 84:
* $slaveType is the "type" of node of the slave is: 'build', 'test', 'stage', 'master', etc.
* $slaveType is the "type" of node of the slave is: 'build', 'test', 'stage', 'master', etc.


Below '$type', are all of the files served by Puppet. They are organized according to where they'll end up on the slave. For example, if /etc/X11/fonts.conf is to be synced to the slave, it should live in:
Below '$type', are all of the files served by Puppet. They are organized according to where they'll end up on the slave. For example, if ''/usr/lib/libsomethinghuge.so'' is to be synced to the slave, it should live in ''usr/lib/libsomethinghuge.so''. Note that as much as possible, text files should not be kept in puppet-files -- use a module and its ''files/'' subdirectory instead.
  etc/X11/fonts.conf


There are two special directories for each level/os/hardwaremodel/type combination, too:
There are two special directories for each level/os/hardwaremodel/type combination, too:
* local -- This directory contains files which should NOT be synced between staging <-> production or between different locations. Files such as the Puppet configs which have different contents depending on location and support level live here.
* local -- This directory contains files which should NOT be synced between staging <-> production or between different locations. Files such as the Puppet configs which have different contents depending on location and support level live here.  Try not to use this.
* DMGs (Mac) / RPMs (Fedora/CentOS) -- These directories contain platform specific packages which Puppet installs.
* DMGs (Mac) / RPMs (Fedora/CentOS) -- These directories contain platform specific packages which Puppet installs.


== Common Use Cases ==
== Common Use Cases ==
=== Updating a password ===
* [[ReleaseEngineering/How To/Reset the cltbld Password with Puppet]]
Passwords are stored in a hashed format alongside other user information. We do not put the hashes in a public location for hopefully obvious reasons - please make sure you don't do this by accident.
* [[https://wiki.mozilla.org/ReleaseEngineering/How_To/Install_a_Package_with_Puppet]]
 
Let's say you want to update cltbld's password. First, you need to generate the new hash. You can do that by running the following:
makepasswd --clearfrom=- --crypt-md5
# now type the password and hit ^D a couple times
Now, copy and paste that password into /etc/puppet/manifests/build/cltbld.pp as the 'password' for the cltbld user. Do this on all active puppet masters.
 
=== Installing a Package ===
After pushing file deployment over NFS to its limit we replaced it with native package formats for software deployment. This switch was made around June, 2010.
 
==== RPM (CentOS, Fedora) ====
We use a combination of 3rd party and in-house RPMs to deploy to our Linux machines. On the manifest side we use the 'rpm' package provider wrapped in a custom type to ensure installation.
 
To build or upgrade a homegrown RPM, see [[ReleaseEngineering/How To/Create a new RPM]].
 
===== Manifests =====
<p>The manifests are pretty simple once have an RPM. We use a wrapper type called 'install_rpm' to perform installation. You can use it as follows:</p>
install_rpm {
    "gcc433-4.3.3-0moz1":
        creates => "/tools/gcc-4.3.3/installed/bin/gcc",
        pkgname => "gcc433";
}
 
The name needs to match the package name + version. Note that RPM requires a 'vendor' version, which is where the 0moz1 comes from. Creates needs to be a file that the package creates, preferably the last one to get installed.
 
==== DMG+pkg (Mac) ====
<p>Mac machines use pkg installers wrapped in a DMG file as a package format. On the manifest side, we use the 'pkgdmg' package provider wrapped in a custom type to deploy them. For things which are distributed in a DMG+pkg (such as Xcode) you can skip down to the manifests.</p>
 
<p>When an upstream DMG file is not available it needs to be created by hand. To do this, we do a manual installation once and then using a script to create the DMG+pkg. Here's an example, which creates a Python 2.5.2 DMG:</p>
# The installation
tar jxvf Python-2.5.2.tar.bz2
cd Python-2.5.2
./configure --prefix=/tools/python-2.5.2
make
make install
cd ..
# DMG creation
hg clone http://hg.mozilla.org/build/puppet-manifests
./puppet-manifests/create-dmg.sh /tools/python-2.5.2 python-2.5.2 python /tools
 
<p>The first argument to create-dmg.sh is the directory to package, which will include the directory itself. The second argument is the name to use on the DMG/pkg filenames. The third is the string to use in the package identifier, it must be alphanumeric only. Lastly, the directory to install the package to.</p>
 
<p>On the manifests side of things a simple use of the install_dmg type will ensure a package gets installed:</p>
install_dmg {
    "python-2.5.2.dmg":
        creates => "/tools/python-2.5.2/share",
}
 
<p>The argument to "creates" should be one of the last files that will be created by the package. Internally, install_dmg checks for this file and mark the package as installed if it exists, skipping installation.</p>
 
<p>If you intend to use a package on multiple platforms always ensure to test on them before rolling out any manifest changes. When in doubt, create a package on each target platform.</p>


== Testing ==
== Testing ==
<p>Before you test on the Puppet server it's good to run the 'test-manifests.sh' scripts locally. This script will test the syntax of the manifest files and catch very basic issues. It will not catch any issues with run-time code such as Exec's.</p>
Before you test on the Puppet server it's good to run the 'test-manifests.sh' scripts locally. This script will test the syntax of the manifest files and catch very basic issues. It will not catch any issues with run-time code such as Exec's. This should really be a Makefile - {{bug|635067}}


<p>Testing of updates is done with staging-puppet.build.mozilla.org and staging slaves. You should book staging-puppet as well as an slaves you intend to test on before making any changes to the manifests on the Puppet server. All Puppet server work is done as the root user.</p>
Staging of updates is done with ''staging-puppet.build.mozilla.org'' and staging slaves. You should book staging-puppet as well as an slaves you intend to test on before making any changes to the manifests on the Puppet server. All Puppet server work is done as the root user.


=== Setting up the server ===
=== Setting up the server ===
If you've never used the Puppet server before you'll want to start a clone of the manifests for yourself. You can clone the main manifests repo or your own user repo to a directory under /etc/puppet. Once you have your clone, two edits are necessary:
If you've never used the Puppet server before you'll want to start a clone of the manifests for yourself. You can clone the main manifests repo or your own user repo to a directory under ''/etc/puppet''. Once you have your clone, two edits are necessary:
 
* Copy the password hash into your clone's build/cltbld.pp. This can be done with the following command, run from the root of your clone:
* Copy the password hash into your clone's build/cltbld.pp. This can be done with the following command, run from the root of your clone:
  hg -R /etc/puppet/manifests.real diff /etc/puppet/manifests.real/build/cltbld.pp | patch -p1
  hg -R /etc/puppet/manifests.real diff /etc/puppet/manifests.real/build/cltbld.pp | patch -p1
or more easily
or more easily
  patch -p1 < /etc/puppet/password
  patch -p1 < /etc/puppet/password
* Comment out all of the "node" entries in staging.pp, except for those which you have booked.
* Copy ''staging.pp'' to ''site.pp'' and comment out all of the "node" entries except for those which you have booked.


<p>If you have a patch to apply to the repository now is the time to do it.</p>
It's easiest to use the ''mq'' extension to make these changes in a patch on your queue.  Then, when you want to change revisions, just pop the patch, use 'hg pull -u', and re-push your patch.


<p>Finally, if your changes involve edits to any files served by Puppet, apply those changes in the appropriate places under /N/staging.</p>
If you have a patch to apply to the repository now is the time to do it.


<p>Staging environments do not have the site.pp manifest. When testing in a staging environment, symlink site.pp to staging.pp with the following command:</p>
Finally, if your changes involve edits to any files served by Puppet, apply those changes in the appropriate places under /N/staging. It's usually easiest to keep a text file tracking these changes - then you can post the contents of that file to the bug for review, so that it's clear to reviewers what changes are being made here.  Because puppet-files are unversioned, try to minimize the amount of change you must make here.
  ln -s staging.pp site.pp


<p>Once all of that is done you can swap your manifests in by adjusting the symlink on /etc/puppet/manifests. If you've added new files or changed staging-fileserver.conf you'll need to restart the Puppetmaster process with:</p>
Once all of that is done you can swap your manifests in with ''/etc/puppet/set-manifests.sh YOURNAME''.  Omit the name to reset them to the default ("real") manifests. If you've added new files or changed staging-fileserver.conf you'll need to restart the Puppetmaster process with:</p>
  service puppetmaster restart
  service puppetmaster restart
although note that the daemon will pick up the changes after some short delay if you do not restart.


Now, you're ready to test.
Now, you're ready to test.
Line 179: Line 139:
Once you're finished testing the manifests symlink needs to be re-adjusted with:
Once you're finished testing the manifests symlink needs to be re-adjusted with:
  cd /etc/puppet
  cd /etc/puppet
  rm manifests
  ./set-manifests.sh
ln -s manifests.real manifests


== Moving file updates to production ==
== Moving file updates to production ==
Line 211: Line 170:


Be sure to do this on all Puppet masters.
Be sure to do this on all Puppet masters.
== Moving slaves between staging/production ==
If you need to move slaves between staging and production, you'll need to delete the existing ssl certs on the slave so it can properly sync with the new puppet master. These certs can be found under <code>/etc/puppet/ssl</code> on mac or <tt>/var/lib/puppet/ssl</tt> on linux.
== Documentation/Links ==
Puppet has reasonably complete documentation, although navigating it can be a challenge.
* [http://docs.puppetlabs.com/references/stable/type.html Type Reference]
* [http://docs.puppetlabs.com/references/latest/metaparameter.html Metaparameter (parameters which apply to all types) Reference]
* [http://docs.puppetlabs.com/references/latest/configuration.html Puppet Command-line and Configuration Reference]
== Troubleshooting ==
Puppet sometimes gets itself into a weird state that needs manual intervention.
=== Parsing YAML ===
<pre>Could not parse YAML data for node linux-ix-ref.build.mozilla.org: syntax error on line 72, col -1</pre>
Try deleting the corresponding file in /var/lib/puppet/yaml/node/
Djmitche
canmove, Confirmed users
1,394

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/285643"

Navigation menu