MozillaWiki

ReleaseEngineering/PuppetAgain/Certificate Chaining: Difference between revisions

Page Discussion

ReleaseEngineering/PuppetAgain/Certificate Chaining (view source)

Revision as of 18:29, 20 March 2013

1,615 bytes added ,  20 March 2013
no edit summary
(add note about ruby bug, workaround in commands)
No edit summary
Line 1: Line 1:
The Mozilla implementation of PuppetAgain has as one of its goals that any client can communicate freely with any master.  Ordinarily, each puppetmaster has its own certificate authority, and a client which has been issued a certificate by one master will not be recognized by another master.
The Mozilla implementation of PuppetAgain has as one of its goals that any client can communicate freely with any master.  Ordinarily, each puppetmaster has its own certificate authority, and a client which has been issued a certificate by one master will not be recognized by another master.


The solution is certificate chaining.  This section aims to include enough detail to help other puppet users reproduce this configuration elsewhere.
The solution is certificate chaining.
 
In this configuration, Puppet fits into a larger SSL certificate system.  At  
Mozilla, this is specifically used to allow hosts to issue and validate all
certificates without being directly connected.  This supports resiliency,
isolation of disclosure risk, and potentially (though not at Mozilla)
integration into an enterprise-wide certification hierarchy.
 


= Certificate Hierarchy =
= Certificate Hierarchy =
Line 7: Line 14:
We have a hierarchy of certificate authorities that looks like this:
We have a hierarchy of certificate authorities that looks like this:


      +-puppetmaster1 CA--+-puppetmaster1 server cert
    Root CA cert (Subject: CN=PuppetAgain Base CA/emailAddress=release@mozilla.com, OU=Release Engineering, O=Mozilla, Inc.)
      |                   |
    |
      |                   +-client 1 server cert
    +--Master CA cert (Subject: CN=CA on $fqdn/emailAddress=release@mozilla.com, O=Mozilla, Inc., OU=Release Engineering)
base--+                  :
    |  |
      |                  
    |  +--Master cert (Subject: CN=$fqdn)
      +-puppetmaster2 CA--+-puppetmaster2 server cert
    |  |
                          |
    |  +--Agent cert (Subject: CN=$fqdn)
                          +-client 10 server cert
    |   |
                          :
    |   +--Agent cert (Subject: CN=$fqdn)
    |  :
    |  .
    |
    +-- Master CA cert (Subject: CN=CA on $fqdn/emailAddress=release@mozilla.com, O=Mozilla, Inc., OU=Release Engineering)
    :  |
    .  +--Master cert (Subject: CN=$fqdn)
        |
        +--Agent cert (Subject: CN=$fqdn)
        |
        +--..
        :
        .
 
Note Master and Agent certs are functionally identical.


= Master Initialization =
= Master Initialization =


When a master is initialized, it is provided with a puppetmaster CA certificate and corresponding key.  This certificate is signed by the base CA.  It immediately creates a "leaf" certificate, suitable for authenticating an SSL connection, signed by this puppetmaster CA certificate.
When a master is initialized, it is provided with a puppetmaster CA certificate and corresponding key.  This certificate is signed by the base CA.  The puppet manifests for masters immediately create a Master cert, suitable for authenticating an SSL connection, signed by this Master CA cert.


= Certificate Issuance =
= Certificate Issuance =


When a client is issued a new certificate, after the requisite authentication has been performed (see [[ReleaseEngineering/PuppetAgain/Puppetization Process|Puppetization Process]]), a new certificate and key are generated and signed by the CA on the puppetmaster doing the issuing.  Any puppetmaster can issue a certificate.  This certificate and key, along with the base CA certificate, are returned to the client.
When a client is issued a new certificate, after the requisite authentication has been performed (see [[ReleaseEngineering/PuppetAgain/Puppetization Process|Puppetization Process]]), a new certificate and key are generated and signed by the CA on the puppetmaster doing the issuing.  Any puppetmaster can issue a certificate.  This certificate and key, along with the root CA certificate, are returned to the client.
 
The process looks something like this:
 
    # make a key
    openssl genrsa -des3 -out "/root/${host}.key.pass" -passout pass:x
 
    # strip its password
    openssl rsa -in "/root/${host}.key.pass" -out "/root/${host}.key" -passin pass:x
 
    # make a signing request
    cat <<EOF > "/root/${host}-openssl.conf"
    [req]
    prompt = no
    distinguished_name = clientcert_dn
 
    [clientcert_dn]
    commonName = ${host}
    EOF
    openssl req -config "/root/${host}-openssl.conf" -new -key "/root/${host}.key" -out "/root/${host}.csr"
 
    # sign it (this openssl.conf points to the local Master CA cert)
    openssl ca -config "${master_ssldir}/ca/openssl.conf" -in "/root/${host}.csr" -notext -out "/root/${host}.crt" \
        -batch -passin "file:${master_ssldir}/ca/private/ca.pass"
 
In particular, the `commonName` is the fqdn of the host.  Agents are configured
with the Root CA cert, the Agent cert, and the private key for the Agent cert.
 
Master CA certificates are generated by hand, using a process something like this:
 
    (put a nice long password in ${fqdn}-ca.password)
 
    $ openssl genrsa -des3 -out ${fqdn}-ca.key -passout file:${fqdn}-ca.password 2048
    $ vi openssl.conf
    [req]
    prompt = no
    distinguished_name = puppetmaster_ca_dn
 
    [puppetmaster_ca_dn]
    commonName = CA on ${fqdn}
    emailAddress = release@mozilla.com
    organizationalUnitName = Release Engineering
    organizationName = Mozilla, Inc.
 
    $ openssl req -config openssl.conf -new -key ${fqdn}-ca.key -out ${fqdn}-ca.csr -passin file:${fqdn}-ca.password
    $ openssl req -text -in ${fqdn}-ca.csr
 
The resulting CSR is then signed by the Root CA cert on an isolated system.


= Validation =
= Validation =
Line 60: Line 128:
   done
   done


Note that this directory is only consulted by Apache when validating the client certificates.
=== Validation of Agents ===
 
Apache takes care of validating agents.  The agents provide their Agent cert, and Apache searches the certdir for parent certificates until it reaches a valid self-signed cert.  Concretely, in this case it finds the Master CA cert that signed the Agent cert in certdir, and then finds the Root CA cert there.
 
=== Validation of Masters ===
 
Agents are configured with the Root CA cert.  Apache's ''SSLCertificateChainFile'' directive supplies the intermediate Master CA cert, and the ''SSLCertificateFile'' provides the Master cert.  These three certificates constitute the entire chain required for validation of the master by the agent.


== CRLs ==
== CRLs ==
Line 68: Line 142:
All of the CA's described above have corresponding CRLs, and these are made available to Apache with the SSLCARevocationPath directive, as seen above.
All of the CA's described above have corresponding CRLs, and these are made available to Apache with the SSLCARevocationPath directive, as seen above.


== Puppet Configuration ==
=== Validation ===
 
Apache consults its ''SSLCARevocationPath'' to check the corresponding CRLs for all CA certificates.  This directory is kept up to date by each master, and synchronized between masters using a cronjob.


Unfortunately, until [https://projects.puppetlabs.com/issues/14550|bug 14550] is fixed, the puppet agent cannot validate a certificate chain's CRLs, so CRL checking is disabled on the agent:
Unfortunately, until [https://projects.puppetlabs.com/issues/14550|bug 14550] is fixed, the puppet agent cannot validate a certificate chain's CRLs, so CRL checking is disabled on the agent:


  certificate_revocation = false
    certificate_revocation = false
 
On the master, you'll need to convince it not to try to do fancy CA-like things:
 
  ca = false
 
== Validation ==
 
Remember, Puppet validates certificates in both directions.  Let's start with the agent validating the master -- the more common direction for HTTPS.  The agent already has the base CA's certificate (provided during certificate issuance, above), and is configured to trust it and anything it signs.  The SSLCertificateFile and SSLCertificateKeyFile options point to the puppetmaster's leaf certificate ("puppetmaster1 server cert" in the ascii-art above) and its key.  The SSLCertificateChainFile contains the puppetmaster's CA certificate, and Apache includes this in the SSL setup transaction.  Thus, the agent has all three certificates needed to validate the master's certificate, and does so.  Some care is needed here to make sure that the necessary extensions are present on all of these certificates, as described below.
 
The master also validates the agent's certificate.  The client presents only its client certificate, so Apache must look up the parent certificates in the certdir.  This is critical, since the client certificate may have been signed by a different master.  After following this certificate chain, Apache consults any CRLs present in the certdir, as well, and refuses access to any revoked certificate, or any certificate signed by a revoked CA certificate.  Apache will only follow two levels of chained certificates (SSLVerifyDepth).


Well, that's not quite true: ''SSLVerifyClient optional'' means that Apache will run the puppetmaster app even if client validation fails.  The three RequestHeader directives pass information on the validation results to puppet, which decides how to handle the request (based, in part, on auth.conf, which is left at its default settings).
This means that agents may continue to trust a compromised Master cert or Master CA cert.


= Revocation =
= Revocation =


When it comes time to revoke a certificate, things can be a bit complicated.  Revocation requires access to the CA certificate and key, but that key only exists on the master that owns it.  So a method is required to indicate that a particular certificate should be revoked.  We accomplish this with a client_cert directory.  This directory has a subdirectory for each puppet master, and each such subdirectory has a ''revoke'' subdirectory for certificates that should be revoked.  The whole thing might look like
When it comes time to revoke a certificate, things can be a bit complicated.  Revocation of a compromised cert requires access to the CA certificate and key, but that key only exists on the master that created the compromised cert it.  So a method is required to indicate that a particular certificate should be revoked.  We accomplish this with a client_cert directory.  This directory has a subdirectory for each puppet master, and each such subdirectory has a ''revoke'' subdirectory for certificates that should be revoked.  The whole thing might look like
  client_certs/puppetmaster1.mozilla.com/client1.mozilla.com.crt
  client_certs/puppetmaster1.mozilla.com/client1.mozilla.com.crt
  client_certs/puppetmaster1.mozilla.com/client3.mozilla.com.crt
  client_certs/puppetmaster1.mozilla.com/client3.mozilla.com.crt
Line 97: Line 163:
where there is a pending revocation on puppetmaster1.  This client_certs directory is synchronized among all masters, and a crontask runs on each master that checks its ''revoke'' subdirectory for any pending revocations, and carries them out.  After each such revocation, a new CRL is generated and placed in the ''certdir'', where it is soon distributed again to all masters.
where there is a pending revocation on puppetmaster1.  This client_certs directory is synchronized among all masters, and a crontask runs on each master that checks its ''revoke'' subdirectory for any pending revocations, and carries them out.  After each such revocation, a new CRL is generated and placed in the ''certdir'', where it is soon distributed again to all masters.


= Commands =
= Puppet Configuration =
 
On the master, you'll need to convince it not to try to do fancy CA-like things:
 
  ca = false
 
= Command Reference =


All of this is accomplished with the command we all love to hate, ''openssl''.  Here are some of the commands used for various parts of this process.  This isn't a complete recipe because, honestly, you should struggle with this stuff a bit so that you understand it - it's easy to accidentally configure encryption to do absolutely nothing of value if you're not paying attention.  I've tried to call out the processes that took me the longest to figure out, and the things I stumbled over, in hopes you can make the journey more quickly than I did.
All of this is accomplished with the command we all love to hate, ''openssl''.  Here are some of the commands used for various parts of this process.  This isn't a complete recipe because, honestly, you should struggle with this stuff a bit so that you understand it - it's easy to accidentally configure encryption to do absolutely nothing of value if you're not paying attention.  I've tried to call out the processes that took me the longest to figure out, and the things I stumbled over, in hopes you can make the journey more quickly than I did.
Line 291: Line 363:
* note that in a certdir, certificates are hashed with an extension of '.0' (or .1 and so on if there are collisions), while CRLs are hashed with an extension ending of '.r0' (or .r1, ..).  This is not obvious *anywhere* except in the output of a 'strace' of Apache (seriously).
* note that in a certdir, certificates are hashed with an extension of '.0' (or .1 and so on if there are collisions), while CRLs are hashed with an extension ending of '.r0' (or .r1, ..).  This is not obvious *anywhere* except in the output of a 'strace' of Apache (seriously).
* Apache only checks CRLs that it can find; a CA certificate without a corresponding CRL is assumed to have no revoked certificates.  This can be helpful if you do not want to worry about expired CRLs, but could also be a security problem if you're not careful.
* Apache only checks CRLs that it can find; a CA certificate without a corresponding CRL is assumed to have no revoked certificates.  This can be helpful if you do not want to worry about expired CRLs, but could also be a security problem if you're not careful.
* Ruby's OpenSSL::SSL class ignores the distinguished name on a certificate if there are subjectAltNames defined.  This matters for server certs, which must thus include both their fqdn and 'puppet' if they are to be usable at both names.  See https://bugs.ruby-lang.org/issues/6493
* Ruby's OpenSSL::SSL class ignores the distinguished name on a certificate if there are subjectAltNames defined.  This matters for server certs, which must thus include both their fqdn and 'puppet' in the subjectAltNames option if they are to be usable at both names.  See https://bugs.ruby-lang.org/issues/6493
* CRLs potentially expire very quickly.  If you don't have plans in place to automatically regenerate CRLs, pick a very long expiration time for them (default_crl_days in openssl.conf).
* CRLs potentially expire very quickly.  If you don't have plans in place to automatically regenerate CRLs, pick a very long expiration time for them (default_crl_days in openssl.conf).


Djmitche
canmove, Confirmed users
1,394

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/639600"