ReleaseEngineering/PuppetAgain/Packages: Difference between revisions

Jump to navigation Jump to search

ReleaseEngineering/PuppetAgain/Packages (view source)

Revision as of 19:14, 25 July 2014

701 bytes added ,  25 July 2014
no edit summary
(Use a template)
No edit summary
Line 26: Line 26:
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/epel/6/latest/x86_64}}</td><td>EPEL 6</td><td>x86_64</td><td>2012-03-07</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/epel/6/latest/x86_64}}</td><td>EPEL 6</td><td>x86_64</td><td>2012-03-07</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/epel/6/latest/i386}}</td><td>EPEL 6</td><td>i386</td><td>2012-03-07*</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/epel/6/latest/i386}}</td><td>EPEL 6</td><td>i386</td><td>2012-03-07*</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/centos/6/latest/os/x86_64}}</td><td>CentOS 6 Base</td><td>x86_64</td><td>2012-03-07</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/centos/6.2/latest/os/x86_64}}</td><td>CentOS 6.2 Base</td><td>x86_64</td><td>2012-03-07</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/centos/6/latest/os/i386}}</td><td>CentOS 6 Base</td><td>i386</td><td>2012-03-07*</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/centos/6.2/latest/os/i386}}</td><td>CentOS 6.2 Base</td><td>i386</td><td>2012-03-07*</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/centos/6/latest/os/Source}}</td><td>CentOS 6 Base</td><td>source</td><td>2012-09-06</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/centos/6.2/latest/os/Source}}</td><td>CentOS 6.2 Base</td><td>source</td><td>2012-09-06</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/centos/6/latest/updates/x86_64}}</td><td>CentOS 6 Updates</td><td>x86_64</td><td>2012-03-07</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/centos/6.2/latest/updates/x86_64}}</td><td>CentOS 6.2 Updates</td><td>x86_64</td><td>2012-03-07</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/centos/6/latest/updates/i386}}</td><td>CentOS 6 Updates</td><td>i386</td><td>2012-03-07*</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/centos/6.2/latest/updates/i386}}</td><td>CentOS 6.2 Updates</td><td>i386</td><td>2012-03-07*</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/centos/6/latest/updates/Source}}</td><td>CentOS 6 Updates</td><td>source</td><td>2012-09-06</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/centos/6.2/latest/updates/Source}}</td><td>CentOS 6.2 Updates</td><td>source</td><td>2012-09-06</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/centos/6.5/os/x86_64}}</td><td>CentOS 6.5 Base</td><td>x86_64</td><td>2013-12-??</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/centos/6.5/os/i386}}</td><td>CentOS 6.5 Base</td><td>i386</td><td>2013-12-??</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/centos/6.5/extras/x86_64}}</td><td>CentOS 6.5 Extras</td><td>x86_64</td><td>2013-12-??</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/centos/6.5/extras/i386}}</td><td>CentOS 6.5 Extras</td><td>i386</td><td>2013-12-??</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/centos/6.5/updates/x86_64}}</td><td>CentOS 6.5 Updates</td><td>x86_64</td><td>2013-12-??</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/centos/6.5/updates/i386}}</td><td>CentOS 6.5 Updates</td><td>i386</td><td>2013-12-??</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/fedora/16/latest/releases/Everything/i386/os}}</td><td>Fedora 16 Base</td><td>i386</td><td></td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/fedora/16/latest/releases/Everything/i386/os}}</td><td>Fedora 16 Base</td><td>i386</td><td></td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/fedora/16/latest/releases/Everything/x86_64/os}}</td><td>Fedora 16 Base</td><td>x86_64</td><td></td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/fedora/16/latest/releases/Everything/x86_64/os}}</td><td>Fedora 16 Base</td><td>x86_64</td><td></td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/fedora/16/latest/updates/i386}}</td><td>Fedora 16 Updates</td><td>i386</td><td>2012-03-07</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/fedora/16/latest/updates/i386}}</td><td>Fedora 16 Updates</td><td>i386</td><td>2012-03-07</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/fedora/16/latest/updates/x86_64}}</td><td>Fedora 16 Updates</td><td>x86_64</td><td>2012-03-07</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/fedora/16/latest/updates/x86_64}}</td><td>Fedora 16 Updates</td><td>x86_64</td><td>2012-03-07</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/puppetlabs/el/6/products/x86_64}}</td><td>Puppetlabs</td><td>x86_64</td><td></td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/puppetlabs/el/6/products/x86_64}}</td><td>Puppetlabs</td><td>x86_64</td><td>as necessary</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/puppetlabs/el/6/products/i386}}</td><td>Puppetlabs</td><td>i386</td><td></td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/puppetlabs/el/6/products/i386}}</td><td>Puppetlabs</td><td>i386</td><td>as necessary</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/puppetlabs/el/6/dependencies/x86_64}}</td><td>Puppetlabs Deps</td><td>x86_64</td><td></td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/puppetlabs/el/6/dependencies/x86_64}}</td><td>Puppetlabs Deps</td><td>x86_64</td><td>as necessary</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/puppetlabs/el/6/dependencies/i386}}</td><td>Puppetlabs Deps</td><td>i386</td><td></td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/puppetlabs/el/6/dependencies/i386}}</td><td>Puppetlabs Deps</td><td>i386</td><td>as necessary</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/passenger/rhel/6/latest/x86_64}}</td><td>Passenger</td><td>x86_64</td><td>2012-07-05</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/passenger/rhel/6/latest/x86_64}}</td><td>Passenger</td><td>x86_64</td><td>2012-07-05</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/hp/proliantsupportpack/CentOS/6/i386/current/}}</td><td>HP Proliant Support</td><td>i386</td><td>2012-08-21</td></tr>
<tr><td>{{PuppetAgain Repo|repos/yum/mirrors/hp/proliantsupportpack/CentOS/6/i386/current/}}</td><td>HP Proliant Support</td><td>i386</td><td>2012-08-21</td></tr>
Line 52: Line 58:


Notes:
Notes:
* Paths under ''yum/releng'' are custom packages, and are not mirrored from anywhere.
* Repos under ''repos/yum/custom'' are custom-built for some purpose, and are not mirrored from anywhere.  They may contain custom packages, or just packages culled from some mirror.  Look for references to the repos in the puppet manifests to find their purpose.
* We generally try to mirror source RPMs for all repositories; this way, if we need to make a small fix to such an RPM, we can easily find the source for it without resorting to things like rpmfind.
* We generally try to mirror source RPMs for all repositories; this way, if we need to make a small fix to such an RPM, we can easily find the source for it without resorting to things like rpmfind.
* The 'releng' repos are deprecated; if you're tempted to add a package there, please create a new custom repository or update another existing custom repository instead
* The ''repos/yum/releng'' repos are deprecated; if you're tempted to add a package there, please create a new custom repository or update another existing custom repository instead
* The Fedora repos are unused and will be deleted soon
* The Fedora repos are unused and will be deleted soon
* Dynamic repositories are snapshots that are made on demand, where <tt>latest</tt> always points to the latest active snapshot.  They are *not* automatically updated.  The date on which they were most recently mirrored is given above.
* Dynamic repositories are snapshots that are made on demand, where <tt>latest</tt> always points to the latest active snapshot.  They are *not* automatically updated.  The date on which they were most recently mirrored is given above.
Line 80: Line 86:
     --delete --delete-excluded rsync://linux.mirrors.es.net/centos/$CENTOS_FULL/ /data/repos/yum/mirrors/centos/$CENTOS_FULL/
     --delete --delete-excluded rsync://linux.mirrors.es.net/centos/$CENTOS_FULL/ /data/repos/yum/mirrors/centos/$CENTOS_FULL/
  time hardlink -v /data/repos/yum/mirrors/centos
  time hardlink -v /data/repos/yum/mirrors/centos
Note that this pulls along 'SCL' and 'extras' and some other stuff.  As long as it's not huge, it doesn't hurt.


==== Fedora 16 ====
==== Fedora 16 ====
Line 122: Line 130:
Generally, if it's important enough to install explicitly, it's important enough to pin a particular version.  If you also need to pin versions for requirements, be sure you model the requirements with ''requires'' in puppet, so that puppet knows to install the requirements first.
Generally, if it's important enough to install explicitly, it's important enough to pin a particular version.  If you also need to pin versions for requirements, be sure you model the requirements with ''requires'' in puppet, so that puppet knows to install the requirements first.


== CentOS: Custom Packages ==
== CentOS: Building Custom Packages ==


See [[ReleaseEngineering/PuppetAgain/HowTo/Build RPMs]] for building RPMs.
See [[ReleaseEngineering/PuppetAgain/HowTo/Build RPMs]] for building RPMs.


== CentOS: Updated Versions of Upstream Packages ==
== CentOS: Installing Updated Versions of Upstream Packages ==


In the event you find a need for an updated package from newer CentOS repositories, first try installing that package manually (''yum install http://wherever.it.is/package.rpm'') on a target host, to ensure that it doesn't have any requirements that aren't satisfied from the mirrored repositories.  If there are any such requirements, consider carefully how many of them to cherry-pick out of the repository, and the effects that will have on other systems.  Upgrading ''librsync'' may be OK, but upgrading ''glibc'' or ''libopenssl'' this way might lead to a world of pain and sadness (noting that security releases often don't!).
In the event you find a need for an updated package from newer CentOS repositories, first try installing that package manually (''yum install http://wherever.it.is/package.rpm'') on a target host, to ensure that it doesn't have any requirements that aren't satisfied from the mirrored repositories.  If there are any such requirements, consider carefully how many of them to cherry-pick out of the repository, and the effects that will have on other systems.  Upgrading ''librsync'' may be OK, but upgrading ''glibc'' or ''libopenssl'' this way might lead to a world of pain and sadness (noting that security releases often don't!).


Before copying anything onto the puppet masters, "pin" the versions of the package in question and any requirements in puppet to what they are before your change, and deploy that patch.  This provides a backout path for you later to install exactly the versions that were installed before your changes.  Only when that change is deployed, add the new packages to the releng repository and run ''createrepo'' (below).  When *that* is deployed, update the puppet manifests to the new versions, omitting the requirements unless their version numbers are important for production.
Once you know the complete set of packages required, but before copying anything onto the puppet masters, "pin" the versions of the package in question and any requirements in puppet to what they are before your change, and deploy that patch.  This provides a backout path for you later to install exactly the versions that were installed before your changes.  Only when that change is deployed, add the new packages to the releng repository and run ''createrepo'' (below).  When *that* is deployed, update the puppet manifests to the new versions, omitting the requirements unless their version numbers are important for production.


{{note|Packages used in the mock environment require both i686 and x86_64 packages to be in the x86_64 repo, just like upstream}}
Now, build a custom repository (or possibly two, one for each architecture) on the distinguished master for the purpose.  For example, if you're updating openssh, create a new {{PuppetAgain Repo|repos/yum/custom/openssh}}.  If the appropriate repo already exists, use it.  Try to include SRPMs as well!


== CentOS: Landing Custom Repository Changes ==
Once you've assembled a directory containing the proper packages, run
  createrepo --update .
in that directory to update the metadata.  Don't forget to run ''puppetmaster-fixperms'' afterward to make sure permissions are correct.


Custom-built packages should be placed in the appropriate ''repos/yum/releng/public/CentOS/6/*'' repository, depending on architecture.  All dependencies should be included in that repository if they are not in the mirrored repositories.
If you added a new repository, you'll need to refer to it from the puppet configs.  Add a clause to ''modules/packages/manifests/setup.pp'', either a regular ''packages::yumrepo'' if the repo should be avialable on every host (like openssh) or a virtual one (prefixed with ''@'') if it should only be available on some hosts.  Only installing the repo on some hosts limits the carnage if something goes wrong with the repo.


When the patch containing the new or updated package spec is r+'d, commit it as usual, and also add *both* the RPM (or multiple RPMs if multiple architectures are required!) and the SRPM into /data on the designated puppet master, so that it will be distributed to other systems.  Debuginfo RPMs are a good idea, too.
Then, write or update the classes under ''modules/packages''.  If your repository is virtual, you'll need to add something like ''realize(Packages::Yumrepo['passenger'])'' to the package class to ensure the repo is in place.


Run
{{note|Packages used in the mock environment require both i686 and x86_64 packages to be in the x86_64 repo, just like upstream}}
  createrepo --update $repo_path
to update the metadata.  Don't forget to run ''puppetmaster-fixperms'' to make sure permissions are correct.
 
Once this is done, the package is available and will be used if possible.
 
Example session:
<pre>
# at releng-puppet2.srv.releng.scl3.mozilla.com
$ wget http://people.mozilla.org/~jhopkins/bug772446/supervisor-3.0-0.10.a12.el6.noarch.rpm
$ wget http://people.mozilla.org/~jhopkins/bug772446/supervisor-3.0-0.10.a12.el6.src.rpm
$ chmod 644 *.rpm
$ sudo chown puppetsync: *.rpm
$ sudo mv -vi *.rpm /data/repos/yum/releng/public/CentOS/6/noarch/
`supervisor-3.0-0.10.a12.el6.noarch.rpm' -> `/data/repos/yum/releng/public/CentOS/6/noarch/supervisor-3.0-0.10.a12.el6.noarch.rpm'
`supervisor-3.0-0.10.a12.el6.src.rpm' -> `/data/repos/yum/releng/public/CentOS/6/noarch/supervisor-3.0-0.10.a12.el6.src.rpm'
$ sudo createrepo --update /data/repos/yum/releng/public/CentOS/6/noarch
2/2 - supervisor-3.0-0.10.a12.el6.src.rpm                                     
Saving Primary metadata
Saving file lists metadata
Saving other metadata
# set permissions and ownership on repodata and package files
$ sudo puppetmaster-fixperms
</pre>


= Ubuntu =
= Ubuntu =
Line 256: Line 243:
As with CentOS packages, it's generally a good idea to pin the version of things that are important enough to be named in puppet.  If requirements need to be pinned too, then model the requirements relationship properly with ''requires''.
As with CentOS packages, it's generally a good idea to pin the version of things that are important enough to be named in puppet.  If requirements need to be pinned too, then model the requirements relationship properly with ''requires''.


== Ubuntu: Custom Packages ==
== Ubuntu: Building Custom Packages ==


See [[ReleaseEngineering/PuppetAgain/HowTo/Build DEBs]] for details on building DEBs.
See [[ReleaseEngineering/PuppetAgain/HowTo/Build DEBs]] for details on building DEBs.
Djmitche
canmove, Confirmed users
1,394

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1000530"

Navigation menu