ReleaseEngineering/PuppetAgain/Repositories: Difference between revisions
Jump to navigation
Jump to search
| Line 50: | Line 50: | ||
rsync -n --no-p --delete --size-only -rv --exclude=EFI --exclude=drpms --exclude=images --exclude=isolinux --exclude='RPM-GPG-KEY*' rsync://mirrors.usc.edu/centos/6/os/x86_64/ /data/repos/yum/mirrors/centos/6/2012-03-07/os/x86_64 | rsync -n --no-p --delete --size-only -rv --exclude=EFI --exclude=drpms --exclude=images --exclude=isolinux --exclude='RPM-GPG-KEY*' rsync://mirrors.usc.edu/centos/6/os/x86_64/ /data/repos/yum/mirrors/centos/6/2012-03-07/os/x86_64 | ||
rsync -n --no-p -rv --exclude=drpms rsync://mirrors.usc.edu/centos/6/updates/x86_64/ /data/repos/yum/mirrors/centos/6/2012-03-07/updates/x86_64 | rsync -n --no-p -rv --exclude=drpms rsync://mirrors.usc.edu/centos/6/updates/x86_64/ /data/repos/yum/mirrors/centos/6/2012-03-07/updates/x86_64 | ||
This was badly botched. Among other things, group information was omitted from the repodata. | This was badly botched. Among other things, group information was omitted from the repodata. Fixing that directly turns out to close the trees. So we have re-mirrored the 6.2 os repos as of 2013-01-17 at `/data/repos/yum/mirrors/centos/6/2012-03-07/os/x86_64-for-ks`. | ||
* Base and Updates (6.2, i386) | * Base and Updates (6.2, i386) | ||
Revision as of 23:53, 17 January 2013
Yum Repositories
Apt Repositories
| url | repository | arch | section | dist | mirror date |
|---|---|---|---|---|---|
| repos/apt/ubuntu | Ubuntu 12.04 LTS | i386,amd64 | main,restricted | precise,precise-security | (note: no updates) |
Sync Commands
Note: Note that we generally try to mirror source RPMs for all repositories; this way, if we need to make a small fix to such an RPM, we can easily find the source for it without resorting to things like rpmfind
EPEL
rsync -n --no-p -rv --delete --size-only rsync://mirror.unl.edu/fedora-epel/6/x86_64/ /data/repos/yum/mirrors/epel/6/2012-03-07/x86_64/ rsync -n --no-p -rv --delete --size-only rsync://mirror.unl.edu/fedora-epel/6/i386/ /data/repos/yum/mirrors/epel/6/2012-03-07/i386/
Note that, as for CentOS below, the i386 mirrors were done much later than 2012-03-07 (July 2012).
CentOS 6
- Base and Updates (6.2, x86_64)
rsync -n --no-p --delete --size-only -rv --exclude=EFI --exclude=drpms --exclude=images --exclude=isolinux --exclude='RPM-GPG-KEY*' rsync://mirrors.usc.edu/centos/6/os/x86_64/ /data/repos/yum/mirrors/centos/6/2012-03-07/os/x86_64 rsync -n --no-p -rv --exclude=drpms rsync://mirrors.usc.edu/centos/6/updates/x86_64/ /data/repos/yum/mirrors/centos/6/2012-03-07/updates/x86_64
This was badly botched. Among other things, group information was omitted from the repodata. Fixing that directly turns out to close the trees. So we have re-mirrored the 6.2 os repos as of 2013-01-17 at `/data/repos/yum/mirrors/centos/6/2012-03-07/os/x86_64-for-ks`.
- Base and Updates (6.2, i386)
rsync -n --exclude isolinux -aP rsync://linux.mirrors.es.net/centos/6.2/os/i386/ /data/repos/yum/mirrors/centos/6/2012-03-07/updates/i386/ rsync -n --exclude drpms -aP rsync://linux.mirrors.es.net/centos/6.2/updates/i386/ /data/repos/yum/mirrors/centos/6/2012-03-07/os/i386/
NOTE: the i386 CentOS 6.2 repos were mirrored in July of 2012, but are in the 2012-03-07 directory. See bug 773379.
- Base and Updates (6.3 and higher)
CENTOS_MAJOR=6 CENTOS_FULL=6.3 DATE=2012-07-12 rsync -v -n -aP \ --filter='-r updates/Source' --filter='-r os/Source' --filter='-r centos-version.txt'\ --exclude isos --exclude drpms \ --delete --delete-excluded \ rsync://linux.mirrors.es.net/centos/$CENTOS_FULL/ /data/repos/yum/mirrors/centos/$CENTOS_MAJOR/$DATE/ rsync -v -n -aP \ rsync://mirror.nsc.liu.se/centos-store/$CENTOS_FULL/os/Source /data/repos/yum/mirrors/centos/$CENTOS_MAJOR/$DATE/os/Source rsync -v -n -aP \ rsync://mirror.nsc.liu.se/centos-store/$CENTOS_FULL/updates/Source /data/repos/yum/mirrors/centos/$CENTOS_MAJOR/$DATE/updates/Source echo $CENTOS_FULL > /data/repos/yum/mirrors/centos/$CENTOS_MAJOR/$DATE/centos-version.txt
- sync these at "nearly" the same time, so that we have a good chance of having the srpm for a particular RPM
- when re-mirroring, it may save a bit of time and space to use rsync's hard-link capabilities to copy the last date to the new date. This can be done with the --link-dest option, but you'll more likely want to do it on every host (this will save a lot of space!), like so:
OLDDATE=2012-07-12 NEWDATE=2012-09-06 mkdir /data/repos/yum/mirrors/centos/6/$NEWDATE/ rsync -v -aP --link-dest=/data/repos/yum/mirrors/centos/6/$OLDDATE/ /data/repos/yum/mirrors/centos/6/$OLDDATE/ /data/repos/yum/mirrors/centos/6/$NEWDATE/
Fedora 16
rsync -n --no-p --delete --size-only -rv --exclude=EFI --exclude=drpms --exclude=images --exclude=isolinux --exclude='RPM-GPG-KEY*' rsync://mirrors.usc.edu/fedora/linux/releases/16/Everything/i386/os/ /data/repos/yum/mirrors/fedora/16/2012-03-07/releases/Everything/i386/os/ rsync -n --no-p --delete --size-only -rv --exclude=EFI --exclude=drpms --exclude=images --exclude=isolinux --exclude='RPM-GPG-KEY*' rsync://fedora.osuosl.org/fedora/linux/releases/16/Everything/x86_64/os/ /data/repos/yum/mirrors/fedora/16/2012-03-07/releases/Everything/x86_64/os/ rsync -n --no-p --delete --size-only -rv --exclude=drpms --exclude=debug rsync://fedora.osuosl.org/fedora/linux/updates/16/i386/ /data/repos/yum/mirrors/fedora/16/2012-03-07/updates/i386/ rsync -n --no-p --delete --size-only -rv --exclude=drpms --exclude=debug rsync://fedora.osuosl.org/fedora/linux/updates/16/x86_64/ /data/repos/yum/mirrors/fedora/16/2012-03-07/updates/x86_64/
PupppetLabs
rsync -n -rLv rsync://yum.puppetlabs.com/packages/yum/el/6/products/x86_64/ /data/repos/yum/mirrors/puppetlabs/el/6/products/x86_64/ rsync -n -rLv rsync://yum.puppetlabs.com/packages/yum/el/6/products/i386/ /data/repos/yum/mirrors/puppetlabs/el/6/products/i386/
Puppet-2.7.18 and 2.7.19 do not work with the SSL certificate chaining (https://projects.puppetlabs.com/issues/15561), so those RPMs have been removed and `createrepo .` run to remove the evidence.
Passenger
rsync -n -av rsync://passenger.stealthymonkeys.com/rpms/rhel/6/x86_64/ /data/repos/yum/mirrors/passenger/rhel/6/2012-07-05/x86_64/ rsync -n -av rsync://passenger.stealthymonkeys.com/rpms/rhel/6/i386/ /data/repos/yum/mirrors/passenger/rhel/6/2012-07-05/i386/
hp
not everything is mirrored, so this just gets the good stuff
rsync -n -av downloads.linux.hp.com::SDR/psp/CentOS/6/x86_64/current/ /data/repos/yum/mirrors/hp/proliantsupportpack/CentOS/6/x86_64/current/ rsync -n -av downloads.linux.hp.com::SDR/psp/CentOS/6/i386/current/ /data/repos/yum/mirrors/hp/proliantsupportpack/CentOS/6/i386/current/
Ubuntu
GNUPGHOME has the Ubuntu arch key in it:
GNUPGHOME=/etc/debmirror-gpg gpg --no-default-keyring --keyring /etc/debmirror-gpg/trustedkeys.gpg --keyserver keyserver.ubuntu.com --recv-keys 437D05B5
mirroring:
GNUPGHOME=/etc/debmirror-gpg/ debmirror --config-file=/etc/debmirror.conf -a i386,amd64 --no-source -s main,restricted -d precise,precise-security -h us.archive.ubuntu.com -r /ubuntu -e rsync --progress /data/repos/apt/ubuntu/
Notes
- Dynamic repositories are snapshots that are made on demand, where latest always points to the latest active snapshot. They are *not* automatically updated. The date on which they were most recently mirrored is given above.
- The current CentOS version is available at http://repos/repos/yum/mirrors/centos/6/latest/centos-version.txt.
- The CentOS base repos each have an images/ subdirectory, so they can be used with the 'url' command in a kickstart script
Mac OS X DMGs
DMGs are stored in repos/DMGs. This is a flat directory, unlike the heavily nested RPM directories. Each DMG is named $packagename-$version.dmg.
DMGs that are downloaded as-is should be documented briefly (at least with the source) in the manifest files that install them (e.g., puppet.pp).
DMGs that are custom built should have a shell script in modules/packages/manifests named $package-dmg.sh which builds the DMG. If there is a corresponding RPM (custom or stock) for the package, then the shell script can require that the source RPM be unpacked first. See ReleaseEngineering/PuppetAgain/HowTo/Build_DMGs for more details.