ReleaseEngineering/How To/Update Mercurial

Mercurial is one of our core binaries that we need to install everywhere.

Philosophy

We install Mercurial into /tools/python27-mercurial/ everywhere* (* - On windows we override the mozilla-build version). This is due to wanting a single, common, place for mercurial, and to avoid needing to update system packages that may depend on mercurial.

This document is designed so it can/should be followed top-down, but with self-contained sections to allow easy skimming/read-through.

Future Issues

There are some forseen reasons this doc could be incorrect for your needs. Below are those reasons, and suggested solutions.

We change the way PuppetAgain goes about doing packaging
- Follow the new documentation to achieve a similar result, this page is informative not normative for how we do things.
We deployed PuppetAgain on windows (adding a need for windows to the puppet code)
- We'll need to do this from scratch once, and update these docs.
OSX 10.9 is no longer used in our infra (bug 1110612)
- Delete the 10.9 instructions from this wiki page, and rejoice

Prepare for the Upgrade

File a bug for the upgrade, (if not already filed)
Run the hgtools tests against the new mercurial version (TODO: Link to doc how)
Send an e-mail to Dev Services <developer-services@mozilla.org> asking if there are any concerns (set a deadline, so you are not indefinitely blocked on replies)

Create the Packages

CentOS

6.x

(This is an informative description of The PuppetAgain How to Build RPMs)

In us-east AWS console find and start rpmpackager1.srv.releng.use1.mozilla.com
Connect

USER=`whomai`
ssh $USER@rpmpackager1.srv.releng.use1.mozilla.com

Get Old RPM, Spec from puppet, and extract spec from rpm, Also get new sources

NEWHGVER=4.0  # Set this appropriately
NEWHGREL=1 #Set this too
OLDHGVER=3.2.1
OLDHGREL=1
OLDHGFULL=$OLDHGVER-$OLDHGREL
NEWHGFULL=$NEWHGVER-$NEWHGREL
PUPPETAGAIN=http://puppetagain.pub.build.mozilla.org
HGREPO=data/repos/yum/releng/public/CentOS/6
mkdir mercurial-$NEWHGVER; cd mercurial-$NEWHGVER
wget $PUPPETAGAIN/$HGREPO/x86_64/mozilla-python27-mercurial-$OLDHGFULL.el6.x86_64.rpm
wget $PUPPETAGAIN/$HGREPO/i386/mozilla-python27-mercurial-$OLDHGFULL.el6.i686.rpm
wget $PUPPETAGAIN/$HGREPO/x86_64/mozilla-python27-mercurial-$OLDHGFULL.el6.src.rpm
mkdir old-extracted; cd old-extracted
rpm2cpio ../mozilla-python27-mercurial-$OLDHGFULL.el6.src.rpm | cpio -ivd
cd ..
wget http://mercurial.selenic.com/release/mercurial-$NEWHGVER.tar.gz

Check for any unexpected differences between puppet and old version

diff py27_mercurial.spec old-extracted/py27_mercurial.spec

Update spec for versions

sed -i "s/\(Version:.*\)$OLDHGVER/\1$NEWHGVER/" py27_mercurial.spec
sed "s/\(Release:.*\)$OLDHGREL/\1$NEWHGREL/" py27_mercurial.spec

Build the src rpm

mock -r puppetagain-centos6-64 --buildsrpm --sources $PWD --spec py27_mercurial.spec

Copy output to a safe place

mkdir output
cp /var/lib/mock/epel-6-x86_64/result/mozilla-python27-mercurial-$NEWHGFULL.el6.src.rpm output/

Now build the actual rpms (and clean caches first)

sudo rm -rf /var/cache/mock/epel-6-x86_64/ccache
sudo rm -rf /var/cache/mock/epel-6-i386/ccache/
mock --rebuild -r puppetagain-centos6-64 output/mozilla-python27-mercurial-$NEWHGFULL.el6.src.rpm
mock --rebuild -r puppetagain-centos6-32 output/mozilla-python27-mercurial-$NEWHGFULL.el6.src.rpm

Copy those out (if nothing failed):

cp /var/lib/mock/epel-6-x86_64/result/*.x86_64.rpm output/
cp /var/lib/mock/epel-6-x86_64/result/*.i?86.rpm output/

IF there are any ERRORS
- Edit the spec file to account for them and retry subsequent steps, you'll need to carry these spec file changes over into the Puppet Section of the work.
scp the files in output to somewhere local, in prep for the #Deploy to PuppetAgain Section
Halt the system (avoids need for aws console)

sudo su -
users # Check for other logged in users first, be nice!
halt

Ubuntu

Precise (12.04)

(This is an informative description of The PuppetAgain How to Build DEBs)

In us-east AWS console find and start ubuntu64packager1.srv.releng.use1.mozilla.com
Connect

USER=`whomai`
ssh $USER@ubuntu64packager1.srv.releng.use1.mozilla.com

Get old from repo

NEWHGVER=4.0  # Set this appropriately
OLDHGVER=3.2.1
PUPPETAGAIN=http://puppetagain.pub.build.mozilla.org
HGREPO=data/repos/apt/custom/mozilla-mercurial/pool/precise
mkdir old
wget $PUPPETAGAIN/$HGREPO/mozilla-python27-mercurial_3.2.1.debian.tar.gz
cd old
tar -xzf ../mozilla-python27-mercurial_3.2.1.debian.tar.gz  
cd ..

Get old deb folder from puppet

hg clone https://hg.mozilla.org/build/puppet repo-puppet
cp -r puppet/modules/packages/manifests/mozilla/mozilla-python27-mercurial-debian ./

Check for surprising differences

diff -r ./mozilla-python27-mercurial-debian/ old/debian/

Edit changelog (adding new entry)
- Note DEB Changelogs follow a specific format, add to top, match the one below yours.

EDITOR=vim
$EDITOR ./mozilla-python27-mercurial-debian/changelog

Get the new .tar.gz from selenic (and make sure its named correctly post-download)

wget http://mercurial.selenic.com/release/mercurial-$NEWHGVER.tar.gz \
     -O mozilla-python27-mercurial_$NEWHGVER.orig.tar.gz

Build it

puppetagain-build-deb precise amd64 mozilla-python27-mercurial-debian
puppetagain-build-deb precise i386 mozilla-python27-mercurial-debian

Copy the artifacts out

cp /tmp/mozilla-python27-mercurial-precise-amd64/* output/
cp /tmp/mozilla-python27-mercurial-precise-i386/* output/

scp the files in output to somewhere local, in prep for the #Deploy to PuppetAgain Section
copy out any/all changes you have made to the puppet deb directory (the changelog edit above) - You'll include that with your puppet patch.
Halt the system (avoids need for aws console)

sudo su -
users # Check for other logged in users first, be nice!
halt

OSX

(This is an informative description based on The PuppetAgain How to Build DMGs)

OSX is funky and needs a package built separately for each version, on the OS version its targeted for. This means we need to pull active slaves from rotation and install XCode on them via a puppet user environment.

I'm not repeating the puppet user environment instructions here, but I will detail what you need to change for these nodes.

10.6 (Snow Leopard)

Disable a random production slave, commenting with whatever bug you need it for, (you're essentially doing a loan - but need puppet running)
Make sure you have a (clean/updated) puppetagain environment setup you can use (See The Docs)
Add packagemaker and xcode to the node def for testers (applies to all OSX jobs)
- Note This will break any tester puppet runs for non OSX (but is fixable by reverting this)

# linux64 and OS X
node /t.*-\d+\.test\.releng\.scl3\.mozilla\.com/ {
    # hosts starting with t and ending in -digit.test.releng.scl3.mozilla.com
    $slave_trustlevel = 'try'
    include toplevel::slave::releng::test::gpu
    # New Lines Here
    include packages::packagemaker
    include packages::xcode
}

10.7 (Lion)

Disable a random production slave, commenting with whatever bug you need it for, (you're essentially doing a loan - but need puppet running)
- There is no need for puppet environment for this, its a builder and already has xcode

10.8 (Mountain Lion)

Disable a random production slave, commenting with whatever bug you need it for, (you're essentially doing a loan - but need puppet running)
Make sure you have a (clean/updated) puppetagain environment setup you can use (See The Docs)
Add packagemaker and xcode to the node def for testers (applies to all OSX jobs)
- Note This will break any tester puppet runs for non OSX (but is fixable by reverting this)

# linux64 and OS X
node /t.*-\d+\.test\.releng\.scl3\.mozilla\.com/ {
    # hosts starting with t and ending in -digit.test.releng.scl3.mozilla.com
    $slave_trustlevel = 'try'
    include toplevel::slave::releng::test::gpu
    # New Lines Here
    include packages::packagemaker
    include packages::xcode
}

10.9 (Mavericks)

WARNING As of this writing we only have mac-v2-signing using mavericks, however we do need mercurial on these systems.
The version built on 10.8 worked with mavericks as of this docs writing, so instead of giving full docs here for building a mavericks version, I'm hopeful next hg ver works on mavericks when built on 10.8 or we no longer have mavericks in production.

10.10 (Yosemite)

Disable a random production slave, commenting with whatever bug you need it for, (you're essentially doing a loan - but need puppet running)
Make sure you have a (clean/updated) puppetagain environment setup you can use (See The Docs)
Add packagemaker and xcode to the node def for testers (applies to all OSX jobs)
- Note This will break any tester puppet runs for non OSX (but is fixable by reverting this)

# linux64 and OS X
node /t.*-\d+\.test\.releng\.scl3\.mozilla\.com/ {
    # hosts starting with t and ending in -digit.test.releng.scl3.mozilla.com
    $slave_trustlevel = 'try'
    include toplevel::slave::releng::test::gpu
    # New Lines Here
    include packages::packagemaker
    include packages::xcode
}

ReleaseEngineering/How To/Update Mercurial

Contents

Philosophy

Future Issues

Prepare for the Upgrade

Create the Packages

CentOS

6.x

Ubuntu

Precise (12.04)

OSX

10.6 (Snow Leopard)

10.7 (Lion)

10.8 (Mountain Lion)

10.9 (Mavericks)

10.10 (Yosemite)

Navigation menu

ReleaseEngineering/How To/Update Mercurial

Philosophy

Future Issues

Prepare for the Upgrade

Create the Packages

CentOS

6.x

Ubuntu

Precise (12.04)

OSX

10.6 (Snow Leopard)

10.7 (Lion)

10.8 (Mountain Lion)

10.9 (Mavericks)

10.10 (Yosemite)

Navigation menu

Search