ReleaseEngineering/How To/Update Mercurial: Difference between revisions

Jump to navigation Jump to search

ReleaseEngineering/How To/Update Mercurial (view source)

Revision as of 14:39, 27 September 2016

2,432 bytes removed ,  27 September 2016
Updated wiki page on how to update Mercurial
No edit summary
(Updated wiki page on how to update Mercurial)
Line 11: Line 11:
* We change the way [[ReleaseEngineering/PuppetAgain|PuppetAgain]] goes about doing packaging
* We change the way [[ReleaseEngineering/PuppetAgain|PuppetAgain]] goes about doing packaging
** Follow the new documentation to achieve a similar result, this page is informative not normative for how we do things.
** Follow the new documentation to achieve a similar result, this page is informative not normative for how we do things.
* We deployed PuppetAgain on windows (adding a need for windows to the puppet code)
* We deployed PuppetAgain on Windows (adding a need for Windows to the puppet code)
** We'll need to do this from scratch once, and update these docs.
** We'll need to do this from scratch once, and update these docs.
* OSX 10.9 is no longer used in our infra ({{bug|1110612}})
* OSX 10.6 will soon no longer be used in our infra
** Delete the 10.9 instructions from this wiki page, and rejoice
** Delete the 10.6 instructions from this wiki page when that happens


== Prepare for the Upgrade ==
== Prepare for the Upgrade ==
* File a bug for the upgrade, (if not already filed)
* File a bug for the upgrade, (if not already filed)
* Run the ''hg''tools tests against the new mercurial version ('''TODO: Link to doc how''')
* Send an e-mail to <code>Dev Services <developer-services@mozilla.org></code> asking if there are any concerns (set a deadline, so you are not indefinitely blocked on replies)
* Send an e-mail to <code>Dev Services <developer-services@mozilla.org></code> asking if there are any concerns (set a deadline, so you are not indefinitely blocked on replies)


Line 31: Line 30:
  ssh $USER@rpmpackager1.srv.releng.use1.mozilla.com
  ssh $USER@rpmpackager1.srv.releng.use1.mozilla.com
* Get Old RPM, Spec from puppet, and extract spec from rpm, Also get new sources
* Get Old RPM, Spec from puppet, and extract spec from rpm, Also get new sources
  NEWHGVER=4.0 # Set this appropriately
  NEWHGVER=3.9.1 # Set this appropriately
  NEWHGREL=1 #Set this too
  NEWHGREL=1 #Set this too
  OLDHGVER=3.2.1
  OLDHGVER=3.7.3
  OLDHGREL=1
  OLDHGREL=1
  OLDHGFULL=$OLDHGVER-$OLDHGREL
  OLDHGFULL=$OLDHGVER-$OLDHGREL
  NEWHGFULL=$NEWHGVER-$NEWHGREL
  NEWHGFULL=$NEWHGVER-$NEWHGREL
  PUPPETAGAIN=http://puppetagain.pub.build.mozilla.org
  PUPPETAGAIN=http://puppetagain.pub.build.mozilla.org
  HGREPO=data/repos/yum/releng/public/CentOS/6
  HGREPO=data/repos/yum/custom/mozilla-mercurial/
  mkdir mercurial-$NEWHGVER; cd mercurial-$NEWHGVER
  mkdir mercurial-$NEWHGVER; cd mercurial-$NEWHGVER
  wget $PUPPETAGAIN/$HGREPO/x86_64/mozilla-python27-mercurial-$OLDHGFULL.el6.x86_64.rpm
  wget $PUPPETAGAIN/$HGREPO/x86_64/mozilla-python27-mercurial-$OLDHGFULL.el6.x86_64.rpm
Line 46: Line 45:
  rpm2cpio ../mozilla-python27-mercurial-$OLDHGFULL.el6.src.rpm | cpio -ivd
  rpm2cpio ../mozilla-python27-mercurial-$OLDHGFULL.el6.src.rpm | cpio -ivd
  cd ..
  cd ..
  wget http://mercurial.selenic.com/release/mercurial-$NEWHGVER.tar.gz
  wget https://www.mercurial-scm.org/release/mercurial-$NEWHGVER.tar.gz
* Check for any unexpected differences between puppet and old version
* Check for any unexpected differences between puppet and old version
  diff py27_mercurial.spec old-extracted/py27_mercurial.spec
  diff py27_mercurial.spec old-extracted/py27_mercurial.spec
Line 82: Line 81:
  ssh $USER@ubuntu64packager1.srv.releng.use1.mozilla.com
  ssh $USER@ubuntu64packager1.srv.releng.use1.mozilla.com
* Get old from repo
* Get old from repo
  NEWHGVER=4.0 # Set this appropriately
  NEWHGVER=3.9.1 # Set this appropriately
  OLDHGVER=3.2.1
  OLDHGVER=3.7.3
  PUPPETAGAIN=http://puppetagain.pub.build.mozilla.org
  PUPPETAGAIN=http://puppetagain.pub.build.mozilla.org
  HGREPO=data/repos/apt/custom/mozilla-mercurial/pool/precise
  HGREPO=data/repos/apt/custom/mozilla-mercurial/pool/precise
  mkdir old
  mkdir old
  wget $PUPPETAGAIN/$HGREPO/mozilla-python27-mercurial_3.2.1.debian.tar.gz
  wget $PUPPETAGAIN/$HGREPO/mozilla-python27-mercurial_$OLDHGVER.debian.tar.gz
  cd old
  cd old
  tar -xzf ../mozilla-python27-mercurial_3.2.1.debian.tar.gz   
  tar -xzf ../mozilla-python27-mercurial_$OLDHGVER.debian.tar.gz   
  cd ..
  cd ..
* Get old deb folder from puppet
* Get old deb folder from puppet
Line 101: Line 100:
  $EDITOR ./mozilla-python27-mercurial-debian/changelog
  $EDITOR ./mozilla-python27-mercurial-debian/changelog
* Get the new .tar.gz from selenic (and make sure its named correctly post-download)
* Get the new .tar.gz from selenic (and make sure its named correctly post-download)
  wget http://mercurial.selenic.com/release/mercurial-$NEWHGVER.tar.gz \
  wget https://www.mercurial-scm.org/release/mercurial-$NEWHGVER.tar.gz \
       -O mozilla-python27-mercurial_$NEWHGVER.orig.tar.gz
       -O mozilla-python27-mercurial_$NEWHGVER.orig.tar.gz
* Build it
* Build it
Line 138: Line 137:
* login to your host and run puppet
* login to your host and run puppet
** '''''CHANGE THE PUPPET ARGS APPROPRIATELY''''
** '''''CHANGE THE PUPPET ARGS APPROPRIATELY''''
  SLAVE=t-snow-r4-0001
  SLAVE=t-snow-r4-0089
  ssh root@$SLAVE.build.mozilla.org
  ssh root@$SLAVE.test.releng.scl3.mozilla.com
  USER=yourusername
  USER=yourusername
  puppet agent --server foo --test --environment $USER
  puppet agent --server foo --test --environment $USER
Line 151: Line 150:
  chmod 755 ./py27_mercurial-dmg.sh
  chmod 755 ./py27_mercurial-dmg.sh
* Update the script to be the correct version
* Update the script to be the correct version
  OLDHGVER=3.2.1
  OLDHGVER=3.7.3
  NEWHGVER=4.0
  NEWHGVER=3.9.1
  sed -i "s/version=$OLDHGVER/version=$NEWHGVER/" ./py27_mercurial-dmg.sh
  sed -i "s/version=$OLDHGVER/version=$NEWHGVER/" ./py27_mercurial-dmg.sh
* Get the new tar.gz  
* Get the new tar.gz  
  wget http://mercurial.selenic.com/release/mercurial-$NEWHGVER.tar.gz
  wget https://www.mercurial-scm.org/release/mercurial-$NEWHGVER.tar.gz
* Run the script
* Run the script
  ./py27_mercurial-dmg.sh
  ./py27_mercurial-dmg.sh
Line 172: Line 171:
  chmod 755 ./py27_mercurial-dmg.sh
  chmod 755 ./py27_mercurial-dmg.sh
* Update the script to be the correct version
* Update the script to be the correct version
  OLDHGVER=3.2.1
  OLDHGVER=3.7.3
  NEWHGVER=4.0
  NEWHGVER=3.9.1
  sed -i "s/version=$OLDHGVER/version=$NEWHGVER/" ./py27_mercurial-dmg.sh
  sed -i "s/version=$OLDHGVER/version=$NEWHGVER/" ./py27_mercurial-dmg.sh
* Get the new tar.gz  
* Get the new tar.gz  
  wget http://mercurial.selenic.com/release/mercurial-$NEWHGVER.tar.gz
  wget https://www.mercurial-scm.org/release/mercurial-$NEWHGVER.tar.gz
* Run the script
* Run the script
  ./py27_mercurial-dmg.sh
  ./py27_mercurial-dmg.sh
* The script will output where the dmg is, copy it outside the host (storing it in a way that you know it was for 10.7)
* The script will output where the dmg is, copy it outside the host (storing it in a way that you know it was for 10.7)
* You now want to reimage the slave following [https://mana.mozilla.org/wiki/pages/viewpage.action?pageId=34014655 this mana doc], and put it back in production in slavealloc
* You now want to reimage the slave following [https://mana.mozilla.org/wiki/pages/viewpage.action?pageId=34014655 this mana doc], and put it back in production in slavealloc
==== 10.8 (Mountain Lion) ====
* Disable a random production slave, commenting with whatever bug you need it for, (you're essentially doing a loan - but need puppet running)
* Make sure you have a (clean/updated) puppetagain environment setup you can use (See [[ReleaseEngineering/PuppetAgain/HowTo/Set_up_a_user_environment|The Docs]])
* Add packagemaker and xcode to the node def for testers (applies to all OSX jobs)
** '''Note''' This will break any tester puppet runs for non OSX (but is fixable by reverting this)
# linux64 and OS X
node /t.*-\d+\.test\.releng\.scl3\.mozilla\.com/ {
    # hosts starting with t and ending in -digit.test.releng.scl3.mozilla.com
    $slave_trustlevel = 'try'
    include toplevel::slave::releng::test::gpu
    # New Lines Here
    include packages::packagemaker
    include packages::xcode
}
* login to your host and run puppet
** '''''CHANGE THE PUPPET ARGS APPROPRIATELY''''
SLAVE=talos-mtnlion-r5-001
ssh root@$SLAVE.build.mozilla.org
USER=yourusername
puppet agent --server foo --test --environment $USER
* change to cltbld
su - cltbld
* Get the necessary file:
mkdir mercurial
cd mercurial
wget -O py27_mercurial-dmg.sh \
    http://hg.mozilla.org/build/puppet/raw-file/default/modules/packages/manifests/mozilla/py27_mercurial-dmg.sh
chmod 755 ./py27_mercurial-dmg.sh
* Update the script to be the correct version
OLDHGVER=3.2.1
NEWHGVER=4.0
sed -i "s/version=$OLDHGVER/version=$NEWHGVER/" ./py27_mercurial-dmg.sh
* Get the new tar.gz
wget http://mercurial.selenic.com/release/mercurial-$NEWHGVER.tar.gz
* Run the script
./py27_mercurial-dmg.sh
* The script will output where the dmg is, copy it outside the host (storing it in a way that you know it was for 10.8)
* You now want to reimage the slave following [https://mana.mozilla.org/wiki/pages/viewpage.action?pageId=34014655 this mana doc], and put it back in production in slavealloc
==== 10.9 (Mavericks) ====
* '''WARNING''' As of this writing we only have mac-v2-signing using mavericks, however we do need mercurial on these systems.
* The version built on 10.8 worked with mavericks as of this docs writing, so instead of giving full docs here for building a mavericks version, I'm hopeful next hg ver works on mavericks when built on 10.8 '''or''' we no longer have mavericks in production.
* Deployment section will do the deploy of the 10.8 version for Mavericks


==== 10.10 (Yosemite) ====
==== 10.10 (Yosemite) ====
Line 242: Line 197:
* login to your host and run puppet
* login to your host and run puppet
** '''''CHANGE THE PUPPET ARGS APPROPRIATELY''''
** '''''CHANGE THE PUPPET ARGS APPROPRIATELY''''
  SLAVE=talos-t-yosemite-r5-0001
  SLAVE=talos-t-yosemite-r7-0089
  ssh root@$SLAVE.build.mozilla.org
  ssh root@$SLAVE.test.releng.scl3.mozilla.com
  USER=yourusername
  USER=yourusername
  puppet agent --server foo --test --environment $USER
  puppet agent --server foo --test --environment $USER
Line 255: Line 210:
  chmod 755 ./py27_mercurial-dmg.sh
  chmod 755 ./py27_mercurial-dmg.sh
* Update the script to be the correct version
* Update the script to be the correct version
  OLDHGVER=3.2.1
  OLDHGVER=3.7.3
  NEWHGVER=4.0
  NEWHGVER=3.9.1
  sed -i "s/version=$OLDHGVER/version=$NEWHGVER/" ./py27_mercurial-dmg.sh
  sed -i "s/version=$OLDHGVER/version=$NEWHGVER/" ./py27_mercurial-dmg.sh
* Get the new tar.gz  
* Get the new tar.gz  
Line 267: Line 222:
=== Windows ===
=== Windows ===


As of this doc's writing; this is done via GPO installing to <code>&lt;mozilla-build&gt;/hg</code> as a ''blow-away-what-was-there'' install. This needs to be created for both builders (c.f. {{bug|1056981}}) and testers (c.f. {{bug|1110304}})
As of this doc's writing; this is done via GPO installing to <code>&lt;mozilla-build&gt;/hg</code> as a ''blow-away-what-was-there'' install (on hardware machines) and via updating AMIs with the new hg version (on AWS machines). This needs to be done for both builders (c.f. {{bug|1302375}}) and testers (c.f. {{bug|1302376}})


== Package Deployment ==
== Package Deployment ==
Line 284: Line 239:
  sudo su -
  sudo su -
  USER=yourusername
  USER=yourusername
  for i in `ls ~$USER/mercurial/mozilla-python27-mercurial-*.el6.{src,x86_64,i686}.rpm`;
  for i in $(ls /home/$USER/mercurial/mozilla-python27-mercurial-*.el6.{src,x86_64}.rpm);
   do
   do
     cp ./$i /data/repos/yum/releng/public/CentOS/6/x86_64/$i;  
     cp $i /data/repos/yum/custom/mozilla-mercurial/x86_64/;  
  done
  done
  for i in `ls ~$USER/mercurial/mozilla-python27-mercurial-*.el6.{src,i686}.rpm`;
  for i in $(ls /home/$USER/mercurial/mozilla-python27-mercurial-*.el6.{src,i686}.rpm);
   do
   do
     cp ./$i /data/repos/yum/releng/public/CentOS/6/i386/$i;  
     cp $i /data/repos/yum/custom/mozilla-mercurial/i386/;  
  done
  done
  pushd /data/repos/yum/releng/public/CentOS/6/i386
  pushd /data/repos/yum/releng/public/CentOS/6/i386
Line 298: Line 253:
  puppetmaster-fixperms
  puppetmaster-fixperms
  popd
  popd
* Update repo metadata
cd /data/repos/yum/custom/mozilla-mercurial/x86_64/; createrepo --update .
cd /data/repos/yum/custom/mozilla-mercurial/i386/; createrepo --update .


=== Ubuntu ===
=== Ubuntu ===
Line 311: Line 269:
  USER=yourusername
  USER=yourusername
  cd /data/repos/apt/custom/mozilla-mercurial
  cd /data/repos/apt/custom/mozilla-mercurial
  cp -v ~$USER/mercurial-apt/amd64/mozilla-python27-mercurial_*{_amd64.changes,_amd64.deb,.debian.tar.gz,.dsc,.orig.tar.gz} ./pool/precise/
  cp -v /home/$USER/mercurial-apt/mozilla-python27-mercurial_*{_amd64.changes,_amd64.deb,_i386.changes,_i386.deb,.debian.tar.gz,.dsc,.orig.tar.gz} ./pool/precise/
cp -v ~$USER/mercurial-apt/i386/mozilla-python27-mercurial_*{_i386.changes,_i386.deb,.debian.tar.gz,.dsc,.orig.tar.gz} ./pool/precise/
  ./update.sh
  ./update.sh
  puppetmaster-fixperms
  puppetmaster-fixperms
Line 331: Line 288:
  cp -v ~$USER/mercurial-DMGs/10.6/* 10.6/
  cp -v ~$USER/mercurial-DMGs/10.6/* 10.6/
  cp -v ~$USER/mercurial-DMGs/10.7/* 10.7/
  cp -v ~$USER/mercurial-DMGs/10.7/* 10.7/
cp -v ~$USER/mercurial-DMGs/10.8/* 10.8/
  cp -v ~$USER/mercurial-DMGs/10.10/* 10.10/
  cp -v ~$USER/mercurial-DMGs/10.10/* 10.10/
# 10.9 uses the same build as 10.8
cp -v ~$USER/mercurial-DMGs/10.8/* 10.9/
  puppetmaster-fixperms
  puppetmaster-fixperms


Line 347: Line 301:
* Update the mercurial versions in <code>&lt;path-to-local-puppet-repo&gt;modules/packages/manifests/mozilla/py27_mercurial.pp</code>
* Update the mercurial versions in <code>&lt;path-to-local-puppet-repo&gt;modules/packages/manifests/mozilla/py27_mercurial.pp</code>
** You can do this by hand too, be sure to change the versions specified
** You can do this by hand too, be sure to change the versions specified
  sed -i "s/3.2.1/3.2.2/" modules/packages/manifests/mozilla/py27_mercurial.pp
  sed -i "s/3.7.3/3.9.1/" modules/packages/manifests/mozilla/py27_mercurial.pp
* Update the repoflag to cause the system package repository cache to refresh
* Update the repoflag to cause the system package repository cache to refresh
** Inside modules/packages/manifests/setup.pp you want to increase the repoflag var everwhere
** Inside modules/packages/manifests/setup.pp you want to increase the repoflag var everwhere
Line 354: Line 308:
== Testing the patch ==
== Testing the patch ==


* On a puppetmaster you should apply the above created patch and run against one of each of the host types (centos builder both arch's, ubuntu testers both arch's, all vers of OSX)
* On a puppetmaster you should apply the above created patch and run against one of each of the host types (centos builder, ubuntu testers both arch's, all vers of OSX)


== Deploy the change ==
== Deploy the change ==
Aselagea
148

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1149406"

Navigation menu