Release:Release Automation on Mercurial:Troubleshooting: Difference between revisions

Jump to navigation Jump to search

Release:Release Automation on Mercurial:Troubleshooting (view source)

Revision as of 16:20, 20 June 2012

8,381 bytes removed ,  20 June 2012
no edit summary
No edit summary
No edit summary
Line 51: Line 51:
#* The [[Releases/Firefox_3.6b4/BuildNotes#updates|Build Notes for 3.6b4]] show an example of how this is done.
#* The [[Releases/Firefox_3.6b4/BuildNotes#updates|Build Notes for 3.6b4]] show an example of how this is done.
# Re-run the update verify builder from the waterfall.
# Re-run the update verify builder from the waterfall.
== Staging Specific Notes ==
Release automation in staging is mostly the same as in production, but does have a few differences you should know about:
* Use the config files with the prefix "<tt>staging_</tt>". These have many values already set correctly for staging.
* All uploading (builds, snippets, symbols) is done to dev-stage01.build.mozilla.org
* It can take a few tries to get repo_setup to run properly. This is because hgweb sometimes returns a 500 (internal server error) when we query about a locale. The best solution is just to start the automation from scratch until it works, to make sure you get a clean run. If this is too frustrating for you, you can manually clone the repositories you care about and start automation from tag (see [[#Restarting_the_automation_from_a_specific_point | Restarting from a specific point]]).
* Staging doesn't have a lot of slaves, and you may need to go around and stop running builds to get your automation run to happen in a reasonable period of time.
=== Staging specific preparation ===
==== Download the previous release ====
Because we point the staging releases at dev-stage01.build.mozilla.org the previous release must be downloaded to it. This is done by the "release_downloader" builder which is fired by a sendchange sent by release_sanity.py. It automatically removes any candidates or relase dir on dev-stage01 which already exists.
The builder could be disabled by setting releaseConfig['skip_release_download'] = True
===== Doing it by hand =====
If you need to download the previous release by hand for some reason you can use the following shell script to do so. Note that the variables are for the *previous* release, not the one you will be running:
# ffxbld @ dev-stage01
export VERSION=3.5.3
export BUILD=1
export PRODUCT=firefox
cd /home/ftp/pub/$PRODUCT/nightly
mkdir -p $VERSION-candidates/build$BUILD
cd $VERSION-candidates/build$BUILD
wget -r -np -nH --cut-dirs=6 -R index.html* -R *unsigned* http://stage.mozilla.org/pub/mozilla.org/$PRODUCT/nightly/$VERSION-candidates/build$BUILD/
cd /home/ftp/pub/$PRODUCT/releases
ln -s /home/ftp/pub/$PRODUCT/nightly/$VERSION-candidates/build$BUILD $VERSION
If you're doing a testrun with a limited number of locales you may delete any locales you don't care about after the above script finishes. (or you can add "-R *locale*", etc for each unwanted locale)
==== Buildbot configs ====
When working in staging you'll need to swap out the buildbot-configs, build/tools, buildbotcustom, and compare-locales repos for either stage-ffxbld ones, or your own. Here's an example of the changes you'll need:
<pre>
diff --git a/mozilla/config.py b/mozilla/config.py
--- a/mozilla/config.py
+++ b/mozilla/config.py
@@ -43,1 +43,1 @@ GLOBAL_VARS = {
-    'compare_locales_repo_path': 'build/compare-locales',
+    'compare_locales_repo_path': 'users/bhearsum_mozilla.com/compare-locales',
diff --git a/mozilla/staging_config.py b/mozilla/staging_config.py
--- a/mozilla/staging_config.py
+++ b/mozilla/staging_config.py
@@ -37,2 +37,2 @@ GLOBAL_VARS = {
-    'config_repo_path': 'build/buildbot-configs',
-    'buildbotcustom_repo_path': 'build/buildbotcustom',
+    'config_repo_path': 'users/bhearsum_mozilla.com/buildbot-configs',
+    'buildbotcustom_repo_path': 'users/bhearsum_mozilla.com/buildbotcustom',
diff --git a/mozilla/staging_release-firefox-mozilla-1.9.2.py b/mozilla/staging_release-firefox-mozilla-1.9.2.py
--- a/mozilla/staging_release-firefox-mozilla-1.9.2.py
+++ b/mozilla/staging_release-firefox-mozilla-1.9.2.py
@@ -106,1 +106,1 @@ releaseConfig['doPartnerRepacks']    = F
-releaseConfig['partnersRepoPath']    = 'users/stage-ffxbld/partner-repacks'
+releaseConfig['partnersRepoPath']    = 'users/armenzg_mozilla.com/partner-repacks'
@@ -132,1 +132,1 @@ releaseConfig['enable_repo_setup'] = Tru
-releaseConfig['build_tools_repo_path'] = "users/stage-ffxbld/tools"
+releaseConfig['build_tools_repo_path'] = "users/asasaki_mozilla.com/tools"
</pre>
As with production releases, you must tag these repositories with the _RELEASE tag prior to starting the release.
It is possible to use your own HG repository (long term release tests, parallel run, etc). Instead of running repo_setup builder, you can use the following script (run it on your laptop, using LDAP credentials):
<pre>
#!/bin/bash
for repo in compare-locales tools buildbotcustom buildbot buildbot-configs partner-repacks; do
    echo "deleting $repo"
    ssh hg.mozilla.org edit $repo delete YES
    echo "cloning $repo"
    ssh hg.mozilla.org clone $repo build/$repo
done
for l in `wget -q -O- http://hg.mozilla.org/mozilla-central/raw-file/tip/browser/locales/shipped-locales |grep -v en-US | awk '{print $1}'`; do
    echo "deleting locale $l"
    ssh hg.mozilla.org edit $l delete YES
    echo "cloning locale $l"
    ssh hg.mozilla.org clone $l l10n-central/$l
done
ssh hg.mozilla.org edit mozilla-central delete YES
ssh hg.mozilla.org clone mozilla-central mozilla-central
</pre>
=== Doing a test run with a limited number of locales ===
To run a test with a limited number of locales, do the following:
# Modify l10n-changesets to include whichever locales you want.
# Reconfig Buildbot and start the automation as normal
# Once the tag builder is finished and before the end of the first en-US build, clone the staging source repository (eg, http://hg.mozilla.org/users/stage-ffxbld/mozilla-1.9.1) do the following:
hg up -C GECKO191_20090115_RELBRANCH # replace the relbranch appropriately, of course
# modify browser/locales/shipped-locales to include the same locales as l10n-changesets
hg commit -m "Reduce number of locales for this test run"
hg tag -f FIREFOX_3_1b3_RELEASE # using the right version number in the tag
hg push ssh://hg.mozilla.org/users/stage-ffxbld/mozilla-1.9.1
If you don't want to clone the full repo you can do this on the slave that did the tagging.
=== How to sign in staging ===
==== Signing with the staging keys ====
When you're specifically testing something related to signing or doing a full end to end run it's best to sign the builds on the staging signing server. Doing so is very similar to signing production builds and fully documented [https://intranet.mozilla.org/Build:CombinedSigning#Signing_in_the_Staging_Environment in the CombinedSigning doc].
==== Faking it out ====
If you're not looking to test signing you can speed up your staging run a bit by shuffling files around so post-signing steps can find them. To do this, log onto dev-stage01.build.mozilla.org and do the following:
# ffxbld@dev-stage01
VERSION=3.5rc1
BUILD=1
cd /home/ftp/pub/firefox/nightly/${VERSION}-candidates/build${BUILD}
mkdir win32 update/win32
rsync -av --exclude=*.zip unsigned/win32/ win32/
rsync -av unsigned/update/win32/ update/win32/
rsync -av unsigned/win32_info.txt .
echo "faked" > win32_signing_build${BUILD}.log
We purposely make copies here rather than symlinking for a couple of reasons: L10n verify scripts barf when they get zip files (hence the --exclude above), 'updates' factory will blow away complete MARs upon upload if update/win32 is a symlink. The echo creates the log the automation looks for, in order to continue to l10n verify and updates.
=== Creating a CVS mirror for patcher and configs ===
If you need a new or modified patcher config, which shouldn't be checked into production CVS, then modify the following method:
WHO=yournamehere
# cltbld@dev-stage01
cd /builds/cvsmirrors
mkdir -p ${WHO}/cvsroot.clean/mozilla/tools/
rsync -av --exclude=CVSROOT/config --exclude=CVSROOT/loginfo cvs-mirror.mozilla.org::mozilla/CVSROOT /builds/cvsmirrors/${WHO}/cvsroot.clean/
rsync -av cvs-mirror.mozilla.org::mozilla/mozilla/tools/patcher-configs /builds/cvsmirrors/${WHO}/cvsroot.clean/mozilla/tools/
rsync -av cvs-mirror.mozilla.org::mozilla/mozilla/tools/patcher /builds/cvsmirrors/${WHO}/cvsroot.clean/mozilla/tools/
rsync -av cvs-mirror.mozilla.org::mozilla/mozilla/tools/release /builds/cvsmirrors/${WHO}/cvsroot.clean/mozilla/tools/
rsync -a --delete-after /builds/cvsmirrors/${WHO}/cvsroot{.clean,}/
To make changes check out using
cvs -d dev-stage01.build.mozilla.org:/builds/cvsmirrors/${WHO}/cvsroot co mozilla/
and specify a cvsroot of <tt>:ext:cltbld@dev-stage01.build.sjc1.mozilla.com:/builds/cvsmirrors/${WHO}/cvsroot</tt> in the config for the release automation.
Bhearsum
canmove, Confirmed users
6,441

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/443629"

Navigation menu