ReleaseEngineering/How To/Generate partial updates: Difference between revisions

Jump to navigation Jump to search

ReleaseEngineering/How To/Generate partial updates (view source)

Revision as of 03:51, 12 July 2012

4,892 bytes added ,  12 July 2012
no edit summary
No edit summary
Line 6: Line 6:


Assumptions:
Assumptions:
* The automated updates builder has already run, and you're re-using the slave for the copy of the utilities and scripts you need
* the automated updates builder has already run, and you're re-using the slave for the copy of the utilities and scripts you need
* the request is for the release channel
* the request is for the release channel
* all locales are being generated, possibly only some of the platforms
For the purposes of this doc we'll say the update will be between $OLDV and $NEWV build$BUILD.


= Config files =  
= Config files =  
== Patcher config ==
== Patcher config ==
To get setup
To get setup
* Grab a copy of the release config, mozRelease-branch-patcher2.cfg, from your own CVS checkout or from [http://mxr.mozilla.org/mozilla/source/tools/patcher-configs/mozRelease-branch-patcher2.cfg?raw=1 mxr]
* grab a copy of the release config, mozRelease-branch-patcher2.cfg, from your own CVS checkout or from [http://mxr.mozilla.org/mozilla/source/tools/patcher-configs/mozRelease-branch-patcher2.cfg?raw=1 mxr]
* Rename it to mozRelease-branch-patcher2-XXXXXX.cfg, where XXXXXX is the bug number for the request
* verify that the file is up to date by checking the values of <tt>from</tt> and <tt>to</tt> in <tt><current-update></tt>, and the right buildN in the <tt>completemarurl</tt> of the <tt><releases></tt> block for the release being created
* Verify that the file is up to date by checking the values of <tt>from</tt> and <tt>to</tt> in <tt><current-update></tt>, and the right buildN in the <tt>completemarurl</tt> of the <tt><releases></tt> block for the release being created
* rename it to mozRelease-branch-patcher2-XXXXXX.cfg, where XXXXXX is the bug number for the request


To modify the config
To modify the config
* in the <tt><current-update></tt> block
* in the <tt><current-update></tt> block
** change the <tt>from</tt> version
** change the <tt>from</tt> version to $OLDV
** in the <tt><partial></tt> block change the from version in the paths
** in the <tt><partial></tt> block change the from version in the paths to $OLDV
* remove all the past-update lines just below <tt><current-update></tt> block
* remove all the past-update lines just below <tt><current-update></tt> block
* remove the unneeded blocks from <tt><release></tt>
* remove the unneeded blocks from <tt><release></tt> (keep only $OLDV and $NEWV, for brevity)
* if only some platforms are needed (eg windows), remove the other platforms from the <tt><platforms></tt> and <tt><exceptions</tt> blocks
* if only some platforms are needed (eg windows), remove the other platforms from the <tt><platforms></tt> and <tt><exceptions</tt> blocks


Line 28: Line 31:
== Update verify configs ==
== Update verify configs ==
To get setup
To get setup
* Grab a copy of the configs, either from your tools checkout or directly from hg.m.o ([http://hg.mozilla.org/build/tools/raw-file/default/release/updates/mozRelease-firefox-win32.cfg windows], [http://hg.mozilla.org/build/tools/raw-file/default/release/updates/mozRelease-firefox-mac64.cfg mac])
* grab a copy of the configs, either from your tools checkout or directly from hg.m.o ([http://hg.mozilla.org/build/tools/raw-file/default/release/updates/mozRelease-firefox-win32.cfg windows], [http://hg.mozilla.org/build/tools/raw-file/default/release/updates/mozRelease-firefox-mac64.cfg mac])
* verify the configs are up to date by checking the current release is referenced in the 2nd line, and that the to definition there refers to the right buildN path
* rename to mozRelease-firefox-win32-XXXXXX.cfg, where XXXXX is the bug number for the request
* rename to mozRelease-firefox-win32-XXXXXX.cfg, where XXXXX is the bug number for the request


To modify the config:
To modify the configs:
* remove all the lines for versions older and newer than the one you need to test
* remove all the lines that aren't for $OLDV and the current shipped release
* in $OLDV's full check line
** add the locales from $OLDV's quick check line
** add the patch_types, aus_server, ftp_server, and to definitions from the current release
* remove the $OLDV quick check comment and config line
* remove current release lines
* repeat for each platform
 
Attach the files to the bug as a patch (eg via mq), request review. You'll download it again later.
 
= Generate the partials =
Initial setup (~15 min)
cd /builds/slave/rel-m-rel-updates/build
mv temp/ temp.automated_run/
curl -sL https://bugzilla.mozilla.org/attachment.cgi?id=YYYYYY \
  > patcher-configs/mozRelease-branch-patcher2-XXXXXX.cfg
# where YYYYYY is the attachment number for the patcher config
#  and XXXXXX is the bug number
 
# get source and build tools - a checksum based run doesn't have these
# NB: not UPDATE_PACKAGING_R16, we need a more recent version of mar
# use $NEWV's tag, eg FIREFOX_13_0_1_RELEASE
export HGROOT=http://hg.mozilla.org/releases/mozilla-release
perl patcher2.pl --build-tools-hg --tools-revision=FIREFOX_VER_RELEASE \
  --app=firefox --brand=Firefox \
  --config=patcher-configs/mozRelease-branch-patcher2-XXXXXX.cfg
 
Get the complete mar files (~45 mins for windows & mac)
perl patcher2.pl --download \
  --app=firefox --brand=Firefox \
  --config=patcher-configs/mozRelease-branch-patcher2-XXXXXX.cfg
 
Generate unsigned partial updates (90 mins??)
perl patcher2.pl --create-patches --partial-patchlist-file=patchlist.cfg \
  --app=firefox --brand=Firefox \
  --config=patcher-configs/mozRelease-branch-patcher2-XXXXXX.cfg
 
Make a backup of the new partials/snippets, destroy snippets (< 5 min)
rsync -a temp/firefox/$OLDV-$NEWV{,-unsigned}/
rm -rf temp/firefox/$OLDV-$NEWV/aus2*
 
Sign the mar files (< 15 min ??)
# get a token
# look up the values for --user in the release-signing part of passwords.py, on the master
unset HISTFILE
IP=`/sbin/ifconfig eth0 | grep "inet addr" | awk -F: '{print $2}' | awk '{print $1}'`
ssh -oBatchMode=no cltbld@buildbot-master12.build.mozilla.org \
  'curl --fail -k -Fslave_ip=$IP -Fduration=3600 --user reda:cted \
    https://signing1.build.scl1.mozilla.com:9120/token' \
      > /builds/slave/rel-m-rel-updates/token
rm /builds/slave/rel-m-rel-updates/nonce
# update the channel id and version, then sign
for f in `find temp/firefox/$OLDV-$NEWV/ftp -type f -name *partial.mar | sort`; do
  echo Signing $f
  mozilla/obj/dist/host/bin/mar -H firefox-mozilla-release -V $NEWV -i $f;
  /tools/python-2.6.5/bin/python \
    /builds/slave/rel-m-rel-updates/tools/release/signing/signtool.py \
    -t /builds/slave/rel-m-rel-updates/token -n /builds/slave/rel-m-rel-updates/nonce \
    -c /builds/slave/rel-m-rel-updates/tools/release/signing/host.cert \
    -H signing1.build.scl1.mozilla.com:9120 -H signing2.build.scl1.mozilla.com:9120 -f gpg -f mar \
    "$f"
done
 
Regenerate the snippets (~ 1 min)
perl patcher2.pl --create-patchinfo \
  --app=firefox --brand=Firefox \
  --config=patcher-configs/mozRelease-branch-patcher2-XXXXXX.cfg
 
Snippet verification (< 1 min)
# release==releasetest ?
cd temp/firefox/$OLDV-$NEWV
bash /builds/slave/rel-m-rel-updates/tools/release/compare-channel-snippets.sh \
  aus2 release aus2.test  releasetest
 
Tidy and upload mar files (~5 min ??)
find ftp/firefox/nightly/$NEWV-candidates/build$BUILD -type f -exec chmod 644 {} \;
find ftp/firefox/nightly/$NEWV-candidates/build$BUILD -type d -exec chmod 755 {} \;
# $BUILD is the build number, eg 1, 2, ...
rsync -nav -e 'ssh -oIdentityFile=~/.ssh/ffxbld_dsa' '--exclude=*complete.mar' \
  ftp/firefox/nightly/$NEWV-candidates/build$BUILD/update \
  ffxbld@stage.mozilla.org:/pub/mozilla.org/firefox/nightly/$NEWV-candidates/build$BUILD/
 
Upload snippets (~ 1 min)
rsync -av -e 'ssh -oIdentityFile=~/.ssh/auspush' --exclude=complete.txt \
  aus2.test/ \
  ffxbld@aus3-staging.mozilla.org:/opt/aus2/snippets/staging/Firefox-$NEWV-build$BUILD-$OLDV-partial-test
 
rsync -av -e 'ssh -oIdentityFile=~/.ssh/auspush' --exclude=complete.txt \
  aus2/ \
  ffxbld@aus3-staging.mozilla.org:/opt/aus2/snippets/staging/Firefox-$NEWV-build$BUILD-$OLDV-partial
 
Backupsnip/pushsnip the test snippets
ssh -oIdentityFile=~/.ssh/auspush ffxbld@aus3-staging.mozilla.org \
  bin/backupsnip Firefox-$NEWV-build$BUILD-$OLDV-partial-test
ssh -oIdentityFile=~/.ssh/auspush ffxbld@aus3-staging.mozilla.org \
  bin/pushsnip  Firefox-$NEWV-build$BUILD-$OLDV-partial-test
 
Announce to r-d so QA can test/include in testing.
NThomas
Confirmed users
4,293

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/449978"

Navigation menu