Software Update:GeneratingSnippetsAndPartialsForReleases

For releases of Firefox and Thunderbird, the Build & Release team at Mozilla Corp uses a program called patcher2. This document describes the basics of how to use it.

Overview

Parts of the update system

To provide updates for releases, you need the following

• for each platform and locale:
  • a partial mar between the current release and the new release, this basically an archive of binary diff's
  • a complete mar for the new release, which is a copy of every file, to be used if the partial doesn't work or the installed build is more than one release behind
• a collection of update information (called snippets) that describe the update paths

The tinderbox produces the complete mars when it creates the installer/archiver. The partial mar and snippets are created by a perl script called patcher2.

How Firefox/Thunderbird updates are tested

We use four update channels for Firefox & Thunderbird releases. The two channels that end users see are

• beta - default channel for the releases 2.0a1, 2.0b1, and up to the last 2.0 beta
• release - default channel for the first release candidate for 2.0. The vast majority of people are on this channel.

Then there are

• betatest - for testing updates, pointing to the ftp server, to check that they apply properly, the UI works etc etc
• releasetest - testing of staging files on ftp server, pointing to download.m.o (bouncer), to make sure everything is available to end users when the release is imminent

To make this possible two groups of update snippets are generated, and pushed live at different times. A correctly setup patcher config will make both sets in one run, as well as creating the partial mars.

You can modify the channel using the Update Channel Changer extension, or edit <firefox_install_dir>/default/pref/channel-prefs.js to change the app.update.channel preference. Using about:config doesn't work.

Creating updates

Define variables

PRODUCT=firefox
VERSION=2.0.0.7

Getting the code

# create a working dir
mkdir $PRODUCT-$VERSION-updates/
cd $PRODUCT-$VERSION-updates/

export CVSROOT=:pserver:anonymous@cvs-mirror.mozilla.org

# pull patcher2 and the MozBuild support library
cvs co -d patcher mozilla/tools/patcher
cd patcher
cvs co -d MozBuild mozilla/tools/release/MozBuild

Create/update config

Here's a sample, which shows three Firefox 2.0.0.x releases, with 2.0.0.7 upcoming.

<app>
    <Firefox>
        <current-update>
            from   2.0.0.6
            to   2.0.0.7
            channel   beta release
            testchannel   betatest releasetest
            details   http://%locale%.www.mozilla.com/%locale%/firefox/%version%/releasenotes/
            <partial>
                path   firefox/nightly/2.0.0.7-candidates/rc2/firefox-2.0.0.6-2.0.0.7.%locale%.%platform%.partial.mar
                url   http://download.mozilla.org/?product=firefox-2.0.0.7-partial-2.0.0.6&os=%bouncer-platform%&lang=%locale%
                betatest-url   http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/2.0.0.7-candidates/rc2/firefox-2.0.0.6-2.0.0.7.%locale%.%platform%.partial.mar
            </partial>
            <complete>
                path   firefox/nightly/2.0.0.7-candidates/rc2/firefox-2.0.0.7.%locale%.%platform%.complete.mar
                url   http://download.mozilla.org/?product=firefox-2.0.0.7-complete&os=%bouncer-platform%&lang=%locale%
                betatest-url   http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/2.0.0.7-candidates/rc2/firefox-2.0.0.7.%locale%.%platform%.complete.mar
            </complete>
        </current-update>

        past-update   2.0.0.5 2.0.0.6 betatest releasetest beta release

        <release>
            <2.0.0.5>
                version   2.0.0.5
                extension-version   2.0.0.5
                completemarurl http://stage.mozilla.org/pub/mozilla.org/firefox/releases/2.0.0.5/update/%platform%/%locale%/firefox-2.0.0.5.complete.mar
                schema   1
                <platforms>
                    linux-i686   2007071317
                    mac   2007071317
                    win32   2007071317
                </platforms>
                locales   af ar be bg ca cs da de el en-GB en-US es-AR es-ES eu fi fr fy-NL ga-IE gu-IN he hu it ja ja-JP-mac ka ko ku lt mk mn nb-NO nl nn-NO pa-IN pl pt-BR pt-PT ro ru sk sl sv-SE tr zh-CN zh-TW
                <exceptions>
                    gu-IN   linux-i686, win32
                    ja   linux-i686, win32
                    ja-JP-mac   mac
                    ka   linux-i686, win32
                </exceptions>
            </2.0.0.5>
            <2.0.0.6>
                version   2.0.0.6
                extension-version   2.0.0.6
                completemarurl   http://stage.mozilla.org/pub/mozilla.org/firefox/releases/2.0.0.6/update/%platform%/%locale%/firefox-2.0.0.6.complete.mar
                schema   1
                <platforms>
                    linux-i686   2007072517
                    mac   2007072517
                    win32   2007072518
                </platforms>
                locales   af ar be bg ca cs da de el en-GB en-US es-AR es-ES eu fi fr fy-NL ga-IE gu-IN he hu it ja ja-JP-mac ka ko ku lt mk mn nb-NO nl nn-NO pa-IN pl pt-BR pt-PT ro ru sk sl sv-SE tr zh-CN zh-TW
                <exceptions>
                    gu-IN   linux-i686, win32
                    ja   linux-i686, win32
                    ja-JP-mac   mac
                    ka   linux-i686, win32
                </exceptions>
            </2.0.0.6>
            <2.0.0.7>
                version   2.0.0.7
                extension-version   2.0.0.7
                completemarurl   http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/2.0.0.7-candidates/rc2/firefox-2.0.0.7.%locale%.%platform%.complete.mar
                schema   1
                <platforms>
                    linux-i686   2007091417
                    mac   2007091417
                    win32   2007091417
                </platforms>
                locales   af ar be bg ca cs da de el en-GB en-US es-AR es-ES eu fi fr fy-NL ga-IE gu-IN he hu it ja ja-JP-mac ka ko ku lt mk mn nb-NO nl nn-NO pa-IN pl pt-BR pt-PT ro ru sk sl sv-SE tr zh-CN zh-TW
                <exceptions>
                    gu-IN   linux-i686, win32
                    ja   linux-i686, win32
                    ja-JP-mac   mac
                    ka   linux-i686, win32
                </exceptions>
            </2.0.0.7>
        </release>
    </Firefox>
</app>

Run patcher2

# build tools
./patcher2.pl --build-tools --app=${PRODUCT} \
  --config=patcher2.cfg 2>&1 | tee  ${PRODUCT}_build-tools.log
# download complete MARs
./patcher2.pl --download --app=${PRODUCT} \
 --config=patcher2.cfg 2>&1 | tee  ${PRODUCT}-download.log
# Create partial patches and snippets
./patcher2.pl --create-patches --app=${PRODUCT} \
 --config=patcher2.cfg 2>&1 | tee ${PRODUCT}-create-patches.log