SeaMonkey:Release Process

Note: This page is currently a work in progress, I'll try to fill in any missing item when we're doing the next SeaMonkey release (which should be 1.1.5 then).

This page is a collection of things currently done to do SeaMonkey 1.1.x releases, so that the process is documented in some way.

Tagging

Creating A New Release Tag

The first step for doing the actual release is to create a tag in CVS. I usually go into an empty directory on my local (Linux) machine and run the following bash script first (after adjusting the pull and release tag variables at the beginning of it):

#!/bin/bash

CVS_LOGIN=kairo%kairo.at
CVS_REPOS="-d :ext:$CVS_LOGIN@cvs.mozilla.org:/cvsroot"
PULLTAG=GECKO181_20071004_RELBRANCH
#PULLTAG=MOZILLA_1_8_BRANCH
CLIENTTAG=FIREFOX_2_0_0_8_RELEASE
MINIBRANCH=SEAMONKEY_1_1_5_MINIBRANCH
RELEASETAG=SEAMONKEY_1_1_5_RELEASE
CO_OPTS=MOZ_CO_DATE=\"`date +"%F %T"`\"
#CO_OPTS=

echo "cvs $CVS_REPOS checkout -r $PULLTAG $CO_OPTS  mozilla/client.mk"
echo "cd mozilla"
echo "make -f client.mk checkout $CO_OPTS MOZ_CO_PROJECT=suite"
echo "cvs $CVS_REPOS tag -b $MINIBRANCH client.mk"
echo "cvs $CVS_REPOS commit -r $MINIBRANCH -m 'initial minibranch commit' client.mk"
echo "mv client.mk ../client.mk.orig"
echo "sed 's/$CLIENTTAG/$RELEASETAG/' ../client.mk.orig > client.mk"
echo "rm ../client.mk.orig"
echo "cvs $CVS_REPOS commit -r $MINIBRANCH -m 'edit tags to pull from $RELEASETAG' client.mk"
echo "mv xpfe/bootstrap/version.txt ../version.txt.orig"
echo "sed 's/pre//' ../version.txt.orig > xpfe/bootstrap/version.txt"
echo "rm ../version.txt.orig"
echo "mv xpfe/bootstrap/module.ver ../module.ver.orig"
echo "sed 's/pre//' ../module.ver.orig > xpfe/bootstrap/module.ver"
echo "rm ../module.ver.orig"
echo "cvs $CVS_REPOS commit -r $PULLTAG -m 'remove \"pre\" postfix from SeaMonkey version on release branch' xpfe/bootstrap/module.ver xpfe/bootstrap/version.txt"
echo "cvs $CVS_REPOS tag -F $RELEASETAG client.mk"
echo "cvs $CVS_REPOS tag $RELEASETAG"

This outputs a list of commands to run for the tagging process, which I execute step by step, checking that the right thing is actually done. SeaMonkey usually pulls from a not-much-changing Gecko release branch near to a Firefox release tag (the CLIENTTAG which is set for pulling in that branch's client.mk), so we usually have a well-defined state to pull, but to make sure when pulling from that branch, I also set the CO_OPTS variable to use an exact date (automatically to the time of pulling in that script) so that the tree is 100% well-defined.

The commands pull a client.mk and use it to pull a complete suite (SeaMonkey) source checkout. We're only tagging files that are part of this checkout. For versions where building own builds with the old calendar extension was still supported, we also included calendar in that pull and in our tag as well as our source tarballs (see later), but nowadays the suite pull is enough.

The first commit to client.mk does not show up anywhere actually, so the message is useless, but I remember that CVS didn't correctly realize that it had the state of that minibranch and might choke afterwards. The mv/sed/rm stuff is to make the client.mk pull our new SeaMonkey release tag, the second commit to client.mk will actually be recorded in the file's CVS history and should tell what that tag change actually was.

We do similar things to module.ver and version.txt in the next steps to remove the "pre" postfix on the SeaMonkey version number, just that we can commit them directly to the Gecko release branch, as those are SeaMonkey-specific files anyways. Note: Ask for appoval on that checkin to the release branch BEFORE committing! (build folks may else get nervous, dveditz can approve)

The actual tagging is done with the last two commands, first the new tag is forced on the client.mk file, then the rest of the tree is tagged.

Usually, the next step from here is to create a first set of candidate builds.

Moving An Existing Release Tag

For later candidates, the tag is moved on any files that have changed compared to the older candidate.

To do this, first I update the files to the respective versions, most of the time this will be something like

cvs -d :ext:kairo%kairo.at@cvs.mozilla.org:/cvsroot checkout -r GECKO181_20071004_RELBRANCH MOZ_CO_DATE="2007-10-08 17:19:35" mozilla/updated/file1.cpp mozilla/updated/file2.h

Then, move the tag like this:

cvs -d :ext:kairo%kairo.at@cvs.mozilla.org:/cvsroot tag -F SEAMONKEY_1_1_5_RELEASE mozilla/updated/file1.cpp mozilla/updated/file2.h

With this, we can recreate candidate builds from the moved release tag.

Creating Builds

Once the cvs-mirror has updated the tagging changes, it's time to log into our release machines. Those machines on all the primary platforms have the latest bug 343891 patch applied so that we can create (stub) installers with the correct URLs in them right out of tinderbox.

As a first step, tinder-config.pl on all of those boxes must be corrected to pull the correct release tag and upload to directories with correct version numbers.

After logging into those boxes and doing this edit, I cd to the release-seamonkey-1.1 directory and execute "./tinderbox clobber restart" there, which does a BuildOnce clobber run producing the builds (for a respin on the same day as a previous one, the last-built files need to be deleted!). Note that only Linux can do this from the remote ssh shell, Mac is done via VNC from a terminal window and Windows via RDP from a cmd window, so all of them can perform basic tinderbox tests on the builds.

The Linux machine has additional stuff added to produce source tarballs. I have a script there that does

export MOZ_POSTFLIGHT_ALL="~/tinderbox/Makefile.tarball"

before starting the tinderbox run, and here's the Makefile.tarball file:

DEPTH		= $(MOZ_OBJDIR)
topsrcdir	= $(TOPSRCDIR)
srcdir		= $(TOPSRCDIR)
VPATH		= $(TOPSRCDIR)

include $(MOZ_OBJDIR)/config/autoconf.mk

include $(TOPSRCDIR)/config/rules.mk

include $(TOPSRCDIR)/toolkit/mozapps/installer/package-name.mk

SRC_TAR_BASE = $(MOZ_PKG_APPNAME)-$(MOZ_PKG_VERSION).source

postflight_all:
	tar --directory=$(TOPSRCDIR)/.. -cjf $(TOPSRCDIR)/$(MOZ_OBJDIR)/dist/$(SRC_TAR_BASE).tar.bz2 mozilla
	tar --directory=$(TOPSRCDIR)/.. -czf $(TOPSRCDIR)/$(MOZ_OBJDIR)/dist/$(SRC_TAR_BASE).tar.gz mozilla

This enables us to conveniently provide a source tarball from exactly the same state and upload it to stage together with the release builds.

All the boxes are configured to upload into a candidates-seamonkey1.1.5/ directory in my personal home dir on stage, while the ftp_url points to the final location on releases.mozilla.org. This enables me to both move that candidates dir to the SeaMonkey nightly dir on public FTP for testing as well as to later copy those builds directly to the release location without any further change, given this candidate is made final after QA.

Getting QA & Testing For The Builds

Publishing Candidates

As creating the builds pushes them right into a candidates-seamonkey1.1.5/ directory in my personal home dir on stage, all I need to technically do to publish the candidates is to move this directory to /pub/mozilla.org/seamonkey/nightly/ on stage, and they are available in [1] for testing by the community.

Notifying SeaMonkey Newsgroups

Following that, a post is made to mozilla.dev.apps.seamonkey and mozilla.support.seamonkey (with followup to the former) to announce the candidates and ask for testing, esp. for the smoketests to be run on all three platforms (pointing to the Litmus smoketest run for SeaMonkey 1.1).

Respinning Due To Problems Found By QA

If any major regressions or major problems are encountered during QA, patches need to go into the release branch (ask the Mozilla build team about that) or a minibranch, the tag needs to be moved, builds recreated and the candidates overwritten (!) with the new candidates, as well as a notification sent out to the newsgroups again.

The same process happens when Core/Gecko problems are encountered in Firefox testing and a further Firefox candidate is being released, so that we include those fixes in a moved tag and new candidates.

Notifying Localizers

If everything is looking good for release, mozilla.dev.l10n should be informed that the release is on its way, along with the planned release date, so that localizers can start preparing their localized contributed release builds for having those available in a timely fashion.

Doing The Actual Release

Preparing The Release Directory

When QA is running well, we can start preparing the actual release directory. For this, candidates-seamonkey1.1.5/ is copied to the personal home directory with a name of just 1.1.5/ and two subdirectories named contrib/ and contrib-localized/ are created. We need to care that those three directories are all group-writeable and have the set-guid flag set, the two former ones are owned by the seamonkey group, the latter by the l10n group.

All normal files, including all builds, should have read/write permissions for user and group and should be world-readable (but NOT world-writable), and only directories should have the executable bits set.

The README file for the release directory can be copied from the /pub/mozilla.org/seamonkey/releases/ subdirectory for the previous release, replacing the old version number with the new one (note that we have 1.1.x notation as well as tag names with 1_1_x notation in that file).

Additionally, a MD5SUMS file is generated for the main 1.1.5/ directory, I'm doing this with a simple script I have stored as ~/scripts/make-md5sums on stage:

#!/bin/bash

rm MD5SUMS
find -name "seamonkey-*" -maxdepth 1 -type f -print0 | xargs -0  md5sum >> MD5SUMS
chmod g+w MD5SUMS
chgrp seamonkey MD5SUMS

Preparing Release Notes

The person responsible for writing up release notes (currently Chris Thomas) has to be notified to generate those and put them up, so that they are in place once the new version get actually released.

Adding Contributed Builds

Once the release directory has been created, either for preparation in the home directory or already moved to the final release location, contributed builds can be added to its contrib*/ directories.

Builds for other platforms or with non-default build configurations need to go into contrib/ and come with a seamonkey-1.1.x.en-US.[platform-special].README file describing what the builds are and how they were built as well as who's responsible for those builds.

Localized builds go into contrib-localized/ and need to be just repackaged release builds or language packs which are signed off or contributed by the respective locale owners/peers listed in the localization teams list.

To avoid mirrors fetching partially uploaded builds, it's usually a good idea to pull the builds into the personal home directory first (I'm usually doing this via wget from the contributor's webspace), check their permissions and only then move them to the public releases/ directory.

Publishing Builds

TODO