Jetpack/Development Process

< Jetpack
Revision as of 06:45, 12 September 2011 by MykMelez (talk | contribs) (clarify what it means to regress tests and change interfaces)

Goals

  • ship frequently on a regular schedule
  • maintain compatibility with new versions of Firefox
  • minimize the number of branches and channels

Non-goals

  • coordinate dependencies between products such that
    • the SDK ships APIs for new Firefox features before the features ship in Firefox
    • Firefox ships support for new SDK features at the same time the features ship in the SDK

Process

We ship a stable SDK release on a regular schedule every six weeks. Each release undergoes six weeks of development and six of stabilization, so a complete cycle lasts twelve weeks, and release cycles overlap by six weeks (while one release is stabilized, the next one is developed).

We commit code to three branches: a development branch (dev), a stabilization branch (stab), and a release branch (rel). Each release spends six weeks on each branch, starting with six weeks of open development on dev, after which it is merged to stab; continuing with six weeks of restricted stabilization on stab, after which it is merged to rel; and concluding with six weeks of statis on rel.

We distribute code to three distribution channels: an unstable development channel (dev), a semi-stable test channel (test), and a stable release channel (rel). The dev channel is the code repository itself; we don't produce dev builds. The test channel is the project discussion group; we produce alpha, beta, release candidate (candidate), and final release (final) builds at least once per week and announce them in the group. The rel channel is the Add-ons Blog (and other communications, as appropriate); we announce final builds in the blog.

We don't provide automated updates to newer releases, but we may do so in the future.

Feature development happens on the development branch. It is subject to quality controls (reviews, unit tests, etc.). We may introduce additional quality controls in the near future (f.e. a requirement that new features and enhancements initially be marked experimental and be isolated from non-experimental use of the product).

Feature stabilization takes place on the stabilization branch. We merge to stab right after we ship the final build for the previous release. The stabilization period lasts six weeks and comprises three two week periods: alpha, beta, and candidate. During the alpha period, we address beta blockers, raising quality to the beta level. During the beta period, we address release blockers, raising quality to the release level. During the candidate period, we bake and prepare the release.

We achieve quality goals by fixing bugs and cutting features.

Releases have versions of the format major.minor[.fix], and we communicate them to users. We typically increment the minor number for each release. We add a "fix" number and ship a release with it only under exceptional circumstances (f.e. an urgent security issue).

The branches look something like this:

 

SDK releases precede the Firefox releases they accompany by enough time for us to repackage AMO-hosted addons with the SDK release but not by too much time to prevent us from tracking and addressing compatibility issues.

Stab maintains compatibility with its accompanying Firefox release (i.e. the one on Firefox's beta branch when the SDK release ships). Dev maintains compatibility with the next two Firefox releases (the ones on Firefox's aurora and central branches when dev merges to stab).

The next SDK release after 1.0 is 1.1, and it accompanies Firefox 7 (skipping 6). Its schedule is slightly irregular to accommodate that target. If SDK 1.0 is found to be incompatible with Firefox 6, we address it in a 1.0.1 release branched from the 1.0 tag.

The schedule for the next few releases (and their accompanying Firefox releases) looks like this:

 

When What Can Land

  THIS PAGE IS A WORKING DRAFT  
The page may be difficult to navigate, and some information on its subject might be incomplete and/or evolving rapidly.
If you have any questions or ideas, please add them as a new topic on the discussion page.

In general, new features and enhancements to existing features can land only on the development branch, while bug fixes can land on the development branch, the stabilization branch, or both.

New docs and docs enhancements that do not involve code changes, however, can land on the stabilization branch during the first two weeks of each stabilization period (i.e. while the code is in alpha), so tech writers have time to document late-breaking changes, and because such changes are less likely to cause regressions.

Landing Requirements

  THIS PAGE IS A WORKING DRAFT  
The page may be difficult to navigate, and some information on its subject might be incomplete and/or evolving rapidly.
If you have any questions or ideas, please add them as a new topic on the discussion page.

In order to land, a change must provide a notable benefit and not unintentionally change existing functionality (f.e. busting tests). In most cases, it must also pass a code review by an SDK code reviewer. And if it changes or adds a user-facing interface, including APIs implemented as CommonJS modules and command-line tools/flags, it must also pass an API/interface review by the technical lead.

Exceptions to the code review requirement include:

  • release management changes (f.e. version number bumps)
  • additions to the credits
  • obvious and trivial test bustage fixes
  • obvious and trivial typo (spelling, grammar) fixes in documentation

TBD

  • where stabilization fixes land (on dev, then cherry-pick to stab, or on stab, then merge to dev?)
  • whether stab is a single branch or a series of them, one per release (if stab is a single branch, it may be easier to integrate with test automation; if each release has its own stab branch, we don't have to cut another branch to take a blocker fix after we release the candidate build and merge dev to stab for the next release)
  • how many weeks to precede each Firefox release (two? three?)
  • conditions under which to increment the major number
  • how to identify, and when to increment, the version in manifests, code, and docs on the two branches
  • when to drop compatibility with older releases of Firefox

Rationale

Shipping frequently on a regular schedule, i.e. quality/date-driven or "train model" releases, gets improvements into users' hands faster and promotes developer productivity and code quality by reducing the stress and tunnel vision associated with its principal alternative (i.e. shipping infrequently and irregularly via quality/feature-driven releases).

Maintaining compatibility with new versions of Firefox is a key goal for the project to reduce the compatibility burden on addon developers and users alike, and we need be able to ship as often as Firefox to achieve it, since any Firefox release can contain a compatibility issue.

We don't provide APIs for new Firefox features before they ship in Firefox because it is difficult to align the two products' schedules to accommodate landing such APIs. However, we do want to release those APIs as soon as possible and will look for ways to make it happen (f.e. by making it possible to land such APIs into core Firefox alongside the features they expose).

Having two active (three total) branches keeps developers productive by avoiding freezes and other branch controls that prevent developers from integrating continuously on a central branch. Firefox uses three active (four total) branches (and thus four channels) for this reason, but our team is too small to justify the branch management burden of an additional branch, and our userbase is too small to fragment across four channels.

Releases every six weeks allow us to ship at least as often as Firefox, while twelve week cycles with six week overlaps let us work on just two active branches.

Two active branches instead of three mean we have to maintain compatibility with two versions of Firefox on the dev branch, but that is not an undue burden, because we already intend to maintain compatibility with older versions of Firefox (although it is unclear how far back we'll go).

We communicate versions, despite our rapid release schedule, because our developer audience understands them.

Our ratio of development to stabilization time (6/6) is significantly larger than Firefox's (6/12) but justified by our ability to isolate features (marking APIs/commands/options experimental, segregating functionality into separate modules/tools that must be explicitly invoked) and willingness to back stuff out.

(An alternative plan of four weeks of development, four weeks of alpha->beta stabilization, and four weeks of beta->release stabilization, with the alpha->beta stabilization split across the branches, would give us a 4/8 ratio, but it would mean branch controls for the last two weeks of each cycle on dev.)

References