When a patch is landed that causes build or test failures, it must be backed out to restore the tree to a passing state again. This page documents the procedure for performing backouts both for Gecko changes and Gaia.

New recommended way of backouts

With unified repos backouts are now more easily - please follow Unified Repos Backup

Gecko Backouts

Prerequisites

Install the qbackout extension from https://hg.mozilla.org/hgcustom/version-control-tools/file/default/hgext/qbackout
Instructions: https://hg.mozilla.org/hgcustom/version-control-tools/file/default/hgext/qbackout/README

Usage

Backout of a single changeset

hg qbackout -e -r 123456789abc

Where -r is the specific revision to be backed out. The -e option opens an editor to edit the commit message prior to committing. This is useful for adding DONTBUILD, CLOSED TREE, etc before pushing.

Backout of multiple consecutive changesets

hg qbackout -e -s -r 123456789abc:cba987654321

Where the -r is now specifying a range of commits (from 123456789abc to cba987654321). The -s option commits the backouts as a single commit. Otherwise, each changeset in the range will be backed out as separate commits.

Backout of multiple non-consecutive changesets

hg qbackout -e -s -r 123456789abc+cba987654321

Useful when there are commits in between the ones that need backing out.

Backout of a combination of consecutive and non-consecutive changesets

hg qbackout -e -s -r 123456789abc:cba987654321+xyz456789123

Useful when a follow-up patch needs backing out in addition to the main one (i.e. someone had a follow-up bustage fix push) or an additional commit conflicts with the backout and needs backing out as well.

Pushing

hg qfinish -a
hg push

Finalizes the patch for pushing to the remote repo.

What Ifs

If the remote repo has additional changesets (i.e. a push race), hg pull --rebase will pull the new changesets and rebase the local changes on top of them.
- This requires the rebase extension to be enabled in .hgrc
If the commit message needs to be modifying after qfinish (i.e. forgotten CLOSED TREE), hg commit --amend will open the backout in an editor for modification.

Best Practices and Communication

Based on Feedback from the Sheriff Survey, when you have to do a backout some best practice

Ping the Developer in IRC if possible and when the developer is around, this gives the developer a chance to fix the problem and so avoid a backout. Some of the sheriffs use a 5 minute rule for the waiting of a response of the developer
Set Need-info! This is very important! Developers get a lot of bugmail and so a backout comment can be missed - please make sure that you set a need-info to the developer in the bug when doing a backout
Make sure that the failures that lead to the backout are stared in treeherder - this helps a lot also the following sheriffs.

Sheriffing/How To/Backouts

Contents