Sheriffing/How To/Unified Repos: Difference between revisions

m (add missing *)
(add internal merge tool setup and xcode install as requirements for macOS)
 
(One intermediate revision by the same user not shown)
Line 8: Line 8:
* Install [https://www.mercurial-scm.org/downloads Mercurial 4.2] or higher
* Install [https://www.mercurial-scm.org/downloads Mercurial 4.2] or higher
* Make a copy of your .hgrc file for safekeeping and delete the original
* Make a copy of your .hgrc file for safekeeping and delete the original
* Use default hg tool to resolve code conflicts:
** Open .hgrc file in home folder.
** Search for line <code>[ui]</code>.
** Search for line starting with <code>merge =</code> between the line above and the next one starting with <code>[</code>.
** Add or update it to match <code>merge = internal:merge3</code>
* hg clone https://hg.mozilla.org/mozilla-central
* hg clone https://hg.mozilla.org/mozilla-central
* cd mozilla-central
* cd mozilla-central
* ./mach mercurial-setup
* ./mach mercurial-setup
* On macOS
** Install xcode which is be required for the execution of some commands: <code>xcode-select --install</code><br>This will open an install dialog which might be minimized. Check the list of open applications and confirm the installation.


Configure mercurial as desired. At a minimum, you need the following:
Configure mercurial as desired. At a minimum, you need the following:
Line 18: Line 25:


The following will pull all (or at least most) of the branches sheriffs need to have on hand:
The following will pull all (or at least most) of the branches sheriffs need to have on hand:
* hg pull inbound && hg pull autoland && hg pull beta && hg pull release && hg pull esr52
* hg pull autoland && hg pull beta && hg pull release && hg pull esr52
If you need other branches, you can find their names in the [https://hg.mozilla.org/hgcustom/version-control-tools/file/default/pylib/mozautomation/mozautomation/repository.py firefoxtree extension source].
If you need other branches, you can find their names in the [https://hg.mozilla.org/hgcustom/version-control-tools/file/default/pylib/mozautomation/mozautomation/repository.py firefoxtree extension source].


Line 29: Line 36:
To pull in newly landed changes, you can use one of the following commands:
To pull in newly landed changes, you can use one of the following commands:
* hg pull fxtrees # this will pull from all branches listed with the fxheads command
* hg pull fxtrees # this will pull from all branches listed with the fxheads command
* hg pull integration # this will pull from the mozilla-inbound and autoland branches
* hg pull treename # this will pull only from the tree with the name "treename"
* hg pull releases # this will pull from pretty much every release branch (FYI: including old, unused branches)


== Merges ==
== Merges ==
This assumes you have your unified repo all set up so that `hg fxheads` lists each of central, autoland, and inbound.
This assumes you have your unified repo all set up so that `hg fxheads` lists each of central and autoland.


To merge a specific non-tip <revision> from mozilla-inbound to mozilla-central:
To merge a specific non-tip <revision> from autoland to mozilla-central:
# hg pull central
# hg pull central
# hg pull inbound
# hg up central
# hg up central
# hg merge -r <revision>
# hg merge -r <revision>
# hg commit -m "Merge mozilla-inbound to mozilla-central a=merge"
# hg commit -m "Merge autoland to mozilla-central a=merge"
# hg push -r . central
# hg push -r . central


To merge mozilla-central into mozilla-inbound:
To merge mozilla-central into autoland:
# hg pull central
# hg pull central
# hg pull inbound
# hg pull autoland
# hg up inbound
# hg up autoand
# hg merge central
# hg merge central
# hg commit -m "Merge mozilla-central to mozilla-inbound a=merge"
# hg commit -m "Merge mozilla-central to autoland. CLOSED TREE"
# hg push -r . inbound
# hg push -r . autoland


* If the `hg merge central` command results in "abort: nothing to merge", you should instead use `hg update -r <mozilla-central's tip revision>` to do a non-merging update, then you can do `hg push -r . inbound` to push it as usual.
* If the `hg merge central` command results in "abort: nothing to merge", you should instead use `hg update -r <mozilla-central's tip revision>` to do a non-merging update, then you can do `hg push -r . autoland` to push it as usual.
 
== Checkin-neededs ==
This assumes we're checking things in to autoland. Replace autoland with another branch name as needed.
 
==== Single repository and single set of patches ====
Update autoland and prepare for the patches:
* hg pull autoland
* hg up autoland
* hg bookmark autoland-checkins
Import the patches:
* hg import https://link.to/correct/checkin-needed.patch
* Repeat ^ as needed for all autoland checkin-needed patches
Verify your outgoing changes:
* hg out -r . autoland
If you haven't lost a push race:
* hg push -r . autoland
If you lost a push race:
* hg pull autoland
* hg rebase -d autoland
* hg push -r . autoland
Delete your no longer needed bookmark:
* hg bookmark -d autoland-checkins
 
==== Multiple repositories with a single set of patches each ====
You can set up multiple sets of patches at once against multiple repositories:
Update autoland
* hg pull autoland && hg up autoland
Create a bookmark to base your autoland checkin-needed patches
* hg bookmark autoland-checkins
Import a bunch of patches against autoland
* hg import <patch1>
* hg import <patch2>
Update inbound and make a bookmark for inbound patches
* hg pull inbound && hg up inbound && hg bookmark inbound-checkins
Import some patches against inbound
* hg import <patch3>
* hg import <patch4>
See what you'd be pushing to inbound and push it
* hg out -r . inbound # this will only show patch3 and patch4
* hg push -r . inbound # this only pushes patch3 and patch4
* hg bookmark -d inbound-checkins # delete inbound's checkin bookmark as it's not needed
Go back to your autoland checkins and add another patch to the bookmark, then push it
* hg up autoland-checkins
* hg import <patch5>
* hg out -r . autoland # this will be patch1, patch2, and patch5
* hg push -r . autoland
* hg bookmark -d autoland-checkins
 
==== Single repository with multiple sets of patches ====
Or have multiple sets of patches against a single repository:
Update autoland
* hg pull autoland && hg up autoland
Make the first bookmark/set
* hg bookmark autoland-checkins-1
* hg import <patch1>
* hg import <patch2>
Make the second set
* hg up autoland && hg bookmark autoland-checkins-2
* hg import <patch3>
* hg import <patch4>
See what's going to be pushed in each
* hg out -r . autoland # patch3 and patch4
* hg up autoland-checkins-1 && hg out -r . autoland # patch1 and patch2
Push the first set
* hg push -r . autoland # pushes patch1 and patch2
Get ready to push the second set
* hg up autoland-checkins-2
* hg rebase -d autoland # Rebase the second set onto the now-pushed first set
And push it
* hg push -r . autoland # pushes patch3 and patch4
Delete the now-unneeded bookmarks
* hg bookmark -d autoland-checkins-1
* hg bookmark -d autoland-checkins-2


== Uplifts ==
== Uplifts ==
If branch-specific patches are posted, follow the checkin-needed instructions above, importing the patches onto the release branch.
If branch-specific patches are posted, follow import the patches onto the relevant branch.


If you're uplifting directly from an m-c checkin to aurora:
If you're uplifting directly from an mozilla-central checkin to mozilla-beta:


Pull and update to prepare for the uplifts:
Pull and update to prepare for the uplifts:
Line 177: Line 109:


== Rebasing after losing a push race ==
== Rebasing after losing a push race ==
This assumes you have commits you're attempting to push to mozilla-inbound when you lose a push race with someone. Change `inbound` to the correct branch name as needed.
This assumes you have commits you're attempting to push to autoland when you lose a push race with someone. Change `autoland` to the correct branch name as needed.


* hg pull inbound
* hg pull autoland
* hg rebase -d inbound
* hg rebase -d autoland
* hg push -r . inbound
* hg push -r . autoland


== Recovering from mistakes ==
== Recovering from mistakes ==
Line 194: Line 126:
You may also need to use some combination of the following to get things back to a known-good state:
You may also need to use some combination of the following to get things back to a known-good state:
* hg purge # http://mercurial.selenic.com/wiki/PurgeExtension
* hg purge # http://mercurial.selenic.com/wiki/PurgeExtension
* hg up -C
* hg up -C .


== See what you're about to push ==
== See what you're about to push ==
`hg out` without any flags added will show you a LOT of unrelated commits. To see just what you'll be pushing to a given branch, use `hg out -r . <branch>`
`hg out` without any flags added will show you a LOT of unrelated commits. To see just what you'll be pushing to a given branch, use `hg out -r . <branch>`
* hg out -r . inbound
* hg out -r . autoland
** This will only display the things you're about to push to the inbound branch
** This will only display the things you're about to push to the autoland branch


== View incoming changes ==
== View incoming changes ==
Line 232: Line 164:
  hg mergetointegration autoland
  hg mergetointegration autoland
This cuts down on a lot of repetitive typing, saving a bit of time.
This cuts down on a lot of repetitive typing, saving a bit of time.
=== rbimport ===
For merging from reviewboard patches, there's this alias
  rbimport = !hg transplant -e -s https://reviewboard-hg.mozilla.org/gecko -b $1
Then, it's matter of importing the right hash like the following command:
  hg rbimport dd1462ba321db983061b94e30cee92b6bd81e908

Latest revision as of 15:49, 12 July 2022

Unified Repos

Setting up the repo

  • Set up your ssh key yourself
  • Install Mercurial 4.2 or higher
  • Make a copy of your .hgrc file for safekeeping and delete the original
  • Use default hg tool to resolve code conflicts:
    • Open .hgrc file in home folder.
    • Search for line [ui].
    • Search for line starting with merge = between the line above and the next one starting with [.
    • Add or update it to match merge = internal:merge3
  • hg clone https://hg.mozilla.org/mozilla-central
  • cd mozilla-central
  • ./mach mercurial-setup
  • On macOS
    • Install xcode which is be required for the execution of some commands: xcode-select --install
      This will open an install dialog which might be minimized. Check the list of open applications and confirm the installation.

Configure mercurial as desired. At a minimum, you need the following:

  • Set your username and email.
  • Enable the firefoxtree extension.
  • Enable the histedit extension.

The following will pull all (or at least most) of the branches sheriffs need to have on hand:

  • hg pull autoland && hg pull beta && hg pull release && hg pull esr52

If you need other branches, you can find their names in the firefoxtree extension source.

You should now have everything set up for proper use!

If you want, you can rename the mozilla-central folder to something like "unified", since it is more than just mozilla-central at this point.

The `hg fxheads` command should list references to each of the relevant branch names, and the current revision and commit message for each.

To pull in newly landed changes, you can use one of the following commands:

  • hg pull fxtrees # this will pull from all branches listed with the fxheads command
  • hg pull treename # this will pull only from the tree with the name "treename"

Merges

This assumes you have your unified repo all set up so that `hg fxheads` lists each of central and autoland.

To merge a specific non-tip <revision> from autoland to mozilla-central:

  1. hg pull central
  2. hg up central
  3. hg merge -r <revision>
  4. hg commit -m "Merge autoland to mozilla-central a=merge"
  5. hg push -r . central

To merge mozilla-central into autoland:

  1. hg pull central
  2. hg pull autoland
  3. hg up autoand
  4. hg merge central
  5. hg commit -m "Merge mozilla-central to autoland. CLOSED TREE"
  6. hg push -r . autoland
  • If the `hg merge central` command results in "abort: nothing to merge", you should instead use `hg update -r <mozilla-central's tip revision>` to do a non-merging update, then you can do `hg push -r . autoland` to push it as usual.

Uplifts

If branch-specific patches are posted, follow import the patches onto the relevant branch.

If you're uplifting directly from an mozilla-central checkin to mozilla-beta:

Pull and update to prepare for the uplifts:

  • hg pull central
  • hg pull aurora
  • hg up aurora

Graft the mc commit and bring up the editor to edit the commit message, adding "a=foo" as needed:

  • hg graft --edit -r <revision>

Repeat the previous step as needed for all uplifts as needed. Verify the outgoing changes and push:

  • hg out -r . aurora
  • hg push -r . aurora

You can graft a range of commits at once if that's easier:

  • hg graft -r <toprevision>::<bottomrevision>

Read the documentation for graft for further help.

Backouts

Better tools are coming:


The standard backout command makes backing out revisions more difficult than it should, as it won't pre-fill the backout commit message with a bug number:

  • Copy backout <revision> from treeherder, note the bug <number>, and remember the <reason>
  • hg backout -r <revision>
    • This should open up your editor with a completely empty commit message. Type in the backout message like "Backout revision <revision> (bug <number>) for <reason>" (possibly with "CLOSED TREE" in there to get around a closure hook).
  • hg push -r . <tree>

qbackout can still be used, though you will want to use `hg oops` in place of `hg qbackout` (eg if you use `hg share` to get multiple working directories, qbackout and other uses of mq do not play nice with shared repos):

  • Copy the backout <revision> from treeherder
  • hg oops -e -r <revision>
    • This should open up your editor with a prepopulated commit message like "Backed out <revision> (bug <number>)". Add in the <reason> and possibly "CLOSED TREE" to get around a closure hook.
  • hg out -r . <tree>
    • to double-check what you will be pushing
  • hg push -r . <tree>

qbackout can also back out a range of commits in a single backout commit:

  • hg oops -e -s -r <toprevision>:<bottomrevision>
    • This will open your editor with a prepopulated commit message like "Backed out <toprevision>,<anymiddlerevisions>,<bottomrevision> (bug <number>,any other bug <numbers>)". Add in the <reason> and possibly "CLOSED TREE" to get around a closure hook.
  • hg push -r . <tree>

(Omit oops's -s flag to back out each individual revision in the range as a separate commit.)

Revert a backout

Rebasing after losing a push race

This assumes you have commits you're attempting to push to autoland when you lose a push race with someone. Change `autoland` to the correct branch name as needed.

  • hg pull autoland
  • hg rebase -d autoland
  • hg push -r . autoland

Recovering from mistakes

The histedit subcommand of hg lets you remove, edit and reorder changesets (in addition to other actions):

  • hg histedit

If a merge commit needs to be removed:

  • hg strip --no-backup -r .

WARNING: The hg strip command below removes any commits that haven't been pushed to the remote repositories, regardless of what branch/label/tag those commits are on. (So if you have local changes as patches for other issues - even based on other trees - they will be removed.) Only do this if you don't care about any of those commits!

  • hg strip --no-backup 'not public()'


You may also need to use some combination of the following to get things back to a known-good state:

See what you're about to push

`hg out` without any flags added will show you a LOT of unrelated commits. To see just what you'll be pushing to a given branch, use `hg out -r . <branch>`

  • hg out -r . autoland
    • This will only display the things you're about to push to the autoland branch

View incoming changes

Since your unified repo will likely have all changesets from the various branches pulled locally, `hg in <somebranch> -r <somerev>` will likely not help you see what would be pulled into your current branch from <somebranch> up to that <somerev>. Instead, you can use the following to print out the equivalent. Say you want to see what would be merged onto mozilla-central from autoland's revision ca142ec8ba0f:

Make sure m-c is fully up to date:

  • hg pull central
  • hg up central

Get all of autoland's changesets:

  • hg pull autoland

Do a preview of the merge without actually performing the merge:

  • hg merge -r ca142ec8ba0f -P

This will print out everything on autoland (up to and including revision ca142ec8ba0f) that is not already on mozilla-central.

See the log for a particular branch

  • hg log -fr [branch]

Extras

hg aliases

mergetocentral/mergetointegration

I set up the following aliases in my global .hgrc file to make merging things around trunk a little faster:

[alias]
# merge tree $1, revision $2 to central, optionally appending $3 (for "CLOSED TREE", etc)
mergetocentral = !$HG pull central ; $HG up central ; $HG pull $1 ; $HG merge $2 ; $HG commit -m "Merge $1 to central, a=merge $3" ; $HG push -r . central
# merge central tip to tree $1, optionally appending $2 (for "CLOSED TREE", etc)
mergetointegration = !$HG pull central ; $HG pull $1 ; $HG up $1 ; $HG merge central ; $HG commit -m "Merge m-c to $1, a=merge $2" ; $HG push -r . $1

With these set up, I can merge autoland's revision 04a3d9130aa0 over to mozilla-central with the following command:

hg mergetocentral autoland 04a3d9130aa0

and then I can merge mozilla-central's tip back to autoland with the following command:

hg mergetointegration autoland

This cuts down on a lot of repetitive typing, saving a bit of time.