From MozillaWiki
Jump to: navigation, search

Mercurial DVCS Service

Overview is our primary Mercurial service access point. This hosts all Firefox related code, including code and configurations for the continuous integration farm. Other names for services provided by Mercurial include "try server".

The service is provided by a clusters: one cluster for commits (ssh protocol only), and one for serving repositories and related data over http(s). These are called the "push heads" and "web heads" respectively below.

We are continually making improvements to the service, so please use this as the launch page for any questions or service requests.


The authoritative source for current HTTPS certificate & SSH host keys is in VCS. The information below is provided as an alternate source.

The hg services are only provided via the host name For HTTPS access, the certificate will only be valid for that hostname.

For SSH access, one of the following fingerprints are presented, depending on the client version of SSH:

   56 SHA256:7MBAdqLe8+aSYkv+5/2LUUxd+WdgYcVSV+ZQVEKA7jA (ED25519)
   256 SHA1:Ft++OU96cvaREKNFCJ6AiuCpGac (ED25519)
   256 MD5:96:eb:3b:78:f5:ca:19:e2:0c:a0:95:ea:04:28:7d:26 (ED25519)
   4096 SHA256:RX2OK8A1KNWdxyu6ibIPeEGLBzc5vyQW/wd7RKjBehc (RSA)
   4096 SHA1:p2MGe4wSw8ZnQ5J9ShBk/6VA+Co (RSA)
   4096 MD5:1c:f9:cf:76:de:b8:46:d6:5a:a3:00:8d:3b:0c:53:77 (RSA)


Status of the web heads is most easily seen on this dashboard

Try status can be seen in try push queue image. Note that it is normal to see spikes to approximately 12-13, as the webhead synchronization process runs after every successful push. Long term values greater than 2 are suspicious.

Other status of the push heads requires access to graphite . The sheriffs in #releng can provide information if needed, see troubleshooting for further info.

Troubleshooting and Problem Reporting

Note: see also the references section for tips and tools to avoid issues.

If you suspect issues, please follow the steps in order until you get:

  1. Check status
  2. For Firefox related work, ask the sheriffs in #releng or #developers
  3. For all other work, ask in #moc
  4. File a bug under Developer Services :: Mercurial:
    • please include an accurate time (with time zone)
    • please be specific about HTTP status codes and command output, as it may shed light on what component failed and why
  5. If urgent and important, escalate bug to #moc (they escalate to DeveloperServices as needed)

Recent Changes

Mercurial Version

The goal is to upgrade to the latest point release (x.y.z; z>0) shortly after they are released. Point upgrades will be announced on the [list].

The initial version of major quarterly Mercurial releases (versions x.y) will not be be deployed to production due to an overabundance of caution. While there are rarely significant bugs or regressions that impact servers, we prefer to not take a risk.

Major quarterly Mercurial releases will be announced on the list in order to raise visibility and to give people time to upgrade code, etc.

Version Date Bug
3.5.1 2015-09-15 bug 1200769 - planned major version release (push nodes only)
3.4.1 2015-06-03 bug 1169656 - planned major version release
3.3.3 2015-04-03 Minor version bump to get performance fixes for annotate on hgweb
3.3.2 2015-03-11 bug 1138680 - planned major version release
3.2.3 2014-12-18 bug 1113362 - off cycle security push
3.1.2 2014-10-21 bug 1090345

Other Changes

Bugs fixed in last 2 weeks:

Full Query
ID Priority Summary
1768512 -- Please create comm-esr102 repository

1 Total; 1 Open (100%); 0 Resolved (0%); 0 Verified (0%);

Requesting Services


Mozilla has used Mercurial for quite a while, so some of the reasons for the current configuration are historical, and (more importantly) separated from the business needs that drove them. These are documented loosely on "/Thanks for asking". Please add any questions you have there.

Mercurial is used slightly differently by different teams (e.g. localization vs coding). In general, consult with others where differences are noted.