14
edits
ReleaseEngineering/How To/VCSSync: Difference between revisions
new mapper url
m (add warning) |
(new mapper url) |
||
| (10 intermediate revisions by 3 users not shown) | |||
| Line 3: | Line 3: | ||
This is documentation on the mozharness-based vcs sync processes. | This is documentation on the mozharness-based vcs sync processes. | ||
<b>See [http://people.mozilla.org/~hwine/tmp/vcs2vcs/ legacy docs] for the legacy process.</b> | <div id="legacy"></div> | ||
<b>See [http://people.mozilla.org/~hwine/tmp/vcs2vcs/ legacy docs] for the legacy (aka "old") process.</b> | |||
= Quick notes = | = Quick notes = | ||
"VCS-sync" is a system which has evolved from it's original purpose to being the tool used whenever "this repository needs to be publicly visible 'over here'", perhaps with modifications, and a few exceptions on "publicly visible". The current (2016 Q1) business needs it addresses are: | |||
# Support for gecko developers who prefer to work with git: | |||
#* All gecko primary development and release branches are merged into a single git repository (with CVS history) in "gecko-dev.git". | |||
#* All gecko twig and misc branches are merged into a single git repository (with CVS history) in "gecko-projects.git". | |||
# Support for Mozilla development tools and processes which (for operational needs (e.g. travis-ci), or developer preference, should appear in both Mercurial and Git repositories. | |||
#* These are "mirrors" from one system to the other. | |||
#* One repo is always designated the RoR (Repository of Record), and is the only one where commits will "stick". | |||
#* Refer to the system configurations for each mirror to determine the RoR. (It can, and has, changed over time for some repositories.) | |||
# Support for Partner facing FxOS development processes. | |||
#* The URL's for these repositories can not change, as they are referenced in manifest files which comprise the delivery to Partners. | |||
#* These repositories on git.mozilla.org satisfy various contractual needs with FxOS partners. | |||
#* These repositories must not contain either object deletions, or non-fast-forward commits. | |||
#* These repositories only contain content that was "delivered" to Partners, with branches named according to FxOS conventions. | |||
#* These repositories include any "Mozilla Contributions" to FxOS, which includes code beyond gecko.git and gaia.git. | |||
# Support for internal FxOS development processes. | |||
#* Many Android repositories have been mirrored to git.mozilla.org for the purpose of having a known-good-repository for the build systems to clone from. | |||
#* Some tests still require translation from Git → Mercurial format. Each such gaia.git branch is mapped to a distinct Mercurial repository. | |||
== What's running where == | == What's running where == | ||
| Line 118: | Line 139: | ||
| vcssync2.srv.releng.usw2.mozilla.com | | vcssync2.srv.releng.usw2.mozilla.com | ||
| project-branches (gecko-projects) | | project-branches (gecko-projects) | ||
| /opt/vcs2vcs | | /opt/vcs2vcs | ||
| vcs2vcs | | vcs2vcs | ||
| Line 216: | Line 232: | ||
== Maintenance == | == Maintenance == | ||
=== How to add a repo to gecko-git === | |||
=== How to add a repo to | '''''CAUTION:''' production gecko.git is still handled by legacy vcs-sync - see [http://people.mozilla.org/~hwine/tmp/vcs2vcs/cookbook.html#adding-a-branch-to-gecko-git legacy docs].''<br /> | ||
'''''CAUTION:''' production gecko.git is still handled by legacy vcs-sync - | This shouldn't be done lightly. Gecko.git is our partner-oriented repo and should be treated with kid gloves. | ||
This shouldn't be done lightly. Gecko.git is our partner-oriented repo and should be treated with kid gloves. gecko-dev is our release-trains and inbound-branches only. | === How to add another repo to mirror for b2g OS === | ||
Mirroring repositories for purpose of b2g os builds is handled by the legacy docs. See [http://people.mozilla.org/~hwine/tmp/vcs2vcs/cookbook.html#mirroring-a-git-repository here]. | |||
=== How to add a repo to gecko-dev === | |||
gecko-dev is our developer focused repository with all gecko branches related to release-trains and inbound-branches only. | |||
You should follow the example [http://hg.mozilla.org/build/mozharness/file/7d9425c91051/configs/vcs_sync/beagle.py#l96 here]. Annotated: | You should follow the example [http://hg.mozilla.org/build/mozharness/file/7d9425c91051/configs/vcs_sync/beagle.py#l96 here]. Annotated: | ||
| Line 267: | Line 286: | ||
If the project isn't under hg.m.o/projects, add it [http://hg.mozilla.org/build/mozharness/file/7d9425c91051/configs/vcs_sync/project-branches.py#l32 here]. | If the project isn't under hg.m.o/projects, add it [http://hg.mozilla.org/build/mozharness/file/7d9425c91051/configs/vcs_sync/project-branches.py#l32 here]. | ||
=== How to add a project cloning to a custom github account/project === | |||
* Key setup | |||
** Generate an ssh key for pushing to the github project (no passphrase) | |||
** Add the key to github/org-name/project-name as a deploy key | |||
** Add the key to the vcs2vcs account on the vcssync server | |||
* Add the project to mapper for hg to git hash mapping | |||
** Get a token from https://mozilla-releng.net/tokens | |||
** POST to /project-name on https://mapper.mozilla-releng.net | |||
* Filesystem setup on the vcssync server | |||
** Clone the source hg repo to /opt/vcs2vcs/vcs_sync_build/build/stage_source/ | |||
** Clone the hg repo from stage_source to conversion <pre>hg clone /opt/vcs2vcs/vcs_sync_build/build/stage_source/$PROJECT /opt/vcs2vcs/vcs_sync_build/build/conversion/$PROJECT | |||
</pre> | |||
* Add to a vcs-sync configuration file. For example: https://hg.mozilla.org/build/mozharness/rev/57a550a9e307863ab125f1cd1c381de974caf79d | |||
** Set the source location | |||
** Set the destination org/project | |||
* Monitor results and fix/tweak | |||
** Join and watch the error emails in https://groups.google.com/a/mozilla.com/forum/?hl=en&pli=1*!forum/releng-ops-trial | |||
** Check the github project for the commits to appear | |||
** Compare the github commits with the hg commits | |||
=== How to add a locale to l10n conversion === | === How to add a locale to l10n conversion === | ||
| Line 295: | Line 335: | ||
This can also be set in the config file per-repository, as <code>check_incoming</code> inside each of the specific <code>conversion_repos</code> where we want to skip the incoming check. | This can also be set in the config file per-repository, as <code>check_incoming</code> inside each of the specific <code>conversion_repos</code> where we want to skip the incoming check. | ||
To force a push manually: | |||
* cd into project-branches jobs top level directory | |||
* wait for projects.lock to disappear (from cron job) | |||
* touch projects.lock # grab lock to avoid race with automation | |||
** or: <code>touch my_user_name.lock; while ! ln my_user_name.lock projects.lock ; do sleep 10 ; done</code> | |||
** wait for return of command prompt | |||
* get the command line from the run_*.sh file: grep python run_*.sh | |||
* add the '--no-check-incoming' option to the command | |||
* wait for processing to complete | |||
** '''NOTE''': the log output is only to your screen. The usual summary, "<code>projects.log</code>" is only updated via the <code>run_*</code> script. ({{bug|1279008|see also}}) | |||
*** The official logs from the last run are in the "<code>./logs/</code>" directory. | |||
*** Older logs are in the "<code>./build/upload/logs/</code>" directory. | |||
* rm 'projects.lock' # let normal processing resume | |||
=== Where to find the keys === | === Where to find the keys === | ||