|
|
| (6 intermediate revisions by 2 users not shown) |
| Line 18: |
Line 18: |
| # b2g_build.py - the build script for b2g - since this needs to lookup shas in order to reference frozen commit versions in manifests | | # b2g_build.py - the build script for b2g - since this needs to lookup shas in order to reference frozen commit versions in manifests |
|
| |
|
| Mapper is written as a RelEng API blueprint - please note RelEng API has its [https://api.pub.build.mozilla.org/docs/ own documentation] too. | | Mapper is written as a RelEng API blueprint - please note RelEng API has its [https://docs.mozilla-releng.net/projects/releng-mapper.html own documentation] too. |
|
| |
|
| = Source =
| |
| mapper's source is currently hosted at https://github.com/petemoore/mapper
| |
|
| |
|
| This will be moving as soon as it is ready to go to production. Currently it is in staging (see [https://bugzilla.mozilla.org/show_bug.cgi?id=847640 bug 847640])
| | = Mapper Development & Deployment = |
|
| |
|
| = Old Mapper Deployment =
| | The mapper is a project in [https://github.com/mozilla-releng/services mozilla-releng/services] repository, and so the details about how to deploy locally for development, and where the staging/production environments are, can be seen in the [https://docs.mozilla-releng.net/ RelEngAPI documentation]. |
|
| |
|
| Until the "New Mapper" goes live, this is the information about the "old mapper" (which is being superceded).
| |
|
| |
|
| mapper only requires bottle to run. It's recommended to run inside a virtual environment
| | = Mapper Data Feed = |
| | |
| Our current production deployment of mapper lives on cruncher under /home/buildduty/mapper. It listens locally on port 8888 (specified in mapper/app.py). The apache instance on cruncher is configured to forward requests to http://cruncher/mapper/* to http://locahost:8888/* (configured in /etc/httpd/conf/httpd.conf)
| |
| | |
| The mapper process is managed by supervisord which will ensure it is started up if the machine is ever rebooted, or if the process crashes. (configured in /etc/supervisord.conf)
| |
| | |
| Source is available at https://github.com/catlee/mapper
| |
| | |
| = New Mapper Deployment = | |
| | |
| The new mapper is a Releng API blueprint, and so the details about how to deploy locally for development, and where the staging/production environments are, can be seen in the [https://api.pub.build.mozilla.org/docs/ RelEngAPI documentation].
| |
| | |
| = Old Mapper Data Feed =
| |
| | |
| This section describes (roughly) how vcs-sync provides the map files served by mapper.
| |
| | |
| The map files are generated and combined on the vcs-sync machines, then pulled onto cruncher and used by `mapper`. See the [http://hg.mozilla.org/users/hwine_mozilla.com/repo-sync-tools/ source] and [http://people.mozilla.com/~hwine/tmp/vcs2vcs/ docs] for vcs-sync for more details. (Especially the [http://people.mozilla.com/~hwine/tmp/vcs2vcs/mapper_support.html mapper support] section.)
| |
| | |
| mapper expects hggit map files to be available under the 'mapfiles' directory of the application. On cruncher, these are in /home/buildduty/mapper/mapfiles. Each subdirectory of mapfiles corresponds loosely to a different repository being tracked. On cruncher, the mapfiles for each of these projects are symlinked to the mapfiles being published to cruncher via the process outlined above.
| |
| | |
| Legacy vcs-sync provides mapfile information for the following repository groupings. ''(These can be seen via "<tt>ls */mapfile-needed</tt>" in the legacy configs directory.)''
| |
| * git -> hg from <tt>gaia.git</tt> code to <tt>integration/*</tt> (11 repositories configured)
| |
| * hg -> git for gaia L10N (75 repositories configured)
| |
| * hg -> git for gecko L10N (50 repositories configured)
| |
| * hg -> git for gecko.git (15 repositories configured - from <tt>releases-gecko/hgrc</tt>)
| |
| | |
| ''Note that the above docs will be integrated into the wiki after vcs-sync development stabilizes.''
| |
| | |
| = New Mapper Data Feed =
| |
|
| |
|
| VCS Sync is making HTTP Post requests to mapper, providing new mappings for mapper to insert into its database. | | VCS Sync is making HTTP Post requests to mapper, providing new mappings for mapper to insert into its database. |
| Line 63: |
Line 32: |
| = API methods = | | = API methods = |
|
| |
|
| Below are the 7 API methods currently supported, together with example urls from the staging environment. | | Below is a summary of the 4 unauthenticated API methods currently usable, together with example urls from the production environment. |
|
| |
|
| Please note, mapper uses secure authentication (provided by RelEng API), so you will not be able to use the POST examples below, but you will be able query mapper using e.g. the example GET requests below.
| | For fuller details, and endpoints requiring authentication, the in process documentation is authoritative: [https://mapper.mozilla-releng.net/docs production] or [https://mapper.staging.mozilla-releng.net/docs staging] instances. |
|
| |
|
| == GET Routes == | | == GET Routes == |
| Line 71: |
Line 40: |
| === Returns a mapping pair === | | === Returns a mapping pair === |
| * GET: /<project>/rev/<vcs_type>/<commit> | | * GET: /<project>/rev/<vcs_type>/<commit> |
| * Example: https://api-pub-build.allizom.org/mapper/build-puppet/rev/git/69d64a8a18e6e001eb015646a82bcdaba0e78a24 | | * Example: https://mapper.mozilla-releng.net/build-puppet/rev/git/69d64a8a18e6e001eb015646a82bcdaba0e78a24 |
| * Example: https://api-pub-build.allizom.org/mapper/build-puppet/rev/hg/68f1b2b9996c4e33aa57771b3478932c9fb7e161 | | * Example: https://mapper.mozilla-releng.net/build-puppet/rev/hg/68f1b2b9996c4e33aa57771b3478932c9fb7e161 |
|
| |
|
| === Returns full mapfile for a given project === | | === Returns full mapfile for a given project === |
| * GET: /<project>/mapfile/full | | * GET: /<project>/mapfile/full |
| * Example: https://api-pub-build.allizom.org/mapper/build-puppet/mapfile/full | | * Example: https://mapper.mozilla-releng.net/build-puppet/mapfile/full |
|
| |
|
| === Returns a subset of a mapfile, since a given date === | | === Returns a subset of a mapfile, since a given date === |
| * GET: /<project>/mapfile/since/<since> | | * GET: /<project>/mapfile/since/<since> |
| * Example: https://api-pub-build.allizom.org/mapper/build-mozharness/mapfile/since/29.05.2014%2017:02:09%20CEST | | * Example: https://mapper.mozilla-releng.net/build-mozharness/mapfile/since/29.05.2014%2017:02:09%20CEST |
| | |
| == POST Routes ==
| |
| | |
| === Inserts a mappings file strictly (no duplicates allowed) ===
| |
| * POST: /<project>/insert
| |
| * Example: https://api-pub-build.allizom.org/mapper/insert
| |
|
| |
|
| === Inserts a mappings file, allowing duplicates === | | === Returns list of projects === |
| * POST: /<project>/insert/ignoredups | | * GET: /projects |
| * Example: https://api-pub-build.allizom.org/mapper/insert/ignoredups | | * Example: https://mapper.mozilla-releng.net/projects |
|
| |
|
| === Inserts an individual mapping === | | = Staging = |
| * POST: /<project>/insert/<git_commit>/<hg_changeset>
| |
| * Example: https://api-pub-build.allizom.org/mapper/insert/69d64a8a18e6e001eb015646a82bcdaba0e78a24/68f1b2b9996c4e33aa57771b3478932c9fb7e161
| |
|
| |
|
| === Inserts a new project ===
| | The staging environment is available under https://mapper.staging.mozilla-releng.net (Look under "Documentation" to find details.) |
| * POST: /<project>
| |
| * Example: https://api-pub-build.allizom.org/mapper/build-puppet
| |