canmove, Confirmed users
1,126
edits
Sheriffing/TBPL/DeveloperDocs: Difference between revisions
Remove developer contacts
(second pass) |
(Remove developer contacts) |
||
| (10 intermediate revisions by 3 users not shown) | |||
| Line 7: | Line 7: | ||
Every checkin to a Firefox repository triggers the execution of several automated test jobs for the new revision. For each push the new source is compiled to a fresh binary on all supported platforms, and the resulting binaries are then tested by lots of automated unit and performance tests. | Every checkin to a Firefox repository triggers the execution of several automated test jobs for the new revision. For each push the new source is compiled to a fresh binary on all supported platforms, and the resulting binaries are then tested by lots of automated unit and performance tests. | ||
TBPL shows a repository's state by correlating pushes (checkins) with the results of these automated jobs. For each push, the corresponding changesets (code changes) are listed in the left half of the page, and the triggered jobs are listed in the right half of the page. Every job (also called "build" | TBPL shows a repository's state by correlating pushes (checkins) with the results of these automated jobs. For each push, the corresponding changesets (code changes) are listed in the left half of the page, and the triggered jobs are listed in the right half of the page. Every job (also called "build" or "run") is represented by one colored letter. The color of each letter depends on the success status of the job. | ||
Developers need to make sure that their changes don't "break the tree", i.e. that they don't cause any tests to fail, so they'll usually monitor TBPL for a while after they've checked in. | Developers need to make sure that their changes don't "break the tree", i.e. that they don't cause any tests to fail, so they'll usually monitor TBPL for a while after they've checked in. | ||
| Line 27: | Line 27: | ||
= Development documentation = | = Development documentation = | ||
There is no design doc for development. Development happens in the repository at | There is no design doc for development. Development happens in the repository at https://hg.mozilla.org/webtools/tbpl/ and in the Bugzilla component Webtools : Tinderboxpushlog. | ||
Installation instructions are given in the [ | Installation instructions are given in the [https://hg.mozilla.org/webtools/tbpl/raw-file/tip/README README file]. | ||
TBPL doesn't have versions or milestones. Changes are usually implemented in response to a bug in Bugzilla (see [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=OPEN+product%3AWebtools+component%3ATinderboxpushlog list of open bugs]) and deployed as they come in. The motivation behind specific parts of the code can be figured out by looking at the hg annotate of the file and tracing it back to the bug where the code was landed. | TBPL doesn't have versions or milestones. Changes are usually implemented in response to a bug in Bugzilla (see [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=OPEN+product%3AWebtools+component%3ATinderboxpushlog list of open bugs]) and deployed as they come in. The motivation behind specific parts of the code can be figured out by looking at the hg annotate of the file and tracing it back to the bug where the code was landed. | ||
= | == Implementation overview == | ||
(I'm going to start a design doc right here, though it should probably be broken out to a different wiki page at some point.) | |||
TBPL is mostly about useful presentation of data that's already available elsewhere. For example, the list of pushes to a repository is at hg.mozilla.org/repository/pushloghtml ([http://hg.mozilla.org/mozilla-central/pushloghtml e.g. mozilla-central pushlog]). Another example is the [https://build.mozilla.org/buildapi/self-serve self-serve UI] that allows one to cancel and restart builds. So it's TBPL's job to reassemble that data in a way that's as accessible for developers as possible. | |||
For some of that source data, TBPL is the only human-readable presentation. Run result data, for example, is only available as JSON (from Buildbot). | |||
There are two exceptions to the "data comes from outside" principle: Job comments (also called "build stars") and the list of hidden builders will be stored on the TBPL server itself, in a MongoDB database. In Tinderbox mode both these things were stored on Tinderbox. | |||
=== Architecture === | |||
TBPL started out as a pure client-side JavaScript webapp. All data used to be gathered and assembled in the user's browser. Over time, TBPL has grown a small collection of PHP scripts that make up its server-side component. | |||
We want to do as much as possible on the client side because that's easier to develop and play with, especially for outsiders that haven't been involved in TBPL development before. If something is handled on the server side, that's for one of these reasons: | |||
* It acts as an interface to the database (e.g. getRevisionBuilds.php or submitBuildStar.php). | |||
* It makes use of "secret" data (submitBugzillaComment.php which needs a bugzilla user password). | |||
* It performs a very expensive operation where the results should be cached (all other php scripts). | |||
== File descriptions == | |||
[https://spreadsheets.google.com/spreadsheet/ccc?key=0AloqqAegSq7udGdXdVc2UHBVdFo3eFpBbFRXcWhmU3c&hl=en_US I'm building a table here, I'll import it into the Wiki when I'm finished] | |||
(to be continued...) | |||
= Hardware = | = Hardware = | ||
| Line 52: | Line 69: | ||
Any form of unix that's supported by Apache, PHP, Python and MongoDB should work. IT will decide what will actually be used. | Any form of unix that's supported by Apache, PHP, Python and MongoDB should work. IT will decide what will actually be used. | ||
= Network flows = | = Network flows = | ||
| Line 95: | Line 102: | ||
(what database server(s) - rw & ro, db name(s), db username) Other requirements | (what database server(s) - rw & ro, db name(s), db username) Other requirements | ||
TBPL | TBPL MySQL databases: | ||
Production lives on generic1.db.phx1.mozilla.com at the db tbpl_mozilla_org | |||
dev and stage both live on dev1.db.phx1.mozilla.com: | |||
Stage: tbpl_allizom_org | |||
Dev: tbpl_dev_allizom_org | |||
= File storage = | = File storage = | ||
| Line 116: | Line 126: | ||
The importer is idempotent; it never destroys data and it doesn't insert duplicates. | The importer is idempotent; it never destroys data and it doesn't insert duplicates. | ||
== | = Other links = | ||
See the main TBPL page: [[Sheriffing/TBPL]] | |||