Confirmed users
9
edits
Sheriffing/TBPL/FuturePlans: Difference between revisions
no edit summary
(arpads ideas) |
No edit summary |
||
| Line 1: | Line 1: | ||
==Goals== | ==Goals== | ||
* get rid of Tinderbox ( | * get rid of Tinderbox ({{Bug|586539}}) | ||
* be super-fast, even when going back in time | * be super-fast, even when going back in time | ||
| Line 8: | Line 8: | ||
* for getting rid of Tinderbox: | * for getting rid of Tinderbox: | ||
** move all the useful stuff on Tinderbox over to TBPL, specifically: | ** move all the useful stuff on Tinderbox over to TBPL, specifically: | ||
*** current sheriff ( | *** current sheriff ({{Bug|540464}}) | ||
*** tree state ( | *** tree state ({{Bug|525464}}) | ||
*** other notes at the top | *** other notes at the top | ||
** put logs somewhere outside Tinderbox | ** put logs somewhere outside Tinderbox | ||
*** where? (FTP is | *** where? (FTP is {{Bug|530318}}) | ||
** move starring system outside Tinderbox | ** move starring system outside Tinderbox | ||
*** where? TBPL server? | *** where? TBPL server? | ||
| Line 28: | Line 28: | ||
* correlate buildbot data with Tinderbox data if possible | * correlate buildbot data with Tinderbox data if possible | ||
** finished Tinderbox jobs have a unique ID, buildbot jobs don't | ** finished Tinderbox jobs have a unique ID, buildbot jobs don't | ||
*** create something (hash starttime + slave name?) - see | *** create something (hash starttime + slave name?) - see {{Bug|586539}} comment 2 | ||
** put the correlation into TBPL database | ** put the correlation into TBPL database | ||
* everything using PHP and MySQL for now | * everything using PHP and MySQL for now | ||
| Line 46: | Line 46: | ||
* Storage and processing of logs and summaries | * Storage and processing of logs and summaries | ||
** Is done by Tinderbox at the moment. | ** Is done by Tinderbox at the moment. | ||
** Plan is to upload logs to FTP ( | ** Plan is to upload logs to FTP ({{Bug|530318}}) and to do the processing (error line linking, summary generation) on a server | ||
* Commenting on random-orange bugs ("Tinderboxpushlog Robot") | * Commenting on random-orange bugs ("Tinderboxpushlog Robot") | ||
** Currently done by a PHP script on TBPL's server | ** Currently done by a PHP script on TBPL's server | ||
| Line 61: | Line 61: | ||
* Pushlog data is cached on hg.mozilla.org by the pushlog itself. | * Pushlog data is cached on hg.mozilla.org by the pushlog itself. | ||
** (Side issue: at the moment cache invalidation is broken there, see | ** (Side issue: at the moment cache invalidation is broken there, see {{Bug|587559}}) | ||
** People have been asking for another cache on the TBPL side, see | ** People have been asking for another cache on the TBPL side, see {{Bug|526159}} | ||
* Tinderbox data is partly cached by Tinderbox itself: the last 24 hours are in json.js, further back in time nothing is cached. | * Tinderbox data is partly cached by Tinderbox itself: the last 24 hours are in json.js, further back in time nothing is cached. | ||
* Full Tinderbox logs are stored as .gz files on the Tinderbox server, but linkified logs and summaries are regenerated on every request, as far as I know. | * Full Tinderbox logs are stored as .gz files on the Tinderbox server, but linkified logs and summaries are regenerated on every request, as far as I know. | ||
| Line 74: | Line 74: | ||
* Pushlog data still only cached by Pushlog itself | * Pushlog data still only cached by Pushlog itself | ||
** Load will be reduced significantly when the new Pushlog JSON is in place ( | ** Load will be reduced significantly when the new Pushlog JSON is in place ({{Bug|584836}}, waiting to be deployed: {{Bug|585882}}) | ||
* Tinderbox data cached in TBPL database | * Tinderbox data cached in TBPL database | ||
* Buildbot data cached in TBPL database | * Buildbot data cached in TBPL database | ||
| Line 99: | Line 99: | ||
==Arpad’s ideas== | ==Arpad’s ideas== | ||
* bitrotted server and client implementation in | * bitrotted server and client implementation in {{Bug|553549}} | ||
** based on php, mysql; parses pushlog html and tinderbox json | ** based on php, mysql; parses pushlog html and tinderbox json | ||
** revived the client part in | ** revived the client part in {{Bug|576489}}, bitrotted a bit again :(, would like to wait for pl json and tb json2 before continuing work | ||
* how fast is the new pushlog json? does it beat a custom database? | * how fast is the new pushlog json? does it beat a custom database? | ||
** the json includes the list of changed files which is irrelevant to tbpl | ** the json includes the list of changed files which is irrelevant to tbpl | ||
* how to properly handle builds that pull from multiple repos (preliminaty ui in | * how to properly handle builds that pull from multiple repos (preliminaty ui in {{Bug|587761}}) | ||
** builds.m.o json has confusing attribute names for multi-repo builds | ** builds.m.o json has confusing attribute names for multi-repo builds | ||
** want to display multiple pushlogs side-by-side, or easy switching without the need to reload the data | ** want to display multiple pushlogs side-by-side, or easy switching without the need to reload the data | ||