Sheriffing/TBPL/FuturePlans

Goals

• get rid of Tinderbox (bug 586539)
• be super-fast, even when going back in time

Necessary prereqs

• for getting rid of Tinderbox:
  • move all the useful stuff on Tinderbox over to TBPL, specifically:
    • current sheriff (bug 540464)
    • tree state (bug 525464)
    • other notes at the top
  • put logs somewhere outside Tinderbox
    • where? (FTP is bug 530318)
  • move starring system outside Tinderbox
    • where? TBPL server?
• for being fast:
  • cache Tinderbox history on TBPL server in a database
    • for how long? a week?
    • do the same for pushlog and buildbot data?

Transition plan for moving away from Tinderbox

• use data from both systems
• give TBPL a server component with a database
• put everything into TBPL's database
• correlate buildbot data with Tinderbox data if possible
  • finished Tinderbox jobs have a unique ID, buildbot jobs don't
    • create something (hash starttime + slave name?) - see bug 586539 comment 2
  • put the correlation into TBPL database
• everything using PHP and MySQL for now

Decisions we need to make

• How much should happen on a server and how much on the client?
• If it doesn't happen on the client, what server should do it?

These things need to go on a server, no matter which one

• Generating buildbot JSON data in the right format
  • In dbaron's experiment this is a script on dbaron's server that transforms data from build.mozilla.org
  • But we should probably generate the data in the right format on build.mozilla.org in the first place
• Starring
  • Resides in Tinderbox at the moment
• Storage and processing of logs and summaries
  • Is done by Tinderbox at the moment.
  • Plan is to upload logs to FTP (bug 530318) and to do the processing (error line linking, summary generation) on a server
• Commenting on random-orange bugs ("Tinderboxpushlog Robot")
  • Currently done by a PHP script on TBPL's server
• Searching bugzilla for random-orange bugs
  • Also happens with PHP script on TBPL server at the moment
  • Too slow to do every time on the client, needs to be done and cached on a server
• (General caching so stuff is fast)

Caching plan

We need to decide what we cache where. At the moment it looks like this:

Caches already in place

• Pushlog data is cached on hg.mozilla.org by the pushlog itself.
  • (Side issue: at the moment cache invalidation is broken there, see bug 587559)
  • People have been asking for another cache on the TBPL side, see bug 526159
• Tinderbox data is partly cached by Tinderbox itself: the last 24 hours are in json.js, further back in time nothing is cached.
• Full Tinderbox logs are stored as .gz files on the Tinderbox server, but linkified logs and summaries are regenerated on every request, as far as I know.
• The summaries of these logs are cached by TBPL.

Caches during transition period

where by transition period I mean the time before we've replaced Tinderbox completely

This is what I'm proposing. Please discuss.

• Pushlog data still only cached by Pushlog itself
  • Load will be reduced significantly when the new Pushlog JSON is in place (bug XXX)
• Tinderbox data cached in TBPL database
• Buildbot data cached in TBPL database
• Full Tinderbox logs still only stored on Tinderbox server
• Summaries still cached by TBPL.

Caches after Tinderbox is gone

At this point we don't need to corellate Tinderbox data with Buildbot data anymore, so most of TBPL's database can go away. The Buildbot server is assumed to be fast.

• Pushlog data still only cached by Pushlog itself
• Buildbot data cached on Buildbot server in the right format
• Full build logs somewhere on FTP
• Processed build logs and summaries cached on some other server (probably on the server that's processing the logs)
• Summaries cached on that same server