Try Server Suggestions and Feature Requests

• One flag for all platforms (desktop & mobile)
• upload script: allow multiple patches
• upload script: use reference to bugzilla attachments instead of uploading a patch
  • Direct integration into bugzilla? "Click here to test patch"?
• upload script: allow users to reference 1..N bug numbers as part of the build description
• upload script: allow users to check an "email me when this build is ready" box
• try script: email users who checked the "email me when this build is ready" box
• upload script: ability to see default .mozconfig
• allow patches compressed with bzip2 or gzip
• copy build logs (and maybe input patches) to output download directory
• give developers an option to make the try-server "rebase and push to mozilla-central" if the patch is green on try-server
  • or starred as "effectively green"
• Display the last green hg id somewhere (this is bug 570859).
• Ability to just build installer/zip/dmg/tar without uploading symbols, running tests, etc.
• Compare talos numbers relative to mozilla-central numbers in proximity hg DAG-wise, see http://blog.mozilla.com/axel/2010/06/07/trying-talos-on-node-js/ for a proof of concept.
• (roc) We have a problem that try server doesn't degrade gracefully. When it's loaded, you lose results, or the results lose relevance when they're really old and far away from where trunk is now. So we get into situations where people are pushing but all the work done for the push is basically wasted. In these kinds of situations we need some kind of admission control so that fewer people can push but those who can actually get useful results.
• (mrbkap) The ability to cancel a try server run once it's started would be useful. Especially after seeing some failures, I often notice that my patch is wrong in some way, and I don't want to continue to receive e-mails or see more results until I fix my patch and push again. This might also help me use less resources whenever I push to try.

(rillian) tbpl.mozilla.org gives me a 'cancel build' button when I mouse over a commit. Does that do what you want? Seems to depend on LDAP credentials though.

• (bent) I'd love a link to the tinderbox log in the success/failure emails that get generated rather than a simple "go check tinderbox" link. Sometimes I have a hard time finding the logs that I'm interested in.
• (sfink) use ccache on the try build boxes, to win back some of the time lost because of clobber builds.
• (sfink) report the change id in the build, test, perf logs.
• (sfink) TBPL - when displaying the changes for a particular push, show everything that doesn't exist in a "main" tree (m-c, TM, etc.) Highlight the ones that are being shown now (which I assume are the ones that are new to the try repo.) (This is probably more difficult than it's worth.)
• (sfink) Job control UI: after pushing, be able to visit a web page specific to your push. Show the dependency graph of all current and potential jobs. Be able to blast away existing ones (eg builds on unwanted architectures) and enable others (eg Talos runs assuming they are disabled by default.)
• (sfink) When pushing, have an option to do a "dependent" job graph: instead of starting everything up in parallel, pick one build (possibly by architecture, possibly by the fastest available builder, possibly random) and make the other builds dependent on its success. Do the same with the tests. This is for reducing wasted resources when either (1) the developer doesn't mind waiting longer for the results eg it's the end of the day or (2) the developer is uncertain whether it will build at all on some arch or with certain compile options.
• (sfink) Export a $platform variable to the mozconfig-extra script for easier selection of specific platforms. (I know, my bug for this is WONTFIX because bigger better things are already underway, but I'm mentioning it for completeness.)
• (sfink) Use distcc for the build boxes. Only useful if the load is currently unevenly distributed.
• (sfink) Allow selecting subsets of tests.

Preventing you from using tryserver effectively for your needs

• (bz) Being able to actually get results in sane amounts of time. It's now been over 24 hours since I pushed a patch, and I have yet to see a single mac test result, Linux debug test result, 32-bit Linux optimized test result, anything at all on Win7. The Win2k3 optimized build died with an infrastructure error... In other words, a bunch of machine time was wasted, and I still can't get the data I wanted.
• (roc) Understanding tryserver performance results is hard. The tbpl comparator is somewhat helpful, but only if you can identify a changeset on the same page of results that would be a good baseline for comparison, and that is error-prone and often impossible. You also don't get a fine-grained breakdown of test results. I generally end up screen-scraping the old-style Tinderbox with a script I wrote. The process is even slower and harder when your test results appear over several days, if at all (see above).
• (karl) Sometimes tinderbox loses results of completed runs. I get emails telling me that they've completed but they do not show up on the tinderbox. A link in the email to the log (or even some other way to find the log) would be helpful. One time some results showed up temporarily in tbpl but then disappeared (and could not, or no longer, be found on tinderbox).