Build:TryServer:Suggestions

From MozillaWiki
Jump to navigation Jump to search

Try Server Suggestions and Feature Requests

  • upload script: allow multiple patches
  • upload script: use reference to bugzilla attachments instead of uploading a patch
    • Direct integration into bugzilla? "Click here to test patch"?
  • upload script: allow users to reference 1..N bug numbers as part of the build description
  • upload script: allow users to check an "email me when this build is ready" box
  • try script: email users who checked the "email me when this build is ready" box
  • upload script: ability to see default .mozconfig
  • allow patches compressed with bzip2 or gzip
  • copy build logs (and maybe input patches) to output download directory
  • give developers an option to make the try-server "rebase and push to mozilla-central" if the patch is green on try-server
    • or starred as "effectively green"
  • Display the last green hg id somewhere (this is bug 570859).
  • Ability to just build installer/zip/dmg/tar without uploading symbols, running tests, etc.
  • Compare talos numbers relative to mozilla-central numbers in proximity hg DAG-wise, see http://blog.mozilla.com/axel/2010/06/07/trying-talos-on-node-js/ for a proof of concept.

Preventing you from using tryserver effectively for your needs

  • (bz) Being able to actually get results in sane amounts of time. It's now been over 24 hours since I pushed a patch, and I have yet to see a single mac test result, Linux debug test result, 32-bit Linux optimized test result, anything at all on Win7. The Win2k3 optimized build died with an infrastructure error... In other words, a bunch of machine time was wasted, and I still can't get the data I wanted.
  • (roc) Understanding tryserver performance results is hard. The tbpl comparator is somewhat helpful, but only if you can identify a changeset on the same page of results that would be a good baseline for comparison, and that is error-prone and often impossible. You also don't get a fine-grained breakdown of test results. I generally end up screen-scraping the old-style Tinderbox with a script I wrote. The process is even slower and harder when your test results appear over several days, if at all (see above).