Sheriffing/How To/Recommended Try Practices
The TryServer is a critical piece of our infrastructure for ensuring that patches are suitable for landing prior to being pushed. However, proper usage is important to avoid pushing under- or over-tested patches. This page is intended to provide some recommended practices for finding the right balance.
Note that the guidelines below are not a substitute for using your best judgment. The guidelines below are intended to be general recommendations but not absolute musts.
- In many cases, running all builds and tests is overkill and should be avoided. A full Try run consumes over 500hrs of machine time to generate a full set of builds and run the full suite of tests across them. This can have a real impact on wait times across resource-constrained platforms, including production runs.
- If in doubt, run both debug and opt builds.
- If the patch only affects one platform, only that platform should have builds and tests.
- Be aware of which test suites are likely to be affected by your patch and avoid running others that won't.
- For example, a devtools-only change is unlikely to break any tests outside of mochitest-dt and xpcshell.
- If the code is cross-platform, a T-style run (build on all platforms, test on one) will find most problems while making best use of resources. (Use copy several filter changes with
./mach try chooserand copy and paste if necessary).
- Linux64 is the best choice for the test platform as it is the least resource-constrained.
- If you no longer need builds/tests from a run, cancel the remaining jobs.
- Use the stop button on Treeherder's job details panel.
- Unneeded jobs are a frequent source of frustration-inducing backlog.
- Any existing jobs on a push can be retriggered without a new push.
- Use the retrigger button on Treeherder's job details panel.
- You can trigger as many additional runs as needed (i.e. trying to track down an intermittent failure).
- Retriggering a build will cause all tests to be run again after the build completes.