Sheriffing/How To/Recommended Try Practices
The TryServer is a critical piece of our infrastructure for ensuring that patches are suitable for landing prior to being pushed. However, proper usage is important to avoid pushing under- or over-tested patches. This page is intended to provide some recommended practices for finding the right balance. The TryChooser Syntax Builder offers control over which builds and tests are run.
Note that the guidelines below are not a substitute for using your best judgement. The guidelines below are intended to be general recommendations but not absolute musts.
- In many cases, a "try: -b do -p all -u all -t none" is overkill and should be avoided. A full Try run consumes over 500hrs of machine time to generate a full set of builds and run the full suite of tests across them. This can have a real impact on wait times across resource-constrained platforms, including production runs.
- If in doubt, run both debug and opt builds.
- If the patch only affects one platform, only that platform should have builds and tests.
- For example, a B2G-only Try run should look something like "try: -b do -p emulator,emulator-jb,emulator-kk,linux32_gecko,linux64_gecko,macosx64_gecko,win32_gecko -u all -t none"
- Be aware of which test suites are likely to be affected by your patch and avoid running others that won't.
- For example, a devtools-only change is unlikely to break any tests outside of mochitest-dt and xpcshell.
- T runs can be created by using the "Restrict tests to platform(s)" checkbox on TryChooser.
- Linux32 is the best choice for the test platform as it is the least resource-constrained.
- "try: -b do -p all -u all[Ubuntu,-x64] -t none" will run builds on all platforms and tests only on Linux32.
- Multiple platforms can be specified as-needed (i.e. Linux + Android or any other combo as needed).
- Use the stop button on Treeherder's job details panel.
- Unneeded jobs are a frequent source of frustration-inducing backlog.
- Use the retrigger button on Treeherder's job details panel.
- You can trigger as many additional runs as needed (i.e. trying to track down an intermittent failure).
- Retriggering a build will cause all tests to be run again after the build completes.