Build:TryChooser

Syntax for Custom Try Requests

The goal of this syntax is to allow user to select particular build/unittest/talos combinations in their hg commit message, a config.info file in the $topsrcdir, or from a web interface.

Syntax

try: --build [d,o,do,all] --p [(desktop_platforms,...),none,all] --m [(mobile_platforms,...),none,all] --u [(test_suites,...),none,all] --t [(talos_suites,...),none,all]

This covers:

• Build Types: both(do) || opt(o) || debug(d) || m-c all override (all)
• Platforms: all || none || --p {linux,linux64,macosx,macosx64,win32}
• Mobile: all || none || --m {mobile-maemo5-qt, android, etc}
• Unittest Suites: all || none || --u {crashtest,reftest,mochitests-{1/5,2/5,3/5,4/5,5/5},mochitest-other,xpcshell,etc}
• Talos Suites: all || none || --t {tp4,tscroll,tspider,etc}

Usage Examples

In an hg commit message (currently active)

 # to run what mozilla-central's automation does ie: all platforms, all tests/talos
 hg commit -m "bug 432432 testing something try: --build all" 

 # both opt/debug builds, all available {desktop,mobile} platforms, select unittests, no talos
 hg commit -m "bug 432432 testing something try: --build do --u crashtest" 

 # linux debug build, all available mobile platforms, no unittests, no talos
 hg commit -m "bug 432432 testing something try: --build d --p linux --u none"

 # mobile-only builds
 hg commit -m "bug 432432 testing something try: --build o --p none"

 # all builds, all available desktop platforms, no mobile, all unittests & select talos
 hg commit -m "bug 432432 testing something try: --build do --m none --u all --t chrome,nochrome"

 # opt builds on all available {desktop,mobile} platforms, select unittests, select talos 
 hg commit -m "bug 432432 testing something try: --build o --u jsreftest,crashtest --t tp4,tscroll"

In a try.info file (not yet active)

try.info file pushed to $topsrc_dir would look like:

 try: --build do --m none --u none --t all

 OR

 try: --build all

 OR

 try: --build d --m none --u reftest --t tp4,scroll

Scheduler/Builder Parsing

• splits on 'try:'
• whatever follows after 'try:' is passed to try_parser as a string
• the string is processed into a list of args handed to parse_known_args (thus keeping anything that is not a recognized argument from causing errors)
• we have the ability to do manual sendchanges to send in buildbot sendchange comments in order to make it possible to override the hg commit comments if necessary to (re)run a particular test/talos/platform or anything that the commit didn't correctly trigger

Notes on Usage

Default Set

If no try hg commit comment (ie: beginning with 'try:') is present then the default set is:

All available platforms for try: 
 desktop opt & debug builds
 mobile builds
 all unittests
 no talos

Optional arguments

If the following exist they will be handled as follows:

Build Types

(if not specified default is debug & opt)

--build o || d || do || all #(all is an override to get m-c run which does all desktop/mobile/test/talos)

Desktop Platforms

(if not specified default is all) specifying platforms should be comma-separated names with no spaces:

--p linux,macosx   ||  --p none || --p all

Mobile Platforms

(if not specified default is all) specifying platforms should be comma-separated names with no spaces:

--m maemo5-gtk,android-r7 || --m none || --m all

Unittests

(if not specified default is all) specifying platforms should be comma-separated names with no spaces:

--u crashtest,jsreftest || --u none || --u all 

Talos

(if not specified default is none) specifying platforms should be comma-separated names with no spaces:

--t tp4,tscroll || --t none || --t all

Tutorial

• You've got your patch ready to try
• Set the commit comment to what you would like to run on try
• In this case, let's say you want a linux opt build, no mobile, no unittests, and only talos tp4 results

hg commit -m "Patch to fix a pixel placement try: --build o --p linux --m none --u none --t tp4"

• Push to try
• What happens? 'Linux tryserver build' builder is created and runs your opt linux build. If the build completes successfully then the talos tests are triggered and once the test-master gets the notice of the change it will read the hg commit comments and only kick off the one builder for tp4 'Rev3 Fedora 12 tryserver talos tp4'

What if I don't put try: in my commit comments?

If you do not specify anything in your comments you will get the default set of builds for try.

What if make a syntax error in my commit comments?

If you any arguments in your comments are unclear you will get the default set of builds for that argument.

eg: try: --build f --u crashtest
# f is not a recognized argument and so you will end up with debug & opt builds
but you will only get crashtests on those builds

How do I know what test/talos suites to request?

We are using a flat file for our list validation list right now. You can see what is in a talos suite here.

bug 589847 has been filed and in the next iteration we will have a way to generate the lists of valid builder info dynamically from config.py which will keep it in sync with try's configs

What do I do if I need something run?

If you missed something in your commit comments you can FILE A BUG with this link to pre-filled in new bug template and replace the template {} tag content with your email-changeset and desired test/talos suites.

eg: Please trigger Callek@gmail.com-6860bd333c84 linux crashtest, no talos

RelEng's buildduty person will see the bug and submit your request.