User:Lukasblakk/TryServerSyntax

Syntax for Custom Try Requests

The goal of this syntax is to allow user to select particular build/unittest/talos combinations in their hg commit message or in pushed config.info file.

Syntax

try: --build [d,o,do,all] --p [desktop_platforms] --m [mobile_platforms] --u [unittests] --t [talos_suites]

This covers:

• Build Types both(do) || opt(o) || debug(d) || m-c all override (all)
• Platforms none || --p {linux,linux64,osx,osx64,win32,win64}
• Mobile none || --m {mobile-maemo5-qt, android, etc}
• Unittest Suites all || --u {crashtest,reftest,mochitest{1,2,3,4,5},xpcshell,etc}
• Talos Suites all || --t {tp4,tscroll,tspider,etc}

Order

Specify build types (opt/debug/both) first then either all platforms or specify what you want then all unittest suites, no unittest suites, or specify a suite (then run for each platform built) then all talos suites, no talos suites, or specify suite (then run for each platform built)

This allows us to cut off at any point and just to a default 'None' for any info not submitted. If the user submits just a build type, they get a "Does is compile?" try run.

Problem with this syntax

Does the test specified for all the builds on a platform, not specific to opt/debug builder combination (eg. doing crashtest for opt build and not debug build) this could potentially be addressed in a future iteration if that level of granularity is needed to help with wait times.

Usage Examples

In an hg commit message

 # to get a complete run that mimics mozilla-central's automation
 hg commit -m "bug 432432 testing something try: --build all" 

 # both opt/debug builds, all available {desktop,mobile} platforms, no unittests/talos
 hg commit -m "bug 432432 testing something try: --build do" 

 # linux debug build, all available mobile platforms, all unittests, no talos
 hg commit -m "bug 432432 testing something try: --build d --p linux --u all"

 # mobile-only builds
 hg commit -m "bug 432432 testing something try: --build o --p none"

 # all builds, all available desktop platforms, no mobile, all unittests & talos
 hg commit -m "bug 432432 testing something try: --build do --m none --u all --t all"

 # opt builds on all available {desktop,mobile} platforms, no unittests, talos tp4 & tscroll 
 hg commit -m "bug 432432 testing something try: --build o --t tp4,tscroll"

In a try.info file

try.info file pushed to $topsrc_dir would look like:

try: --build b --m none --u all --t all

Scheduler/Builder Parsing

• split on 'try:'
• then split on ' ' to create args to pass to argparse

Default Set

If you only specify try: or no commit comment is present (or info file) then the default set would be a "Does it compile?" set comprised of:

 All available platforms' desktop debug and all available mobile builds no tests, no talos

Optional arguments

If the following exist they will be handled as follows:

build (default is debug for now)

Options are o (opt) || d (debug) || b (gets you both)
We use this to build the desktop and test builder names

desktop_platforms (if not present default is all available)

Now we can create a builderName for each combo of opt/debug platform builders specified

mobile_platforms (if not present default is all available)

Now we can create a builderName for each mobile platform builder specified

unittests (if not present default is none)

--u array is used to add test builderNames (if ['all'] then all available testBuilders will be added for that platform ie: no mac a11y if that's not a test suite on mac)

talos (if not present default is none)

--t array is used to add talos builderNames (if ['all'] then all available talosBuilders will be added for that platform)