WebExtensions/Try Server: Difference between revisions

Jump to navigation Jump to search

WebExtensions/Try Server (view source)

Revision as of 14:55, 10 June 2025

3,150 bytes removed ,  10 June
Remove mentions of classic try syntax, as it is no longer supported: https://groups.google.com/a/mozilla.org/g/dev-platform/c/OtwGNKEUXew/m/MEXTRW3aAQAJ ; also replace ini with toml
(Add mach try fuzzy)
(Remove mentions of classic try syntax, as it is no longer supported: https://groups.google.com/a/mozilla.org/g/dev-platform/c/OtwGNKEUXew/m/MEXTRW3aAQAJ ; also replace ini with toml)
 
Line 5: Line 5:


Try jobs can be scheduled via the web interface or via the "mach try" command.
Try jobs can be scheduled via the web interface or via the "mach try" command.
The following platforms (-p) are of interest:
* win32,win64
* linux,linux64
* macosx64
* android-x86


Symbols for the following [https://web.archive.org/web/20191005073152/https://developer.mozilla.org/en-US/docs/Mozilla/QA/Automated_testing#In_production test suites] are shown in the test results on Treeherder. Test suite identifiers for use with "-t" in "mach try" are listed below the main bullet points:
Symbols for the following [https://web.archive.org/web/20191005073152/https://developer.mozilla.org/en-US/docs/Mozilla/QA/Automated_testing#In_production test suites] are shown in the test results on Treeherder. Test suite identifiers for use with "-t" in "mach try" are listed below the main bullet points:
Line 27: Line 20:
Pass --artifact to speed up builds (only if your changes do not require recompilation of C++/Rust code, such as Java (Android) and JavaScript-only changes).
Pass --artifact to speed up builds (only if your changes do not require recompilation of C++/Rust code, such as Java (Android) and JavaScript-only changes).


Tests can be run in a debug build ("-b d"), an optimized ("-b o") or both ("-b do"). It is recommended to run both, because together they can catch more potential failures:
Tests can be run in a debug build, an optimized build, or both. It is recommended to run both, because together they can catch more potential failures:


* Some tests are only enabled in one of the two build types because of intermittent/permanent test failures ("skip-if" in the .ini files that describe the tests).
* Some tests are only enabled in one of the two build types because of intermittent/permanent test failures ("skip-if" in the test manifests (.toml files) that describe the tests).
* Debug builds usually run slower than optimized builds, and reveal weaknesses in tests (e.g. time-sensitive race conditions).
* Debug builds usually run slower than optimized builds, and reveal weaknesses in tests (e.g. time-sensitive race conditions).
* Debug builds have additional checks (e.g. validation of extension API schema against the actual implementation).
* Debug builds have additional checks (e.g. validation of extension API schema against the actual implementation).
Line 36: Line 29:


By default, "mach try" is an alias for [https://firefox-source-docs.mozilla.org/tools/try/selectors/auto.html "mach try auto"], which attempts to select some relevant tests. If unsure about what to run, "mach try" is recommended.
By default, "mach try" is an alias for [https://firefox-source-docs.mozilla.org/tools/try/selectors/auto.html "mach try auto"], which attempts to select some relevant tests. If unsure about what to run, "mach try" is recommended.
To have more control over the tests to run, the newer [https://firefox-source-docs.mozilla.org/tools/try/selectors/fuzzy.html "mach try fuzzy"] or the classic [https://firefox-source-docs.mozilla.org/tools/try/selectors/syntax.html "mach try syntax"] can be used instead.
To have more control over the tests to run, [https://firefox-source-docs.mozilla.org/tools/try/selectors/fuzzy.html "mach try fuzzy"] can be used instead.


=== mach try fuzzy ===
=== mach try fuzzy ===
Line 50: Line 43:


If in doubt about what to use in the query (<nowiki>-q</nowiki>), run <nowiki>./mach try fuzzy --no-push</nowiki> to launch an interactive program that displays the matched jobs.
If in doubt about what to use in the query (<nowiki>-q</nowiki>), run <nowiki>./mach try fuzzy --no-push</nowiki> to launch an interactive program that displays the matched jobs.
=== mach try syntax ===
Examples of "mach try syntax" commands (add "--no-push" to see the generated try syntax without actually triggering the try job):
* Run all relevant tests. This is expensive and should only be used when you don't know which tests should be run, because it also includes many non-extension tests.
  <nowiki>mach try syntax -b do -p win64,linux64,macosx64,android-x86 -u mochitests,xpcshell,test-verify --artifact</nowiki>
* Run all relevant tests, without non-extension tests. The command with the `--tag=webextensions` flag uses your local checkout and .mozconfig configuration to identify the tests. Some Android-only tests may not run if your current configuration targets desktop.
  <nowiki>mach try syntax -b do -p win64,linux64,macosx64,android-x86 -u test-verify --artifact --tag=webextensions</nowiki>
* Run most extension tests (xpcshell, plain mochitests and browser-chrome extension tests (and many other mochitests/browser-chrome tests, but at least less expensive than just "mochitests"). The mochitest-bc test type is only used on desktop, so Android was omitted from the platform flag ("-p").
  <nowiki>mach try syntax -b do -p win64,linux64,macosx64 -u xpcshell,mochitest,mochitest-bc,test-verify --artifact</nowiki>
* Run extension tests on Android (requires a [https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Simple_Firefox_for_Android_build#Prepare_a_Firefox_for_Android_mozconfig_file mozconfig file for Android]; otherwise tests in mobile/android/components/extensions/test/ will not be run. The MOZCONFIG environment variable is used to select a different mozconfig file, so that you can develop for multiple platforms with a single checkout):
  <nowiki>MOZCONFIG=/path/to/mozconfig-for-android mach try syntax -b do -p android-x86 --artifact --tag=webextensions</nowiki>
It may take several seconds before a "mach try syntax" command with the "--tag=webextensions" flag completes. If you already know the directory that contains the test, it is recommended to use the directory instead of "--tag=webextensions", e.g.:
* Run tests in specific directories for desktop:
  <nowiki>mach try syntax -b do -p win64,linux64,macosx64 -u mochitests,mochitest-bc,xpcshell --artifact --and toolkit/components/extensions/test/ browser/components/extensions/test/</nowiki>
* Run tests in specific directories for Android:
  <nowiki>MOZCONFIG=/path/to/mozconfig-for-android mach try syntax -b do -p android-x86 -u mochitests,xpcshell --artifact --and toolkit/components/extensions/test/ mobile/android/components/extensions/test/</nowiki>
By default, a test that match any specified test tags ("--tag") or suite ("-u") will be scheduled. To only run tests that match all specified rules, include "--and" in the try syntax.
When the "--and" flag is used, the set of tests is computed locally. To see what tests are going to be scheduled, add "--no-push".


== Test failures ==
== Test failures ==
Robwu
47

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1254321"

Navigation menu