canmove, Confirmed users
1,570
edits
Jetpack/Testing: Difference between revisions
Jump to navigation
Jump to search
Update about test manifests
No edit summary |
(Update about test manifests) |
||
| (16 intermediate revisions by 5 users not shown) | |||
| Line 1: | Line 1: | ||
This page explains how to run the SDK's unit tests, and how to interpret the results. It's aimed at people who don't have any familiarity with the SDK, but nevertheless have to run the tests to diagnose any test failures, for example because of regressions introduced by the underlying platform. | This page explains how to run the SDK's unit tests, and how to interpret the results. It's aimed at people who don't have any familiarity with the SDK, but nevertheless have to run the tests to diagnose any test failures, for example because of regressions introduced by the underlying platform. | ||
For any help with these tests come to the [irc://irc.mozilla.org/#jetpack #jetpack channel] and ask the team, Gozala is the module owner. | |||
You can run tests using the <code>cfx</code> command- | == Running tests from compiled Firefox == | ||
You have two options to run tests from a built version of Firefox. Both do basically the same thing and are quite limited right now, if you want more functionality (like being able to select specific tests) you'll need to checkout the SDK repository separately and follow the later instructions. | |||
=== Running with mach === | |||
From the source directory or object directory run: | |||
mach mochitest addon-sdk/source | |||
You can also run specific tests like so: | |||
mach mochitest addon-sdk/source/test/test-sandbox.js | |||
or possibly even: | |||
mach mochitest test-sandbox.js | |||
if the name doesn't collide with any other tests. | |||
=== Running with make === | |||
Don't. | |||
== Disabling tests == | |||
Tests can be disabled using the regular mochitest ini manifest. See [https://dxr.mozilla.org/mozilla-central/source/addon-sdk/source/test/jetpack-package.ini jetpack-package.ini] and [https://dxr.mozilla.org/mozilla-central/source/addon-sdk/source/test/addons/jetpack-addon.ini jetpack-addon.ini]. | |||
== Running tests from the SDK repository == | |||
First, get the SDK from git: | |||
git clone git://github.com/mozilla/addon-sdk.git | |||
cd addon-sdk | |||
Or go into the SDK directory in mozilla-central: | |||
cd addon-sdk | |||
Activate the SDK (this isn't necessary but puts the cfx tool into your path, you can instead run it from addon-sdk/bin/cfx): | |||
source bin/activate | |||
To run all tests: | |||
cfx testall | |||
To run only the API tests (typically the ones that break when Firefox changes), invoke <code>testpkgs</code> instead of <code>testall</code>: | |||
cfx testpkgs | |||
To run only certain test suites, specify the <code>--filter</code> flag with a regular expression that matches the filenames of the test suites you want to run: | |||
cfx testpkgs --filter panel | |||
=== Specifying the Firefox Build === | |||
To run tests on a specific build of Firefox, specify the build via the <code>--binary</code> flag: | |||
cfx testall --binary /Applications/Nightly.app/ | |||
cfx testall --binary ~/Mozilla/source/obj-dir/dist/bin/firefox | |||
cfx testall --binary /c/Program\ Files\Nightly\firefox.exe | |||
=== Attaching a Debugger === | |||
To attach gdb before running tests, specify the <code>--no-run</code> flag: | |||
cfx testpkgs --no-run | |||
<code>--no-run</code> doesn't actually run the command, it just prints out the shell command that would be invoked in order to run it. So the above command would produce output like: | |||
To launch the application, enter the following command: | |||
/Applications/Firefox.app/Contents/MacOS/firefox-bin -profile /var/folder | |||
/me/me2B1lFDE0WZCgd33s2OTE+++TU/-Tmp-/tmp72nW_f.mozrunner -foreground -no-remote | |||
You can then run the debugger using that command: | |||
gdb --args /Applications/Firefox.app/Contents/MacOS/firefox-bin -profile /var/folder | |||
/me/me2B1lFDE0WZCgd33s2OTE+++TU/-Tmp-/tmp72nW_f.mozrunner -foreground -no-remote | |||
== Gory Details of Testing With cfx == | |||
There's [https://addons.mozilla.org/en-US/developers/docs/sdk/latest/dev-guide/addon-development/cfx-tool.html cfx documentation here], and command line help is available by typing: | |||
cfx --help | cfx --help | ||
| Line 17: | Line 100: | ||
testall - test whole environment | testall - test whole environment | ||
== cfx test == | === cfx test === | ||
<code>cfx test</code> runs the tests implemented for a single SDK "package". A package is a collection of JavaScript modules, and is defined as such by the presence of a file called "package.json" in its root directory. Among other things, "package.json" includes an string called "tests" which describes where the package's test suites live. If this is not defined, the default is the "tests" or "test" directory under the package root. | <code>cfx test</code> runs the tests implemented for a single SDK "package". A package is a collection of JavaScript modules, and is defined as such by the presence of a file called "package.json" in its root directory. Among other things, "package.json" includes an string called "tests" which describes where the package's test suites live. If this is not defined, the default is the "tests" or "test" directory under the package root. | ||
| Line 29: | Line 112: | ||
An add-on is a package, and the SDK's APIs are currently collected into two packages called "addon-kit" and "api-utils" (you can see them under the top-level "packages" directory). | An add-on is a package, and the SDK's APIs are currently collected into two packages called "addon-kit" and "api-utils" (you can see them under the top-level "packages" directory). | ||
=== Testing an Add-on === | ==== Testing an Add-on ==== | ||
From the SDK root, navigate to "examples/reading-data" and explore the "tests" directory: | From the SDK root, navigate to "examples/reading-data" and explore the "tests" directory: | ||
| Line 78: | Line 161: | ||
Program terminated successfully. | Program terminated successfully. | ||
=== Testing an SDK Package === | ==== Testing an SDK Package ==== | ||
An SDK package like "addon-kit" or "api-utils" is a package too, so the procedure is the same. Executing <code>cfx test</code> from the root of "addon-kit" will execute all the following test suites: | An SDK package like "addon-kit" or "api-utils" is a package too, so the procedure is the same. Executing <code>cfx test</code> from the root of "addon-kit" will execute all the following test suites: | ||
| Line 125: | Line 208: | ||
Program terminated successfully. | Program terminated successfully. | ||
== cfx testcfx == | === cfx testcfx === | ||
<code>cfx testcfx</code> tests the <code>cfx</code> tool itself. <code>cfx</code> is written in Python, and <code>testcfx</code> uses Python's <code>unittest</code> framework. | <code>cfx testcfx</code> tests the <code>cfx</code> tool itself. <code>cfx</code> is written in Python, and <code>testcfx</code> uses Python's <code>unittest</code> framework. | ||
| Line 153: | Line 236: | ||
It also looks for [http://docs.python.org/library/doctest.html DocTests] in the documentation files, and executes any that it finds. But there aren't any, at the moment. | It also looks for [http://docs.python.org/library/doctest.html DocTests] in the documentation files, and executes any that it finds. But there aren't any, at the moment. | ||
== cfx testex == | === cfx testex === | ||
<code>cfx | <code>cfx testex</code> runs <code>cfx test</code> for all the add-ons which ship with the SDK under the examples/ directory. | ||
== cfx testpkgs == | === cfx testpkgs === | ||
<code>cfx testpkgs</code> finds all packages installed under the packages/ directory, and executes <code>cfx test</code> for each package it finds. | <code>cfx testpkgs</code> finds all packages installed under the packages/ directory, and executes <code>cfx test</code> for each package it finds. | ||
== cfx testall == | === cfx testall === | ||
<code>cfx testall</code> tests everything: | <code>cfx testall</code> tests everything: | ||