Jetpack/Testing: Difference between revisions

no edit summary
(Created page with "{{draft|}} This page explains how to run the SDK's unit tests, and how to interpret the results. It's aimed at people who don't have any familiarity with the SDK, but neverthele...")
 
No edit summary
Line 27: Line 27:
So if you navigate to a package's root directory (containing a package.json file) and execute "cfx test", cfx will:
So if you navigate to a package's root directory (containing a package.json file) and execute "cfx test", cfx will:


1) initialize the tests directory using the "tests" value inside "package.json", or the top-level "tests/" or "test" directory
# initialize the tests directory using the "tests" value inside "package.json", or the top-level "tests/" or "test" directory
2) load every module it finds in the tests directory which starts with "test"
# load every module it finds in the tests directory which starts with "test"
3) call each function exported by that module, passing it a [test](http://127.0.0.1:8888/packages/api-utils/docs/unit-test.html) object as an argument.
# call each function exported by that module, passing it a [https://addons.mozilla.org/en-US/developers/docs/sdk/latest/packages/api-utils/docs/unit-test.html test] object as an argument.


### Testing an Add-on ###
=== Testing an Add-on ===


From the SDK root, navigate to "examples/reading-data" and explore the "tests" directory:
From the SDK root, navigate to "examples/reading-data" and explore the "tests" directory:


(addon-sdk)~/mozilla/addon-sdk/examples > cd reading-data/
(addon-sdk)~/mozilla/addon-sdk/examples > cd reading-data/
(addon-sdk)~/mozilla/addon-sdk/examples/reading-data > ls tests
(addon-sdk)~/mozilla/addon-sdk/examples/reading-data > ls tests
test-main.js
test-main.js
(addon-sdk)~/mozilla/addon-sdk/examples/reading-data > more tests/test-main.js  
(addon-sdk)~/mozilla/addon-sdk/examples/reading-data > more tests/test-main.js  
const m = require("main");
const m = require("main");
const self = require("self");
const self = require("self");
exports.testReplace = function(test) {
  const input = "Hello World";
  const output = m.replaceMom(input);
  test.assertEqual(output, "Hello Mom");
  var callbacks = { quit: function() {} };


exports.testReplace = function(test) {
  // Make sure it doesn't crash...
  const input = "Hello World";
  m.main({ staticArgs: {} }, callbacks);
  const output = m.replaceMom(input);
};
  test.assertEqual(output, "Hello Mom");
  var callbacks = { quit: function() {} };
exports.testID = function(test) {
  // The ID is randomly generated during tests, so we cannot compare it against
  // anything in particular.  Just assert that it is not empty.
  test.assert(self.id.length > 0);
  test.assertEqual(self.data.url("sample.html"),
                    "resource://reading-data-example-at-jetpack-dot-mozillalabs-dot-com-reading-data-data /sample.html");
};
So this add-on implements two unit tests. You can see that each test expects a <code>test</code> object as an argument, and that <code>test</code> defines functions like <code>assert</code>, <code>assertEqual</code> and timeout functionality like <code>waitUntilDone</code>. To learn more about what you can do with a test object check the [https://addons.mozilla.org/en-US/developers/docs/sdk/latest/packages/api-utils/docs/unit-test.html api documentation].


  // Make sure it doesn't crash...
If you type <code>cfx test</code> it will execute these two tests. We'll use the <code>-v</code> option to make it show us the test names:
  m.main({ staticArgs: {} }, callbacks);
};


exports.testID = function(test) {
(addon-sdk)~/mozilla/addon-sdk/examples/reading-data > cfx test -v
  // The ID is randomly generated during tests, so we cannot compare it against
Using binary at '/Applications/Firefox.app/Contents/MacOS/firefox-bin'.
  // anything in particular. Just assert that it is not empty.
Using profile at '/var/folders/me/me2B1lFDE0WZCgd33s2OTE+++TU/-Tmp-/tmpASJfvx.mozrunner'.
  test.assert(self.id.length > 0);
Running tests on Firefox 6.0/Gecko 6.0 ({ec8030f7-c20a-464f-9b0e-13a3a9e97384}) under Darwin/x86_64-gcc3.
  test.assertEqual(self.data.url("sample.html"),
info: executing 'test-main.testReplace'
                  "resource://reading-data-example-at-jetpack-dot-mozillalabs-dot-com-reading-data-data/sample.html");
info: pass: a == b == "Hello Mom"
};
info: My ID is reading-data-example@jetpack.mozillalabs.com
info: executing 'test-main.testID'
info: pass: assertion successful
info: pass: a == b == "resource://reading-data-example-at-jetpack-dot-mozillalabs-dot-com-reading-data-data /sample.html"
3 of 3 tests passed.
OK
Total time: 6.058659 seconds
Program terminated successfully.


So this add-on implements two unit tests. You can see that each test expects a `test` object as an argument, and that `test` defines useful functions like `assert`, `assertEqual` and timeout functionality like `waitUntilDone` to test callbacks. TO learn more about what you can do with a test object check the [api documentation](http://127.0.0.1:8888/packages/api-utils/docs/unit-test.html).
=== Testing an SDK Package ===


If you type `cfx test` it will execute these two tests. We'll use the -v option to make it show us the test names:
An SDK package like "addon-kit" or "api-utils" is a package too, so the procedure is the same. Executing <code>cfx test</code> from the root of "addon-kit" will execute all the following test suites:


(addon-sdk)~/mozilla/addon-sdk/examples/reading-data > cfx test -v
(addon-sdk)~/mozilla/addon-sdk >
Using binary at '/Applications/Firefox.app/Contents/MacOS/firefox-bin'.
(addon-sdk)~/mozilla/addon-sdk > cd packages/addon-kit/
Using profile at '/var/folders/me/me2B1lFDE0WZCgd33s2OTE+++TU/-Tmp-/tmpASJfvx.mozrunner'.
(addon-sdk)~/mozilla/addon-sdk/packages/addon-kit > ls tests
Running tests on Firefox 6.0/Gecko 6.0 ({ec8030f7-c20a-464f-9b0e-13a3a9e97384}) under Darwin/x86_64-gcc3.
pagemod-test-helpers.js  test-notifications.js  test-request.js
info: executing 'test-main.testReplace'
test-clipboard.js  test-page-mod.js  test-selection.js
info: pass: a == b == "Hello Mom"
test-context-menu.html  test-page-worker.js  test-simple-storage.js
info: My ID is reading-data-example@jetpack.mozillalabs.com
test-context-menu.js  test-panel.js  test-tabs.js
info: executing 'test-main.testID'
test-hotkeys.js  test-passwords.js  test-widget.js
info: pass: assertion successful
test-module.js  test-private-browsing.js  test-windows.js
info: pass: a == b == "resource://reading-data-example-at-jetpack-dot-mozillalabs-dot-com-reading-data-data/sample.html"


3 of 3 tests passed.
This isn't ideal if you only want to test a single module, so you can use the <code>-f</code> option to cfx to filter the set of tests to run. This option takes a regex and runs only the test suites whose names match the regex. This example runs only the tests for the "clipboard" module:
OK
Total time: 6.058659 seconds
Program terminated successfully.


### Testing an SDK Package ###
(addon-sdk)~/mozilla/addon-sdk/packages/addon-kit > cfx test -f clipboard -v
Using binary at '/Applications/Firefox.app/Contents/MacOS/firefox-bin'.
Using profile at '/var/folders/me/me2B1lFDE0WZCgd33s2OTE+++TU/-Tmp-/tmpVLFIqe.mozrunner'.
Running tests on Firefox 6.0/Gecko 6.0 ({ec8030f7-c20a-464f-9b0e-13a3a9e97384}) under Darwin/x86_64-gcc3.
info: executing 'test-clipboard.testWithNoFlavor'
info: pass: assertion successful
info: pass: a == b == "text"
info: pass: a == b == "hello there"
info: pass: a == b == "hello there"
info: pass: a == b == "hello there"
info: executing 'test-clipboard.testWithFlavor'
info: pass: assertion successful
info: pass: a == b == "html"
info: pass: a == b == null
info: pass: a == b == "<b>hello there</b>"
info: pass: a == b == "<b>hello there</b>"
info: executing 'test-clipboard.testWithRedundantFlavor'
info: pass: assertion successful
info: pass: a == b == "text"
info: pass: a == b == "<b>hello there</b>"
info: pass: a == b == "<b>hello there</b>"
info: pass: a == b == "<b>hello there</b>"
info: executing 'test-clipboard.testNotInFlavor'
info: pass: assertion successful
info: pass: a == b == null
17 of 17 tests passed.
OK
Total time: 2.909257 seconds
Program terminated successfully.


An SDK package like "addon-kit" or "api-utils" is a package too, so the procedure is the same. Executing 'cfx test' from the root of "addon-kit" will execute all the following test suites:
=== cfx testcfx ===
 
(addon-sdk)~/mozilla/addon-sdk >
(addon-sdk)~/mozilla/addon-sdk > cd packages/addon-kit/
(addon-sdk)~/mozilla/addon-sdk/packages/addon-kit > ls tests
pagemod-test-helpers.js  test-notifications.js  test-request.js
test-clipboard.js  test-page-mod.js  test-selection.js
test-context-menu.html  test-page-worker.js  test-simple-storage.js
test-context-menu.js  test-panel.js  test-tabs.js
test-hotkeys.js  test-passwords.js  test-widget.js
test-module.js  test-private-browsing.js  test-windows.js
 
This isn't ideal if you only want to test a single module, so you can use the "-f" option to cfx to filter the set of tests to run. This option takes a regex and runs only the test suites whose names match the regex. This example runs only the tests for the "clipboard" module:
 
(addon-sdk)~/mozilla/addon-sdk/packages/addon-kit > cfx test -f clipboard -v
Using binary at '/Applications/Firefox.app/Contents/MacOS/firefox-bin'.
Using profile at '/var/folders/me/me2B1lFDE0WZCgd33s2OTE+++TU/-Tmp-/tmpVLFIqe.mozrunner'.
Running tests on Firefox 6.0/Gecko 6.0 ({ec8030f7-c20a-464f-9b0e-13a3a9e97384}) under Darwin/x86_64-gcc3.
info: executing 'test-clipboard.testWithNoFlavor'
info: pass: assertion successful
info: pass: a == b == "text"
info: pass: a == b == "hello there"
info: pass: a == b == "hello there"
info: pass: a == b == "hello there"
info: executing 'test-clipboard.testWithFlavor'
info: pass: assertion successful
info: pass: a == b == "html"
info: pass: a == b == null
info: pass: a == b == "<b>hello there</b>"
info: pass: a == b == "<b>hello there</b>"
info: executing 'test-clipboard.testWithRedundantFlavor'
info: pass: assertion successful
info: pass: a == b == "text"
info: pass: a == b == "<b>hello there</b>"
info: pass: a == b == "<b>hello there</b>"
info: pass: a == b == "<b>hello there</b>"
info: executing 'test-clipboard.testNotInFlavor'
info: pass: assertion successful
info: pass: a == b == null
 
17 of 17 tests passed.
OK
Total time: 2.909257 seconds
Program terminated successfully.
 
## cfx testcfx ##
canmove, Confirmed users
737

edits