Visit Mozilla.org

SoftwareTesting:Tools:SimpleXulTestHarness

From MozillaWiki

Contents

Description [edit]

davel used this set of scripts to help verify that the default search engine plugins bundled with localized builds of Firefox 2.0b2 work as expected.

The wrapper script extract_search.sh expects paths to Firefox disk image (.dmg) files on the command line. It installs Firefox from each disk image, creates a new profile, runs the "test" to list the search engine stuff, then deletes the profile and that instance of Firefox.

The wrapper script makes the following assumptions:

  • Running on Mac OS X
  • APP_DATA_DIR set to ~/Library/Application Support/Firefox
    • davel keeps a symlink to that directory from his home directory
  • SEARCHTEST_DIR set to directory containing all these files
  • WORK_DIR set to a writable directory, use to install Firefox and hold the newly-created profile
  • tests.manifest contains a file: url link to the SEARCHTEST_DIR directory
    • A possible future improvement: we could make extract_search.sh edit the manifest file on the fly
  • Nothing is creating or deleting profiles while this script is running
    • Changes to profiles.ini made by other things while this script is running will be removed

How to run the test [edit]

  1. Download the contents of this directory to a directory on your Macintosh
  2. Edit extract_search.sh and tests.manifest, replacing paths with the ones on your machine
  3. Download the Firefox disk image files you want to "test"
  4. Invoke extract_search.sh

Example [edit]

./extract_search.sh "en-US/Firefox 2.0 Beta 2.dmg" "en-GB/Firefox 2.0 Beta 2.dmg"

Output [edit]

en-US/Firefox 2.0 Beta 2.dmg
Agree Y/N? "disk1" unmounted.
"disk1" ejected.
Success: created profile 'sbtestprofile /tmp/profile'
1|Google|http://www.google.com/search?q=foofoo&ie=utf-8&oe=utf-8&rls=org.mozilla:en-US:official&client=firefox-a
2|Yahoo|http://search.yahoo.com/search?p=foofoo&ei=UTF-8&fr=moz2
3|Amazon.com|http://www.amazon.com/s/002-3363096-7452819?ie=UTF8&tag=mozilla-20&index=blended&link%5Fcode=qs&field-keywords=foofoo&sourceid=Mozilla-search
### MRJPlugin:  getPluginBundle() here. ###
### MRJPlugin:  CFBundleGetBundleWithIdentifier() succeeded. ###
### MRJPlugin:  CFURLGetFSRef() succeeded. ###
4|Answers.com|http://www.answers.com/main/ntquery?s=foofoo&gwp=13
5|Creative Commons|http://search.creativecommons.org/?q=foofoo&sourceid=Mozilla-search
6|eBay|http://search.ebay.com/foofoo_W0QQfrppZ50QQfsopZ1QQmaxrecordsreturnedZ300
en-GB/Firefox 2.0 Beta 2.dmg
Agree Y/N? "disk1" unmounted.
"disk1" ejected.
Success: created profile 'sbtestprofile /tmp/profile'
1|Google|http://www.google.com/search?q=foofoo&ie=utf-8&oe=utf-8&rls=org.mozilla:en-GB:official&client=firefox-a
2|Yahoo.co.uk|http://uk.search.yahoo.com/search?p=foofoo&ei=UTF-8&fr=moz2
3|Amazon.co.uk|http://www.amazon.co.uk/s/202-2144193-3391812?ie=UTF8&tag=firefox-uk-21&index=blended&link%5Fcode=qs&field-keywords=foofoo&sourceid=Mozilla-search
### MRJPlugin:  getPluginBundle() here. ###
### MRJPlugin:  CFBundleGetBundleWithIdentifier() succeeded. ###
### MRJPlugin:  CFURLGetFSRef() succeeded. ###
4|Answers.com|http://www.answers.com/main/ntquery?s=foofoo&gwp=13
5|Chambers (UK)|http://www.chambersharrap.co.uk/chambers/features/chref/chref.py/main?query=foofoo&title=21st&sourceid=Mozilla-search
6|Creative Commons|http://search.creativecommons.org/?q=foofoo&sourceid=Mozilla-search
7|eBay.co.uk|http://search.ebay.co.uk/foofoo_W0QQfrppZ50QQfsopZ1QQmaxrecordsreturnedZ300

Comments [edit]

In the example, extract_search.sh demonstrates how to create a profile. This takes two steps:

  1. CreateProfile makes the directory and prefs.js file.
  2. The first run populates the profile directory with all the other data.

The sleep 5 is necessary because Firefox (at least on the Mac) restarts itself during the first-run process when the Extension Manager checks things out.

In addition, this example doesn't cover installing an extension.

To do list [edit]

  • Factor out the generic harness bits from the search engine lister bits
  • Figure out where/how to check this in to the tree
    • As an extension, to populate chrome://tests/?
    • How can test stuff from other parts of the tree get installed into chrome://tests/
  • Add invocation hooks
    • What test to run
    • Auto-discovery of tests?
    • Command-line pointer to test manifest file, like layout reftest?
  • Add test result reporting
    • Simple dump statements
    • Maybe a chrome test runner with progress bar, like jsunit's test runner
  • Make this work for already-installed Firefox
  • Make this work on Windows and Linux
  • Clean up output of search engine lister bits
Retrieved from "http://wiki.mozilla.org/SoftwareTesting:Tools:SimpleXulTestHarness"