SoftwareTesting:Tools:SimpleXulTestHarness

From MozillaWiki
Jump to: navigation, search

Description

davel used this set of scripts to help verify that the default search engine plugins bundled with localized builds of Firefox 2.0b2 work as expected.

The wrapper script extract_search.sh expects paths to Firefox disk image (.dmg) files on the command line. It installs Firefox from each disk image, creates a new profile, runs the "test" to list the search engine stuff, then deletes the profile and that instance of Firefox.

The wrapper script makes the following assumptions:

  • Running on Mac OS X
  • APP_DATA_DIR set to ~/Library/Application Support/Firefox
    • davel keeps a symlink to that directory from his home directory
  • SEARCHTEST_DIR set to directory containing all these files
  • WORK_DIR set to a writable directory, use to install Firefox and hold the newly-created profile
  • tests.manifest contains a file: url link to the SEARCHTEST_DIR directory
    • A possible future improvement: we could make extract_search.sh edit the manifest file on the fly
  • Nothing is creating or deleting profiles while this script is running
    • Changes to profiles.ini made by other things while this script is running will be removed

How to run the test

  1. Download the contents of this directory to a directory on your Macintosh
  2. Edit extract_search.sh and tests.manifest, replacing paths with the ones on your machine
  3. Download the Firefox disk image files you want to "test"
  4. Invoke extract_search.sh

Example

./extract_search.sh "en-US/Firefox 2.0 Beta 2.dmg" "en-GB/Firefox 2.0 Beta 2.dmg"

Output

en-US/Firefox 2.0 Beta 2.dmg
Agree Y/N? "disk1" unmounted.
"disk1" ejected.
Success: created profile 'sbtestprofile /tmp/profile'
1|Google|http://www.google.com/search?q=foofoo&ie=utf-8&oe=utf-8&rls=org.mozilla:en-US:official&client=firefox-a
2|Yahoo|http://search.yahoo.com/search?p=foofoo&ei=UTF-8&fr=moz2
3|Amazon.com|http://www.amazon.com/s/002-3363096-7452819?ie=UTF8&tag=mozilla-20&index=blended&link%5Fcode=qs&field-keywords=foofoo&sourceid=Mozilla-search
### MRJPlugin:  getPluginBundle() here. ###
### MRJPlugin:  CFBundleGetBundleWithIdentifier() succeeded. ###
### MRJPlugin:  CFURLGetFSRef() succeeded. ###
4|Answers.com|http://www.answers.com/main/ntquery?s=foofoo&gwp=13
5|Creative Commons|http://search.creativecommons.org/?q=foofoo&sourceid=Mozilla-search
6|eBay|http://search.ebay.com/foofoo_W0QQfrppZ50QQfsopZ1QQmaxrecordsreturnedZ300
en-GB/Firefox 2.0 Beta 2.dmg
Agree Y/N? "disk1" unmounted.
"disk1" ejected.
Success: created profile 'sbtestprofile /tmp/profile'
1|Google|http://www.google.com/search?q=foofoo&ie=utf-8&oe=utf-8&rls=org.mozilla:en-GB:official&client=firefox-a
2|Yahoo.co.uk|http://uk.search.yahoo.com/search?p=foofoo&ei=UTF-8&fr=moz2
3|Amazon.co.uk|http://www.amazon.co.uk/s/202-2144193-3391812?ie=UTF8&tag=firefox-uk-21&index=blended&link%5Fcode=qs&field-keywords=foofoo&sourceid=Mozilla-search
### MRJPlugin:  getPluginBundle() here. ###
### MRJPlugin:  CFBundleGetBundleWithIdentifier() succeeded. ###
### MRJPlugin:  CFURLGetFSRef() succeeded. ###
4|Answers.com|http://www.answers.com/main/ntquery?s=foofoo&gwp=13
5|Chambers (UK)|http://www.chambersharrap.co.uk/chambers/features/chref/chref.py/main?query=foofoo&title=21st&sourceid=Mozilla-search
6|Creative Commons|http://search.creativecommons.org/?q=foofoo&sourceid=Mozilla-search
7|eBay.co.uk|http://search.ebay.co.uk/foofoo_W0QQfrppZ50QQfsopZ1QQmaxrecordsreturnedZ300

Comments

In the example, extract_search.sh demonstrates how to create a profile. This takes two steps:

  1. CreateProfile makes the directory and prefs.js file.
  2. The first run populates the profile directory with all the other data.

The sleep 5 is necessary because Firefox (at least on the Mac) restarts itself during the first-run process when the Extension Manager checks things out.

In addition, this example doesn't cover installing an extension.

To do list

  • Factor out the generic harness bits from the search engine lister bits
  • Figure out where/how to check this in to the tree
    • As an extension, to populate chrome://tests/?
    • How can test stuff from other parts of the tree get installed into chrome://tests/
  • Add invocation hooks
    • What test to run
    • Auto-discovery of tests?
    • Command-line pointer to test manifest file, like layout reftest?
  • Add test result reporting
    • Simple dump statements
    • Maybe a chrome test runner with progress bar, like jsunit's test runner
  • Make this work for already-installed Firefox
  • Make this work on Windows and Linux
  • Clean up output of search engine lister bits