SoftwareTesting:Tools:SimpleXulTestHarness

Revision as of 19:08, 15 September 2006 by Davel (talk | contribs) (→‎to do list)

description

davel used this set of scripts to help verify that the default search engine plugins bundled with localised builds of firefox 2.0b2 were as expected.

The wrapper script extract_search.sh expects paths to firefox dmg files on the command line. It installs firefox from each dmg, creates a new profile, runs the "test" to list the search engine stuff, then deletes the profile and that instance of firefox.

The wrapper script has the following assumptions:

  • running on a mac
  • APP_DATA_DIR set to ~/Library/Application Support/Firefox
    • on my machine, I have a symlink to that dir from my home dir
  • SEARCHTEST_DIR set to directory containing all these files
  • WORK_DIR set to a writable directory, use to install firefox and hold the newly-created profile
  • tests.manifest contains a file: url link to the SEARCHTEST_DIR directory
    • hmm, we could make extract_search.sh edit the manifest file on the fly . . .
  • nothing is creating or deleting profiles while this script is running
    • changes to profiles.ini made by other things while this script is running will be clobbered.

how to run

  1. download contents of this directory to a directory on your mac
  2. edit extract_search.sh and tests.manifest, replacing paths with the ones on your machine
  3. download firefox .dmg files you want to "test"
  4. invoke extract_search.sh

example

 ./extract_search.sh "en-US/Firefox 2.0 Beta 2.dmg" "en-GB/Firefox 2.0 Beta 2.dmg"

output from above example

en-US/Firefox 2.0 Beta 2.dmg
Agree Y/N? "disk1" unmounted.
"disk1" ejected.
Success: created profile 'sbtestprofile /tmp/profile'
1|Google|http://www.google.com/search?q=foofoo&ie=utf-8&oe=utf-8&rls=org.mozilla:en-US:official&client=firefox-a
2|Yahoo|http://search.yahoo.com/search?p=foofoo&ei=UTF-8&fr=moz2
3|Amazon.com|http://www.amazon.com/s/002-3363096-7452819?ie=UTF8&tag=mozilla-20&index=blended&link%5Fcode=qs&field-keywords=foofoo&sourceid=Mozilla-search
### MRJPlugin:  getPluginBundle() here. ###
### MRJPlugin:  CFBundleGetBundleWithIdentifier() succeeded. ###
### MRJPlugin:  CFURLGetFSRef() succeeded. ###
4|Answers.com|http://www.answers.com/main/ntquery?s=foofoo&gwp=13
5|Creative Commons|http://search.creativecommons.org/?q=foofoo&sourceid=Mozilla-search
6|eBay|http://search.ebay.com/foofoo_W0QQfrppZ50QQfsopZ1QQmaxrecordsreturnedZ300
en-GB/Firefox 2.0 Beta 2.dmg
Agree Y/N? "disk1" unmounted.
"disk1" ejected.
Success: created profile 'sbtestprofile /tmp/profile'
1|Google|http://www.google.com/search?q=foofoo&ie=utf-8&oe=utf-8&rls=org.mozilla:en-GB:official&client=firefox-a
2|Yahoo.co.uk|http://uk.search.yahoo.com/search?p=foofoo&ei=UTF-8&fr=moz2
3|Amazon.co.uk|http://www.amazon.co.uk/s/202-2144193-3391812?ie=UTF8&tag=firefox-uk-21&index=blended&link%5Fcode=qs&field-keywords=foofoo&sourceid=Mozilla-search
### MRJPlugin:  getPluginBundle() here. ###
### MRJPlugin:  CFBundleGetBundleWithIdentifier() succeeded. ###
### MRJPlugin:  CFURLGetFSRef() succeeded. ###
4|Answers.com|http://www.answers.com/main/ntquery?s=foofoo&gwp=13
5|Chambers (UK)|http://www.chambersharrap.co.uk/chambers/features/chref/chref.py/main?query=foofoo&title=21st&sourceid=Mozilla-search
6|Creative Commons|http://search.creativecommons.org/?q=foofoo&sourceid=Mozilla-search
7|eBay.co.uk|http://search.ebay.co.uk/foofoo_W0QQfrppZ50QQfsopZ1QQmaxrecordsreturnedZ300

comments

  • in the example, extract_search.sh demonstrates how to create a profile
    • 2 steps - CreateProfile makes the directory and prefs.js file, and the first run populates the profile directory with all the other stuff
  • the sleep 5 is necessary because firefox (on mac at least) restarts itself during the first-run process. probably extension manager checking for stuff
  • installing an extension is not covered by this example

to do list

  • factor out the generic harness bits from the search engine lister bits
  • figure out where/how to check this in to the tree
    • as an extension, to populate chrome://tests/ ?
    • how can test stuff from other parts of the tree get installed into chrome://tests/
  • add invocation hooks
    • what test to run
    • auto-discovery of tests?
    • command -line pointer to test manifest file, like layout reftest?
  • add test result reporting
    • simple dump statements
    • maybe a chrome test runner with progress bar, like jsunit's test runner
  • make this work for already-installed firefox
  • make this work on windows and linux
  • clean up output of search engine lister bits