The update testing was also ported and it is being integrated as part of the automated release process ({{bug|1148546}}).
The tests are harness is being integrated inside of developed as a mozharness script [https://github.com/mozillaarmenzg/build-toolsmozharness/blob/masterupdate_testing/releasescripts/updates/verifyfirefox_ui_updates.sh verifypy firefox_ui_updates.shpy] (not yet).
= Temporary steps =
Until the code is reviewed and landed you can checkout the code like this:
git clone https://github.com/armenzg/build-tools.gitmozharness cd build-tools/release/updatesmozharness
git checkout update_testing
= Run the tests =
The script that is used by Release Engineering can be run like this:
hg git clone http://hg.mozilla.org/build/tools cd release/updatesbuild-mozharness .python scripts/verifyfirefox_ui_tests.sh py --marionette cfg <update_verify_config_file_name> (e.g. mozBeta-firefox-linux64.cfg # Config depends on your host OS)
This script will create the following directoriesNOTE:* cache - where downloaded installers are store to speed up future downloads* downloads - temporary dir where installers being tested on a specific iteration* venv - where the dependencies of firefox-ui-tests are installed '''Trick'''You can see all available configs in here http:If you want to capture the whole output, you should append this to the command: 2>&1 | tee run//hg.txt == On a loaner ==* Linux** Use DISPLAY=:2 in the verify.sh command for the tests to run properly* Mac** TBD == --keep-venv, --dont-clear-cache ==You can also use --keep-env and --dont-clear-cache to prevent blowing up the dirs every timemozilla.org/build/tools/file/default/release/updates
== Smaller subset of locales ==
To run a small set of tests, simply create a new config with only the lines from the original configs.
You can also reduce the list of locales in each line.
== Output of the script ==
Two text files are generated:
* short_log.txt - it contains the output of firefox-ui-update
* joint_output.txt - it contains the output of gecko intertwined withe firefox-ui-update
** This is dumped into verify.sh's output if a test does not pass
= Run on your local machine =
Besides verify.sh, you have marionette_testing.sh which allows you to run. If you run this: ./marionette_testing.sh config_fileit will enable developer options for you: ../common/setup_firefox_ui_updates.sh --developer-mode venv ./verify.sh $options --marionette $config_file You can use --keep-venv and --dont-clear-chache if you would like to. == Run step-by-step ==Setup: virtualenv --no-site-packages venv source venv/bin/activate pip install --no-index --find-links http://pypi.pub.build.mozilla.org/pub -r ../firefox_ui_updates_requirements.txt Download the binary you want to try: wget ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2015-04-20-03-02-04-mozilla-central/firefox-40.0a1.en-US.win32.installer.exe Run the tests: firefox-ui-update --installer firefox-40.0a1.en-US.win32.installer.exe NOTE: By default, a gecko.log file is generated to remove noise from running firefox-ui-update. Read the help message to know how to adjust it if need be. For beta and release builds which have not yet been released, you need the right update channel: firefox-ui-update --installer "Firefox Setup 38.0b6.exe" --update-channel=beta-localtest = How to use a newer version of firefox-ui-tests =Run these commands: git clone https://github.com/mozilla/firefox-ui-tests.git cd firefoxAppend -ui-tests python setupcfg developer_config.py sdist wget https://hg.mozilla.org/build/braindump/raw-file/default/utils/publish_package_our_pypi.sh chmod +x publish_package_our_pypi.sh # Connect to the VPN ./publish_package_our_pypi.sh dist/firefox-ui-tests-*.tar.gz
= Archive - Failure modes =
The failures reported below are known issues when the infrastructure does not behave as expected.
All other harness issues have been dealt with.
