QA/Execution/Web Testing/Automation/Jenkins

< QA‎ | Execution‎ | Web Testing‎ | Automation

Contents

Summary

Jenkins CI is used by Web QA to run automated tests. The production instance is publicly accessible and can be found here. Details of the setup and administration are available on our internal Mana page. We also have a staging instance for adhoc jobs and experimentation. This is secured behind Mozilla VPN, and can be found here. Documentation for the staging instance is also available on Mana.

Plugins

Below is a list of plugins in use (dependencies have been omitted):

Global Environment Variables

Several environment variables are configured:

  • APPIUM_BROWSER_NAME - Browser name to use with Appium. The default is Browser.
  • APPIUM_DEVICE_NAME - Device name to use with Appium. The default is Android Emulator.
  • APPIUM_PLATFORM_NAME - Platform name to use with Appium. The default is Android.
  • APPIUM_PLATFORM_VERSION - Platform version to use with Appium. The default is 5.1.
  • APPIUM_VERSION - Version of Appium to use. The default is 1.5.3.
  • FIREFOX_VERSION - Firefox version to use. The default is 47.0.
  • MAC_VERSION - MacOS version to use. The default is OS X 10.11.
  • PIP_INDEX_URL - Sets the PyPI index URL. The default is empty, to use the main PyPI index.
  • PIP_USE_MIRRORS - Controls use of the PyPI mirrors. By default this is false to speed up PIP installations.
  • PIP_VERBOSE - Controls the verbosity of PIP output. By default this is false and useful to enable for debugging.
  • PYTEST_ADDOPTS - Add py.test options to all jobs. The default is -r=fsxXR --verbose --tb=short.
  • PYTEST_PROCESSES - Number of parallel py.test processes to use. The default is 15.
  • PYTEST_RERUNS - Number of times to rerun a failed test (used by pytest-rerunfailures). The default is 5.
  • SAUCELABS_USERNAME - Username to use for Sauce Labs.
  • SELENIUM_EXCLUDE_DEBUG - Exclude capturing these debug types from Selenium tests. The default is logs.
  • SELENIUM_VERSION - Selenium version to use. The default is 2.53.1.
  • TOX_TESTENV_PASSENV - Environment variables to pass down into Tox environments. The default is http_proxy HTTP_PROXY https_proxy HTTPS_PROXY no_proxy NO_PROXY SAUCELABS_USERNAME SAUCELABS_API_KEY.
  • WINDOWS_VERSION - Windows version to use. The default is Windows 7.

Known Issues

  • None

Troubleshooting

Try enabling verbose PIP output to determine if installation of Python packages is causing issues.

PIP index is unavailable

We use the main PyPI (Python Package Index) for installing Python packages. If this experiences issues then we can enable use of PIP mirrors.

PIP installation is slow

If Python packages are slow to install you should check that PIP mirrors are not enabled. If not, it may be that the main PyPI mirror is slow to respond.