QA/Execution/Web Testing/Automation/Jenkins

Summary

Jenkins CI is used by Web QA to run automated tests. The web dashboard can be accessed here. Details of the setup and administration are available on our internal Mana page.

Plugins

Below is a list of plugins in use (dependencies have been omitted):

• Build Timeout - Abort builds that take too long
• Config File Provider - Centrally maintained configuration files for jobs
• EnvInject - Set environment variables for jobs
• github - Integrates GitHub
• Green Balls - Green is better than blue!
• HTML Publisher - Publishes HTML reports
• IRC - IRC bot for triggering builds and notifications
• Job Configuration History - Provides history of configuration changes
• LDAP - User authentication
• Managed Script - Centrally maintained scripts for jobs
• Ownership - Show job owners and co-owners
• Parameterized Trigger - Trigger builds with parameters
• Rebuild - Rebuild a parametrized build without re-entering the parameters
• Role Strategy - Role-based strategy for user management
• Testdroid Marionette - Provisions a Firefox OS device via Testdroid
• Test Stability - Historical information about the stability of tests
• Timestamper - Adds timestamps to the console output
• URLTrigger - Trigger builds based on the response of a URL
• View Job Filters - Advanced filters for views
• Workspace Cleanup - Deletes the workspace before or after the build

Global Environment Variables

Several environment variables are configured:

• DEFAULT_FIREFOX_VERSION - The default Firefox version to use. The default should be the latest compatible version.
• PIP_DOWNLOAD_CACHE - Cache PIP downloads. The default is ${HOME}/.pip_download_cache.
• PIP_INDEX_URL - Sets the PyPI index URL. The default is empty, to use the main PyPI index.
• PIP_USE_MIRRORS - Controls use of the PyPI mirrors. By default this is false to speed up PIP installations.
• PIP_VERBOSE - Controls the verbosity of PIP output. By default this is false and useful to enable for debugging.
• PYTEST_ADDOPTS - Add py.test options to all jobs. The default is -r=fsxXR --verbose --tb=short.
• PYTEST_PROCESSES_MAC - Number of parallel py.test processes to use on Mac nodes.
• PYTEST_PROCESSES_SAUCE - Number of parallel py.test processes to use on Sauce Labs jobs.
• PYTEST_PROCESSES_WINDOWS - Number of parallel py.test processes to use on Windows nodes.
• PYTEST_RERUNS - Number of times to rerun a failed test (used by pytest-rerunfailures). The default is 5.
• PYTEST_TIMEOUT - Timeout (in seconds) for each test (used by pytest-timeout). The default is 300 (5 minutes).
• SAUCE_CAPABILITIES - Single capability to set for all desktop Sauce Labs jobs.
• SAUCE_CAPABILITIES_MOBILE - Single capability to set for all mobile Sauce Labs jobs.
• SAUCE_FIREFOX_VERSION_MAC - The default Firefox version to use for OS X Sauce Labs jobs.
• SAUCE_FIREFOX_VERSION_WINDOWS - The default Firefox version to use for Windows Sauce Labs jobs.
• SAUCE_WINDOWS_VERSION - The default Windows version for Sauce Labs jobs.
• SELENIUM_HOST - The host running the remote Selenium server or hub. The default is selenium-hub1.qa.scl3.mozilla.com.
• WEBQA_TIMEOUT - Timeout (in seconds) used by WebDriver for its own operations (page loads, etc). The default is 60.

Troubleshooting

Try enabling verbose PIP output to determine if installation of Python packages is causing issues.

PIP index is unavailable

We use the main PyPI index for installing Python packages. If this experiences issues then we can enable use of PIP mirrors.

PIP installation is slow

If Python packages are slow to install you should check that PIP mirrors are not enabled. If not, it may be that the main PyPI mirror is slow to respond.