Overview

Excerpt

Mozilla QA is using the existing Mozmill tests to run automated tests against any kind of Firefox build. While this is reducing the time we have to spent on manual testing, we can also enhance our testing to more than only Firefox BFT tests. We plan to have automated test-runs available for software update tests, l10n tests, add-on tests, and others. Those will have to be run on any platform and also against localized builds of Firefox.

Project details

The way to reach the state when everything is automated is long can can be divided into the following sub-projects:

• Setting up a machine which can be used for automated tests
• Creation of scripts which cover all automation aspects
• Running the automation for release builds

Machine for Automated Testing

A machine in the QA lab is needed at least for the first time. Once we have a stable process we can consider to integrate the automation into the Releng process.

This sub-project has been finished. All requirements can be read on bug 537840

Automation Scripts

The automation scripts should allow us to have a complete automated process in-place. That means that only a trigger would be necessary to start those release tests. The following areas have to be covered:

• Download script to fetch builds from the FTP server (bug 528064
• Execution script for smoketests, BFT, and FFT tests
• Execution script for software update tests

Automation Execution

Right now Mozilla QA is using the machine in the QA lab to run release tests. All tests have to be triggered manually. We consider to use Hudson in the future to control the complete process.