canmove, Confirmed users, Bureaucrats and Sysops emeriti
4,747
edits
QA/Automation/Projects/Mozmill Automation: Difference between revisions
Jump to navigation
Jump to search
QA/Automation/Projects/Mozmill Automation (view source)
Revision as of 20:22, 13 June 2010
, 13 June 2010→Automation Scripts
| Line 47: | Line 47: | ||
The automation scripts should allow us to have a complete automated process in-place. That means that only a trigger would be necessary to start those release tests. The following areas have to be covered: | The automation scripts should allow us to have a complete automated process in-place. That means that only a trigger would be necessary to start those release tests. The following areas have to be covered: | ||
* Script for Downloading builds from the FTP Server ({{bug|528064}} | * Script for Downloading builds from the FTP Server ({{bug|528064}}) | ||
* Script for Smoketests, BFT, and FFT Test-runs ({{bug|563523}}) | * Script for Smoketests, BFT, and FFT Test-runs ({{bug|563523}}) | ||
* Script for Software Update Test-runs ({{bug|564539}}) | * Script for Software Update Test-runs ({{bug|564539}}) | ||
| Line 54: | Line 54: | ||
* Script for Accessibility Test-run ({{bug|565201}}) | * Script for Accessibility Test-run ({{bug|565201}}) | ||
=== | ===Downloading builds from the FTP Server=== | ||
For a complete automation of our Mozmill tests for release testing, a download script is necessary which will fetch all the builds to test from the FTP server and store those locally. The work is being tracked on {{bug|528064}}. | |||
=== | ===Smoketests, BFT, and FFT Test-run=== | ||
To execute all existing Mozmill tests which have been written for the Smoketest, BFT, and FFT testgroup of Litmus this script is useful. Once it has been called it should automatically execute the steps below: | |||
===Software Update | * Install a build (if requested) | ||
* Run the tests from all those three testgroups | |||
* Uninstall the build (if requested) | |||
The work is being tracked on {{bug|563523}}. | |||
===Software Update Test-run=== | |||
To run the software update tests automatically a test-run script is required which will take care of any aspect of the tests to be run. Once it has been called it should automatically execute the steps below: | To run the software update tests automatically a test-run script is required which will take care of any aspect of the tests to be run. Once it has been called it should automatically execute the steps below: | ||
* Install a build | * Install a build (if requested) | ||
* Create a backup of the build | |||
* Run the partial or complete software update tests | * Run the partial or complete software update tests | ||
* | * Restore the backup | ||
* Run the fallback software update tests | * Run the fallback software update tests | ||
* | * Restore the backup | ||
* Run background software update tests | * Run background software update tests | ||
* Uninstall the build | * Uninstall the build (if requested) | ||
A script with this feature set has already been created on {{bug|504653}} and is able to execute all the steps except step 6. The reason is that we do not have any automated tests for background software updates yet. | A script with this feature set has already been created on {{bug|504653}} and is able to execute all the steps except step 6. The reason is that we do not have any automated tests for background software updates yet. | ||
=== | ===Add-on Test-run=== | ||
The script for the add-ons test-run simplifies the work which would have to be done to run Mozmill tests against available add-ons. Once it has been called it should automatically execute the steps below: | |||
* Perform tests against | * Install a build (if requested) | ||
** Download the | * Perform Mozmill tests against any available add-on | ||
** Download the specified version of the add-on | |||
** Start Firefox with the extension installed | ** Start Firefox with the extension installed | ||
** Run all available tests | ** Run all available tests | ||
** Clean-up the system | ** Clean-up the system | ||
* Uninstall the build (if requested) | |||
The work is being tracked on {{bug|562445}}. | |||
===L10n Test-run=== | |||
With the creation of [[/QA/Mozmill_Test_Automation/L10n_Tests|l10n tests]] we will be able to perform specific tests for localizers. Once it has been called it should automatically execute the steps below: | |||
* Install a build (if requested) | |||
* Run all the l10n tests | |||
* Uninstall the build (if requested) | |||
The work is being tracked on {{bug|565196}}. | |||
===Accessibility Test-run=== | |||
With the creation of [[/QA/Mozmill_Test_Automation/Accessibility_Tests|accessibility tests]] we will be able to perform specific tests for keyboard navigation. Once it has been called it should automatically execute the steps below: | |||
* Install a build (if requested) | |||
* Run all the accessibility tests | |||
* Uninstall the build (if requested) | |||
The work is being tracked on {{bug|565201}}. | |||