QA/Automation/Projects/Addons/MozmillCrowd: Difference between revisions
No edit summary |
|||
| Line 1: | Line 1: | ||
__NOTOC__ | |||
=Overview= | =Overview= | ||
{| | {| | ||
| style="width: | | style="width: 45%" | | ||
| style="width: | | style="width: 55%" | | ||
|- valign="top" | |- valign="top" | ||
| ''' | | '''Project:''' | ||
| | | Mozmill-Crowd Extension | ||
|- valign="top" | |- valign="top" | ||
| ''' | | '''Leads:''' | ||
| | | [mailto:hskupin@mozilla.com Henrik Skupin], [mailto:atrain@mozilla.com Aaron Train] | ||
|- valign="top" | |- valign="top" | ||
| ''' | | '''Team Assistance:''' | ||
| | | n/a | ||
|} | |||
Running Mozmill tests should be as easy as possible and doable by any interested tester. No complicated setup routines or pre-requisites should stop anyone by successfully running our existing Mozmill tests in their Firefox browser. As result a rich set of results, which will be collected from testers world-wide, can be collected and used to nail down regressions or to find problems related to the geographical position or installed locale of Firefox. | |||
=Mozmill-Crowd Platform= | |||
* [[QA/Mozmill_Test_Automation/Crowd_Testing/Documentation|Documentation]] | |||
* [http://etherpad.mozilla.com:9000/qa-auto-mozmill-crowd Etherpad] for brainstorming and discussions | |||
* [[QA/Mozmill_Test_Automation/Crowd_Testing/FAQ|FAQ]] | |||
* [[QA/Mozmill_Test_Automation/Crowd_Testing/Presentations|Presentations]] | |||
=Mozmill-Crowd Production= | |||
* [[QA/Mozmill_Test_Automation/Crowd_Testing/Roadmap|Product Roadmap]] | |||
* [[QA/Mozmill_Test_Automation/Crowd_Testing/Release|Release Process]] | |||
* [[QA/Mozmill_Test_Automation/Crowd_Testing/Schedule|Release Schedule]] | |||
* [[QA/Mozmill_Test_Automation/Crowd_Testing/Contribution|How to Contribute]] | |||
=Resources= | |||
* [http://couchdb.apache.org/ CouchDB] | |||
* [http://guide.couchdb.org/ CouchDB the definitive guide] | |||
* [http://code.quirkey.com/sammy/index.html Sammy JS framework] | |||
= | =To move= | ||
[https://docs.google.com/Doc?id=d9cz3wk_17hmkx6p2w Google Document] | |||
[https://bugzilla.mozilla.org/show_bug.cgi?id=564648 Bug 564648] | |||
=Project Details= | =Project Details= | ||
Revision as of 17:48, 26 January 2011
Overview
| Project: | Mozmill-Crowd Extension |
| Leads: | Henrik Skupin, Aaron Train |
| Team Assistance: | n/a |
Running Mozmill tests should be as easy as possible and doable by any interested tester. No complicated setup routines or pre-requisites should stop anyone by successfully running our existing Mozmill tests in their Firefox browser. As result a rich set of results, which will be collected from testers world-wide, can be collected and used to nail down regressions or to find problems related to the geographical position or installed locale of Firefox.
Mozmill-Crowd Platform
- Documentation
- Etherpad for brainstorming and discussions
- FAQ
- Presentations
Mozmill-Crowd Production
Resources
To move
Project Details
Only running Mozmill tests on buildbot or on the release-test machine in the QA lab is not sufficient to find misbehavior's of Firefox. A more wide-spread test-run is necessary. Such a possibility will be the result of the crowd testing project. An extension has to be developed which will automatically prepare the system, executes the test-run, and send the results to our report server for analysis.
The Extension
The easiest way to add additional features to Firefox is to install an extension. The same way will be used for this project. The extension, which will be offered on addons.mozilla.org, will add it's own toolbar right below of the existing ones. With a dropdown field the user will be able to select the wanted test-run, e.g. normal tests, update tests, or l10n tests. By clicking the 'Run' button the test-run gets executed.
Preparing the System
To run Mozmill tests, the system has to be prepared to match a specific set of requirements. The steps which are necessary aren't the easiest ones and can cause trouble, especially when you are doing it for the first time.
That's why the extension has to create a virtual environment, which will contain all needed applications and tools. The following steps will be necessary:
- Retrieving the Python interpreter (or bundle it with the extension)
- Create a virtual Python environment
- Install the latest release of Mozmill
- Retrieve the automation scripts
Executing Mozmill Tests
Given by the selected dropdown entry on the toolbar the Python script of the related test-run gets executed. It will perform the following actions:
- Cloning the Mozmill tests from the mozmill-tests repository
- Updating the tests branch to the Gecko branch of the currently running version of Firefox
- Starting another instance of Firefox
- Execute all existent tests
Results Analysis
Once a test-run has been finished the results can be uploaded to our report server. There they can be examined and checked for new failures. A web dashboard will be created to offer the most valuable information.