QA/Desktop Firefox/Automation
UNDER CONSTRUCTION
This is being refactored from the old documentation here
If you are looking for the Automation Services team, please go to their team page.
Overview
Mozilla QA uses Mozmill, a tool developed for test automation of applications based on the Gecko Platform (XUL Runner).
We use this tool to automate our manual tests in Litmus. The primary goal is to lessen the time we spend actively testing Smoketest, Basic Functional, and Full Functional level regression tests; enabling us to focus our time more on deep testing of bleeding-edge features and bugs.
An added benefit of using automation is that we can run tests across multiple platforms and locales in parallel, thereby increasing the coverage of regression tests in a smaller amount of time. To put that all into perspective, a single person running the BFTs on one platform and one locale takes about 8 hours. Mozmill can run the same tests across all platforms and all locales in a couple of hours.
The Mozmill tool and APIs are maintained and developed by the Automation Services team. The relatively small Desktop Automation team is responsible for developing the tests. This is where we need you, the community.
The Team
| Person | Photo | Location | Role |
| Anthony Hughes IRC: ashughes |
Vancouver, Canada | Team Lead | |
| Vlad Maniac IRC: vladmaniac |
Cluj, Romania | Contractor Lead | |
| Alex Lakatos IRC: AlexLakatos |
Cluj, Romania | Developer | |
| Remus Pop IRC: remuspop |
Cluj, Romania | Developer |
You can get in contact with us in a few different ways:
- Take part in a discussion on the mozmill-dev mailing list
- Join the #mozmill IRC channel to ask about Mozmill
- Join the #automation IRC channel to ask about test automation
Meetings
Every two weeks we meet to discuss the progress we have made, where we want to go, and how we are going to get there. You can learn a lot about what we do just by listening in to one of our meetings. These meetings are open to everyone.
Meeting Information:
When: Mondays, 8am PT / 4pm GMT Dial-in: +1 (800) 707-2533 Password 369 Conference 654
Areas of Work
Projects
| Project | Summary | Priority | Status | % COMPLETE | Documents |
| Failures | Repair broken tests | P1 | 6 / 17 | 33.3 % | worksheet |
| Endurance | Tests measuring resource usage in various scenarios | P2 | 6 / 27 | 23.1 % | worksheet |
| Smoketests | Tests covering the most basic functionality | P3 | 15 / 32 | 45.5 % | worksheet |
| Restart | Tests requiring Firefox to restart | P4 | 19 / 29 | 65.5 % | worksheet |
| Functional | Tests covering more advanced functionality | P5 | 75 / 122 | 61.5 % | worksheet |
| l10n | Tests for the numerous localizations/languages | P7 | 2 / 2 | 100 % | worksheet |
Prioritized by increasing complexity and decreasing need
How To Help
Test Failures
- RESOLVED FIXED
- Verify test passes using the next day's dashboard results
- Update the spreadsheet
- Re-enable the test in Litmus
New Tests
- RESOLVED FIXED
- Verify test passes using next day's dashboard results
- Update spreadsheet
- Add boilerplate to Litmus test
<hr /> Covered by Mozmill test:<br> <a href="URL TO TEST IN default BRANCH ON hg.mozilla.org">folder/testModule.js</a>
- Add Litmus test to Aurora Mozmill Tests subgroup
Getting Started
To help you get started, we have put together a couple of guides:
- Learn about installing Mozmill
- Learn about the test repository
- Learn about running tests
- Learn about writing tests
Of course, if you have any questions, you can always reach us by:
- Joining the #automation channel on IRC
- Sending an email directly to Anthony Hughes