QA/Desktop Firefox/Automation: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
Line 106: Line 106:


== How To Help ==
== How To Help ==
; Before you Begin


=== Test Failures ===
; Getting Started
; RESOLVED FIXED
* Set up your Mozmill development environment (if you have not done so already)
* Verify test passes using the next day's dashboard results
* Pick an project which fits best with your area of interest
* Update the spreadsheet
* Familiarize yourself with some existing code
* Re-enable the test in Litmus
* Find a test you want to automate using the worksheet
* File a bug:
** instructions for bug...
* Start coding
* Submit your first patch into the review process


=== New Tests ===
; What to do when RESOLVED FIXED?
; RESOLVED FIXED
* Wait until the next day's testrun to appear on the dashboard, and verify that your test passes
* Verify test passes using next day's dashboard results
* Assuming it passes, update the worksheet
* Update spreadsheet
* Add the following boilerplate to the Litmus test
* Add boilerplate to Litmus test
<pre>
<pre>
<hr />
<hr />
Line 123: Line 127:
<a href="URL TO TEST IN default BRANCH ON hg.mozilla.org">folder/testModule.js</a>
<a href="URL TO TEST IN default BRANCH ON hg.mozilla.org">folder/testModule.js</a>
</pre>
</pre>
* Add Litmus test to ''Aurora Mozmill Tests'' subgroup
* Add the Litmus test to ''Aurora Mozmill Tests'' subgroup
* ''should you be unable to perform any of the above duties, please comment in your bug''


=== l10n Tests ===
== l10n Tests ==
l10n Tests are used to verify the numerous language translations (locales) do not break functionality and UI in Firefox.
l10n Tests are used to verify the numerous language translations (locales) do not break functionality and UI in Firefox.
* [https://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=1&branch_id=44&testgroup_id=271&subgroup_id= Manual testcases which need automation]
* [https://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=1&branch_id=44&testgroup_id=271&subgroup_id= Manual testcases which need automation]

Revision as of 06:48, 30 November 2011

UNDER CONSTRUCTION

This is being refactored from the old documentation here


If you are looking for the Automation Services team, please go to their team page.

Overview

Mozilla QA uses Mozmill, a tool developed for test automation of applications based on the Gecko Platform (XUL Runner).

We use this tool to automate our manual tests in Litmus. The primary goal is to lessen the time we spend actively testing Smoketest, Basic Functional, and Full Functional level regression tests; enabling us to focus our time more on deep testing of bleeding-edge features and bugs.

An added benefit of using automation is that we can run tests across multiple platforms and locales in parallel, thereby increasing the coverage of regression tests in a smaller amount of time. To put that all into perspective, a single person running the BFTs on one platform and one locale takes about 8 hours. Mozmill can run the same tests across all platforms and all locales in a couple of hours.

The Mozmill tool and APIs are maintained and developed by the Automation Services team. The relatively small Desktop Automation team is responsible for developing the tests. This is where we need you, the community.

The Team

Person Photo Location Role
Anthony Hughes
IRC: ashughes 		Vancouver, Canada Team Lead
Vlad Maniac
IRC: vladmaniac 		Cluj, Romania Contractor Lead
Alex Lakatos
IRC: AlexLakatos 		Cluj, Romania Developer
Remus Pop
IRC: remuspop 		Cluj, Romania Developer

You can get in contact with us in a few different ways:

  • Take part in a discussion on the mozmill-dev mailing list
  • Join the #mozmill IRC channel to ask about Mozmill
  • Join the #automation IRC channel to ask about test automation

Meetings

Every two weeks we meet to discuss the progress we have made, where we want to go, and how we are going to get there. You can learn a lot about what we do just by listening in to one of our meetings. These meetings are open to everyone.

Meeting Information: 

   When: Mondays, 8am PT / 4pm GMT
Dial-in: +1 (800) 707-2533 Password 369 Conference 654

Areas of Work

Projects

Project Summary Status % COMPLETE Documents
Failures Repair broken tests 6 / 17 33.3 % worksheet
Endurance Tests measuring resource usage in various scenarios 6 / 27 23.1 % worksheet
Smoketests Tests covering the most basic functionality 15 / 32 45.5 % worksheet
Restart Tests requiring Firefox to restart 19 / 29 65.5 % worksheet
Functional Tests covering more advanced functionality 75 / 122 61.5 % worksheet
l10n Tests for the numerous localizations/languages 2 / 25 8 % worksheet

Prioritized by increasing complexity and decreasing need

How To Help

Before you Begin
Getting Started
  • Set up your Mozmill development environment (if you have not done so already)
  • Pick an project which fits best with your area of interest
  • Familiarize yourself with some existing code
  • Find a test you want to automate using the worksheet
  • File a bug:
    • instructions for bug...
  • Start coding
  • Submit your first patch into the review process
What to do when RESOLVED FIXED?
  • Wait until the next day's testrun to appear on the dashboard, and verify that your test passes
  • Assuming it passes, update the worksheet
  • Add the following boilerplate to the Litmus test
<hr />
Covered by Mozmill test:<br>
<a href="URL TO TEST IN default BRANCH ON hg.mozilla.org">folder/testModule.js</a>
  • Add the Litmus test to Aurora Mozmill Tests subgroup
  • should you be unable to perform any of the above duties, please comment in your bug

l10n Tests

l10n Tests are used to verify the numerous language translations (locales) do not break functionality and UI in Firefox.

Getting Started

To help you get started, we have put together a couple of guides:

Of course, if you have any questions, you can always reach us by:

Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Desktop_Firefox/Automation&oldid=374381"