QA/Execution/MozTrap: Difference between revisions

← Older edit

QA/Execution/MozTrap (view source)

Revision as of 12:34, 18 January 2013

1,008 bytes removed , 18 January 2013

no edit summary

Whimboo

canmove, Confirmed users, Bureaucrats and Sysops emeriti

4,747

edits

@@ Line 1: / Line 1: @@
 =Overview=
-[https://litmus.mozilla.org Litmus] is Mozilla's Test Case Management tool. Use it to run and manage test suites of Mozilla products.
+[https://moztrap.mozilla.org MozTrap] is Mozilla's Test Case Management tool. Use it to run and manage test suites of Mozilla products.
-=Running Litmus Tests=
+=Running MozTrap Tests=
 TBD.
-= Handling of Litmus Tests  =
+= Handling of Tests  =
-== Creating of Litmus Tests ==
+== Creation of Tests==
-With the flip to the rapid release cycle for Firefox, changes have had to be made when Litmus tests will be added and to which branch. This process is important for QA to sign-off on new features and allow a continuous testing by our community.
+With the flip to the rapid release cycle for Firefox, changes have had to be made when MozTrap tests will be added. This process is important for QA to sign-off on new features and allow a continuous testing by our community.
-If you are already aware of the steps which have to be taken but don't know how to write a Litmus test in general, you should simply check our [http://quality.mozilla.org/docs/litmus/test-case-writing-primer/ Test Case Writing Primer on QMO] in how to get started.
+If you are already aware of the steps which have to be taken but don't know how to write a MozTrap test in general, you should simply check our [http://quality.mozilla.org/docs/litmus/test-case-writing-primer/ Litmus Test Case Writing Primer on QMO] in how to get started.
-Please keep the following items in mind when you create new Litmus tests:
+Please keep the following items in mind when you create new MozTrap test:
-* Start with planning of new Litmus tests while new features are getting implemented on mozilla-central and are available in Nightly builds. Have that list available in a i.e. Google spreadsheet for manual testing until the tests are available in Litmus.
+* Start with planning of new tests while the feature is getting implemented on mozilla-central and are available in Nightly builds. Have that list available in a i.e. Google spreadsheet for manual testing until the tests are available in MozTrap.
-* Once the ''mozilla-central'' -> ''mozilla-aurora'' merge happened, add your already prepared Litmus tests to the '''Aurora (Firefox)''' branch and enable them for the community.
+* Once the ''mozilla-central'' -> ''mozilla-aurora'' merge happened, add your already prepared manual tests and set the appropriate version tag for the Firefox version. Also enable them for the community.
-* Once the ''mozilla-aurora'' -> ''mozilla-beta'' merge happened, a new Firefox release branch will be created as clone from the '''Aurora (Firefox)''' branch. Your testcases will automatically be available. But note that when you add your tests after that clone process, you will have to add them manually on both branches.
-== Disabling of Litmus Tests  ==
+== Disabling of Tests  ==
-If some of the Litmus tests aren't valid anymore, they will have to be disabled instead of deleted. The reason is simply to retain all the test results. By following the steps below you can make sure that disabled tests don't appear in active test-runs or when managing testgroups/subgroups.<br>
+If some of the MozTrap tests aren't valid anymore, they will have to be disabled instead of deleted. The reason is simply to retain all the test results. By following the steps below you can make sure that disabled tests don't appear in active test-runs.<br>
-#Move the test from the testgroup/subgroup to i.e. "Aurora Disabled Tests" / "Aurora - Disabled" (depending on branch), and delete all remaining group assignments.
 #If the test is covered by Mozmill, inform the Desktop Automation team so the automated test can be also removed. Further delete the additional content at the bottom of the list of steps.
 #Update the summary to list the testgroup name (see [[QA/Execution/Fx3.7LitmusOverhaul|Litmus Overhaul]])
@@ Line 31: / Line 29: @@
 == Test Data Handling  ==
+A lot of our existing MozTrap tests have external content as test data. While it is relatively easy to add steps in a test which interact with a web page, it is also very likely that those tests will be broken over time, when the website owner decides to change its structure. That's why we should invest a bit of our time for the creation of minimized test data, wherever it is possible.
-A lot of our existing Litmus tests have external content as test data. While it is relatively easy to add steps in a test which interact with a web page, it is also very likely that those tests will be broken over time, when the website owner decides to change its structure. That's why we should invest a bit of our time for the creation of minimized test data, wherever it is possible.
+When you start to create a new MozTrap test or update an older test, please make sure to check the following list if test data is involved in that test. If you can answer all the items with yes, make sure to create minimized test data.
-When you start to create a new Litmus test or update an older test, please make sure to check the following list if test data is involved in that test. If you can answer all the items with yes, make sure to create minimized test data.
+;Requirements:
+* The content you are referring to is located on an external website
+* The test itself doesn't require different websites (domains)
-;Requirements
+The work which has to be done to create test data can be different. In the simplest case you only have to copy an existing image (make sure we obey the copyright permissions), but there can also be situations when you have to create your own or minimizing an existing HTML document. Put together all needed files before continuing.
-:
-*The content you are referring to is located on an external website
+To be able to reference the test data from within a MozTrap test, the created files have to be uploaded to a place which is accessible by everyone. Therefore we are using our [http://hg.mozilla.org/qa/testcase-data/file/ testcase-data repository]. Whenever you want to upload test data step through the steps given below.
-*The test itself doesn't require different websites (domains)
-*The required test data doesn't have to be created dynamically by a script on the server
-The work which has to be done to create test data can be different. In the simplest case you only have to copy an existing image (make sure we have permissions to do that), but there can also be situations when you have to create your own or minimizing an existing HTML document. Put together all needed files before continuing.
-To be able to reference the test data from within a Litmus test, the created files have to be uploaded to a place which is accessible by everyone. Therefore we are using our [http://hg.mozilla.org/qa/litmus-data/file/ litmus-data repository]. Whenever you want to upload test data step through the steps given below.
 ;How to get data on the server
+* Open the [http://hg.mozilla.org/qa/testcase-data/file/ testcase-data repository] and check under which subgroup the files have to be added.
+* Check for existing files in that subgroup and name your own files accordingly.
+* File a bug in 'Mozilla QA / Infrastructure' to get the tests added and attach a patch to it. As example see bug 780286. Once the patch is up request review from Henrik (:whimboo).
-*Open the [http://hg.mozilla.org/qa/litmus-data/file/ litmus-data repository] and check under which subgroup the files have to be added.
+When the files are in the repository, you can update the MozTrap test and reference the test data via mozqa.com, which is the public mirror and shadows the test data here: http://mozqa.com/data/.
-*Check for existing files in that subgroup and name your own files accordingly.
-*Send an email to [mailto:hskupin@mozilla.com Henrik Skupin] with the files attached and the subgroup
-Once the files are in the repository, you can update the Litmus test and reference the test data from the repository. Make sure to use the raw link to get the real data served. Also the default changeset id with a cryptic number has to be replaced with 'tip' to make sure that always the most recent version of the test data will be used.
-Here an example:
-*Before: [http://hg.mozilla.org/qa/litmus-data/file/06b7d85c6e9e/firefox/technical_tools/js-error.html file/06b7d85c6e9e/firefox/technical_tools/js-error.html]
+To add the link to the MozTrap test please use the following syntax: [link name](url)<br>
-*After: [http://hg.mozilla.org/qa/litmus-data/raw-file/tip/firefox/technical_tools/js-error.html raw-file/tip/firefox/technical_tools/js-error.html]
-To add the link to the Litmus test please use the following syntax: [link name](url)<br>
 A real example you can find in [https://litmus.mozilla.org/show_test.cgi?id=10077 Litmus test 10077].
-=Litmus ownerships=
+=MozTrap ownerships=
+A list of people that are responsible for certain subgroups of tests can be found here: [https://wiki.mozilla.org/QA/Execution/Litmus/Ownership Desktop] and [https://wiki.mozilla.org/QA/Execution/Litmus/Fennec Mobile]
-A list of people that are responsible for certain subgroups of Litmus can be found here: [https://wiki.mozilla.org/QA/Execution/Litmus/Ownership Desktop] and [https://wiki.mozilla.org/QA/Execution/Litmus/Fennec Mobile]