Testrunner: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
m (Reverted edit of Bongo Bluxo, changed back to last version by Ghendricks)
 
(27 intermediate revisions by 6 users not shown)
Line 1: Line 1:
== Introduction ==
'''Testrunner is now known as [[Testopia]]'''


[http://www.willowriver.net/products/testrunner.php Testrunner] is a test development tool built on top of Bugzilla, our installation of which is located at http://testrunner.mozilla.org (t.m.o.) --where QA ([[MozillaQualityAssurance:Home_Page|quality assurance]]) can create, modify and monitor test plans, test cases, test runs and test results.
For Mozilla QA see [[Litmus]]
 
Examples of other Testrunner installations:
* Eduardo Fuentetaja's server running v0.6.1-beta: http://testrunner.citat.se/
 
* Opi's server running a recent CVS pull (Warning: Under development! Might go poof at any time, but ideally this is what Seamonkey QA might use): http://www2.le-bit.de/bugzilla/index.cgi
 
At present the tests in Testrunner are manual, with a black-box persepective.
 
== To-do list ==
 
# Upgrade from Testrunner 0.2 to 0.6 (or whatever the most recent version).
# Migrate existing test plans, functional groups and test cases into the more recent Testrunner version. Also, ensure that they remain accessible and useable in the new version.
#* Note that the old, pre-Bugzilla Component Makeover components had been used. We'll need to use the latest component organization.
# More seamless integration with the https://bugzilla.mozilla.org (b.m.o.) database, such as
## Allowing bug links in test cases and results (regardless of result status).
## Cross-referencing products and components from b.m.o. into Testrunner, instead of recreating them.
# Define (and publically document) usage guidelines for Testrunner, such as describing priviledges, outlining typical workflow, etc. This might also include a tutorial (as time permits).
# Improve access control:
#* Define who can create testplans, which could differ from those who can modify test plans.
#* Same goes for test cases, functional groups and components.
#* For test runs, control who can modify which runs, but still have flexibility to allow >1 user per active run —particularly useful for long test runs like BFT's.
#Improve Testrunner UI:
## Clarify navigation amongst test plans, test cases, test runs and test results. (Might be resolved in a more recent Testrunner build.)
## Improve maintenance, creation/editing and monitoring of test plans, test cases, test runs and test results.
## Allow for easier entry of comments for test runs and test cases.
# Integrate with [[Mister:Home Page|Mister]], the proposed tool to integrate https://update.mozilla.org (u.m.o.), Bouncer, the build systems (including Tinderbox/Bonsai), QA (noteably Testrunner and b.m.o.) and the release process.
# Integrate with other QA tools, notably more automated and/or white/grey-box oriented testing (Eggplant? Smoketest extension?)
# Integrate with Talkback (tracking build ids, test results, crash bugs, etc.)
 
== Known issues and bugs ==
Some of these are observed in t.m.o., in which case they might've been resolved in a more recent Testrunner build.
 
# (t.m.o.) We found that the old style products and components are being used, and currently hinder organization of test cases such as those for Seamonkey. (More or less point (2) under "To-do list" above.)
# (t.m.o.) In spite of a case being in the "confirmed" state, a few such cases still don't appear in a test run. Why?
# (t.m.o.) Unable to delete cases (and perhaps functional groups, components, etc.) when there's a test run including that cases. Might need a more obvious-looking "disabled" state?

Latest revision as of 09:37, 14 January 2007

Testrunner is now known as Testopia

For Mozilla QA see Litmus

Retrieved from "https://wiki.mozilla.org/index.php?title=Testrunner&oldid=46410"