canmove, Confirmed users
2,850
edits
Litmus:Design: Difference between revisions
→Test Runs (Was Replicate Testrunner Functionality)
ChrisCooper (talk | contribs) (→Design) |
ChrisCooper (talk | contribs) |
||
| Line 10: | Line 10: | ||
In Testrunner, existing test runs or lists are cloned to create new test runs. Each test result in the test run is created in advance from the corresponding testcase, and that result is updated when the result is submitted. Aside from the overhead of creating all the results for each run at the outset, this also precludes having multiple results for the same testcase in the same test run, i.e. a new test run must be created for each separate tester. | In Testrunner, existing test runs or lists are cloned to create new test runs. Each test result in the test run is created in advance from the corresponding testcase, and that result is updated when the result is submitted. Aside from the overhead of creating all the results for each run at the outset, this also precludes having multiple results for the same testcase in the same test run, i.e. a new test run must be created for each separate tester. | ||
With Litmus, we have a chance to make '''test runs''' both more lightweight ''and'' (hopefully) more powerful. | With Litmus, we have a chance to make '''test runs''' both more lightweight ''and'' (hopefully) more powerful. | ||
We will implement each test run as a set of search criteria. Results that match the search criteria are included for display in any test run reports. Test runs can be created specifically for certain events, e.g. a testday. This also allows us to create test runs after-the-fact that can automatically be matched up against existing reports in the database. | |||
If not limited by date, test runs can also be ongoing. This will allow us to have have long-lived test runs that can take the place of the test groups that are currently used to aggregate testing in Litmus. | |||
This meshes well with the current search and reporting structure of Litmus. | |||
== Design == | == Design == | ||
I'm going to structure this design as a series of ordered tasks so that I can easily check elements off as I finish them. | I'm going to structure this design as a series of ordered tasks so that I can easily check elements off as I finish them. | ||
=== Database Changes === | |||
# Update database schema to allow for more complex testcase relationships ([https://bugzilla.mozilla.org/show_bug.cgi?id=323768 bug 323768]). This will require the following schema changes: | # Update database schema to allow for more complex testcase relationships ([https://bugzilla.mozilla.org/show_bug.cgi?id=323768 bug 323768]). This will require the following schema changes: | ||
| Line 34: | Line 42: | ||
#** drop subgroup ref from tests table | #** drop subgroup ref from tests table | ||
#** add product ref to tests table | #** add product ref to tests table | ||
# Update affected Perl modules and CGI scripts to reflect the database schema changes, preserving existing functionality. | |||
=== Perl modules === | |||
# Update affected Perl modules and CGI scripts to reflect the database schema changes, preserving existing functionality. Affected modules: | |||
## Litmus::DB::Test.pm | |||
=== CGI Scripts === | |||
=== HTML Templates and CSS === | |||
=== JavaScript Libraries === | |||
= Test Case Management = | = Test Case Management = | ||