151
edits
QA/TDAI/Inventory: Difference between revisions
m
no edit summary
(→Test Suites: adding www.mozilla.org link to js test information) |
mNo edit summary |
||
| Line 22: | Line 22: | ||
=== Criteria for Colors === | === Criteria for Colors === | ||
< | <div class="note">'''NOTE''': that judgments are subjective and not "hard and fast". Caring enough to discuss will change a rating. Not caring enough to discuss is explanation enough.</div> | ||
* Green: Generously defined now. Info is current or findable. Something is buildable and runnable by someone. There may still be obvious improvements needed. | * Green: Generously defined now. Info is current or findable. Something is buildable and runnable by someone. There may still be obvious improvements needed. | ||
| Line 31: | Line 31: | ||
{| class="fullwidth-table" | {| class="fullwidth-table" | ||
!Test Harness Name | |||
!Brief Description | |||
!Language Used | |||
!Status of Harness | |||
!Notes | |||
|- | |- | ||
| MochiTest | | MochiTest | ||
| Line 41: | Line 41: | ||
| JavaScript | | JavaScript | ||
| Automation being set up on QA Farm. Exec by community is possible. | | Automation being set up on QA Farm. Exec by community is possible. | ||
| [ | | [[mdc:Writing MochiTest-based unit tests|mochitest]] | ||
|- | |- | ||
| reftest | | reftest | ||
| comparison of two pages, via JavaScript and Canvas, for visual equivalency | | comparison of two pages, via JavaScript and Canvas, for visual equivalency | ||
| JavaScript | | JavaScript | ||
| runnable on built trunk. There is a list of [ | | runnable on built trunk. There is a list of [[mdc:reftest opportunities files|reftest opportunities]], but it is not clear if we want to use them. | ||
| [ | | [[mdc:Creating reftest-based unit tests|reftest]] | ||
|- | |- | ||
| xpcshell | | xpcshell | ||
| Line 53: | Line 53: | ||
| JavaScript | | JavaScript | ||
| recent care and feeding, very usable | | recent care and feeding, very usable | ||
| [ | | [[mdc:Writing_xpcshell-based_unit_tests|xpcshell]] | ||
|- | |- | ||
| eggplant | | eggplant | ||
| Line 62: | Line 62: | ||
|- | |- | ||
| litmus | | litmus | ||
| Catalog of manual test cases. [ | | Catalog of manual test cases. [[Litmus|Being developed]] to allow community to [http://litmus.mozilla.org/run_tests.cgi run] the manual smoke tests, BFT, FFT and L10n tests. [http://litmus.mozilla.org/search_results.cgi GUI to capture results] and provide feedback to the QA team. Reporting function to check test and platform coverage. | ||
| English, German, French, etc. | | English, German, French, etc. | ||
| The place to be on Community Test Day! Still in refinement by [mailto:zlipton@mozilla.com zlipton] and [mailto:ccooper@deadsquid.com coop]. 2445 test cases, 1994 tester accounts with 845 active | | The place to be on Community Test Day! Still in refinement by [mailto:zlipton@mozilla.com zlipton] and [mailto:ccooper@deadsquid.com coop]. 2445 test cases, 1994 tester accounts with 845 active | ||
| Line 71: | Line 71: | ||
| | | | ||
| running, passing, run by tinderbox | | running, passing, run by tinderbox | ||
| [ | | [[MozillaQualityAssurance:Performance_Testing|wiki]] | ||
|- | |- | ||
| test.js | | test.js | ||
| Line 101: | Line 101: | ||
{| class="fullwidth-table" | {| class="fullwidth-table" | ||
! | |||
!Test Suite Name | |||
!Brief Description | |||
!Test Harness(es) | |||
!Status | |||
!Last Info | |||
|- | |- | ||
| style="background:#00ff00" | | | style="background:#00ff00" | | ||
| Line 159: | Line 159: | ||
| style="background:#00ff00" | | | style="background:#00ff00" | | ||
| T* | | T* | ||
| includes Tdhtml, Ts, Txul, [ | | includes Tdhtml, Ts, Txul, [[Trender]] (aka Tgfx), Z, and mZ, [[Performance:Tinderbox_Tests|wiki for most T]] | ||
| | | | ||
| running, but not growing | | running, but not growing | ||
| Line 166: | Line 166: | ||
| style="background:#00ff00" | | | style="background:#00ff00" | | ||
| Accessibility | | Accessibility | ||
| [ | | [[Accessibility/qa]] | ||
| | | | ||
| Info is current. A lot is going on, but opportunities to automate are not as developed as they could be. See [mailto:aaronleventhal@moonset.net Aaron Leventhal]. | | Info is current. A lot is going on, but opportunities to automate are not as developed as they could be. See [mailto:aaronleventhal@moonset.net Aaron Leventhal]. | ||
| Line 174: | Line 174: | ||
| Update Checker | | Update Checker | ||
| validity of update mar files | | validity of update mar files | ||
| currently automated on release machines, moving to buildbot, [ | | currently automated on release machines, moving to buildbot, [[MozillaQualityAssurance:Update_Checker|wiki]] | ||
| | | | ||
| Info is out of date. See [mailto:rhelmer@mozilla.com rhelmer]. | | Info is out of date. See [mailto:rhelmer@mozilla.com rhelmer]. | ||
| Line 182: | Line 182: | ||
| verify that :<ul><li>en-US build matches locale build</li><li>ensure no read-only files</li><li>show differences from last release</li> | | verify that :<ul><li>en-US build matches locale build</li><li>ensure no read-only files</li><li>show differences from last release</li> | ||
| | | | ||
| currently automated on release machines, moving to buildbot, [ | | currently automated on release machines, moving to buildbot, [[MozillaQualityAssurance:MetaDiff|wiki]] | ||
| Info is out of date. See [mailto:rhelmer@mozilla.com rhelmer]. | | Info is out of date. See [mailto:rhelmer@mozilla.com rhelmer]. | ||
|- | |- | ||
| Line 195: | Line 195: | ||
| Download Checker | | Download Checker | ||
| verify integrity of major released | | verify integrity of major released | ||
| manual, [ | | manual, [[MozillaQualityAssurance:Download_Checker|wiki]] | ||
| runnable, passing | | runnable, passing | ||
| Info is current. See [mailto:bc@mozilla.com bc]. | | Info is current. See [mailto:bc@mozilla.com bc]. | ||
| Line 202: | Line 202: | ||
| L10N Search Verifier | | L10N Search Verifier | ||
| | | | ||
| manual, [ | | manual, [[MozillaQualityAssurance:l10n_Search_Verifier|wiki]] | ||
| runnable, passing | | runnable, passing | ||
| Info is current, but current plans are unclear. | | Info is current, but current plans are unclear. | ||
| Line 229: | Line 229: | ||
| style="background:#ffff00" | | | style="background:#ffff00" | | ||
| XBL2 | | XBL2 | ||
| [ | | [[MozillaQualityAssurance:Test_Development_Projects#Platform_Content_Area|wiki]] | ||
| | | | ||
| XBL2 still in design | | XBL2 still in design | ||
| Line 258: | Line 258: | ||
| Java Applets | | Java Applets | ||
| list suggested by Java team at Apple | | list suggested by Java team at Apple | ||
| browser and eyeballs, [ | | browser and eyeballs, [[MozillaQualityAssurance:Test_JavaApplets|wiki]] | ||
| Just a list. No automation is currently in process. | | Just a list. No automation is currently in process. | ||
| Info is current. See [mailto:rkiddy@mozilla.com ray]. | | Info is current. See [mailto:rkiddy@mozilla.com ray]. | ||
| Line 278: | Line 278: | ||
| style="background:#00ff00" | | | style="background:#00ff00" | | ||
| HTML Parsing | | HTML Parsing | ||
| Will need to deal with WHATWG/html5lib tests | | Will need to deal with WHATWG/html5lib tests {{bug|366936}}. The htmlparser/tests/html test vs a baseline, but a baseline has not been stored. | ||
| [[Custom Test Executables|custom executable/C++]] | | [[Custom Test Executables|custom executable/C++]] | ||
| | | | ||
| Line 305: | Line 305: | ||
|- | |- | ||
| style="background:#ffff00" | | | style="background:#ffff00" | | ||
| | | MathML | ||
| Do mathematical glyphs display correctly on all platforms? | | Do mathematical glyphs display correctly on all platforms? | ||
| "browsers and eyeballs" | | "browsers and eyeballs" | ||
| Line 378: | Line 378: | ||
{| class="fullwidth-table" | {| class="fullwidth-table" | ||
!Test Infrastructure Name | |||
!Brief Description | |||
!Harnesses and Test Suites in the Infrastructure | |||
!Status of the Instructure | |||
!Notes | |||
|- | |- | ||
| tinderboxen | | tinderboxen | ||