QA/TDAI/Inventory: Difference between revisions

{| class="fullwidth-table"

| style="background:#efefef" | '''Test Suite Name'''

| style="background:#efefef" | '''Brief Description'''

| style="background:#efefef" | '''Test Harness(es)'''

| style="background:#efefef" | '''Status'''

|-  

| JS tests

| well-developed suite of tests of the JavaScript language in [http://lxr.mozilla.org/mozilla/source/js/tests/ js/tests]

| [http://bclary.com/projects/spider/ SpiderMonkey] Runs on QA Farm. Stable and not necessary to use or share.

| Semi-automated.  Plans to fully automate with Alice and Rob's help in Q1 2007

|-

| Security tests

|  

| Custom framework.  Runs on QA Farm. When bugs were filed, regression test cases are usually added.

| Semi-automated.  Plans to fully automate with Eggplant in Q1 2007

|-

| Automated Smoke Tests

| A "Smoke Test" is a quick, basic qualification.

| Eggplant.  Runs on Tracy's machines. Tests for FFx 1.5 and 2.0

|-

| Basic Functionality Tests (BFT)

| 130+ manual test cases, broadly covering major features

| litmus for [http://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=1&testgroup_id=7 1.5] and [http://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=1&testgroup_id=25 2.0]

| will be updated for FF3

|-

| Full Functionality Tests (FFT)

| approx. 280 manual test cases, covering most features

| litmus for [http://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=1&testgroup_id=2 1.0], [http://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=1&testgroup_id=26 1.5] and [http://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=1&testgroup_id=27 2.0]

| will be updated for FF3

|-

| Localization Tests

| 15 test cases

| litmus for [http://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=1&testgroup_id=10 1.5] and [http://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=1&testgroup_id=39 2.0]

| engaged and open to design

|-

| Software Update Suite

| approx. 50 tests, targeted to update process itself

| [http://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=1&testgroup_id=38 litmus]

|  

|-

| T*

| includes Tdhtml, Ts, Txul, [http://wiki.mozilla.org/Mozilla2:Trender Trender] (aka Tgfx), Z, and mZ, [http://wiki.mozilla.org/Performance:Tinderbox_Tests wiki for most T]

| "make check"

| running, but not growing

|  

|-

| Tp2

| new page load tests

| "make check"/buildbot

| nearing complete automation

|  

|-

| Download Checker

| verify integrity of major released

| manual, [http://wiki.mozilla.org/MozillaQualityAssurance:Download_Checker wiki]

| runnable, passing

|-

| L10N Search Verifier

|

| manual, [http://wiki.mozilla.org/MozillaQualityAssurance:l10n_Search_Verifier wiki]

| runnable, passing

|-

| XSLT

| re-purposing of the Adobe Xalan tests.

| [[Custom Test Executables|custom executable/C++]]

| Driver is in mozilla, but test content is at Apache.

|-

| DOM

| being transitioned into MochiTest

| browser and eyeballs?

|  

|-

| CSS

| verify spec compliance

| reftest

| running, passing, needing much to be added

|-

| XBL2

| [http://wiki.mozilla.org/MozillaQualityAssurance:Test_Development_Projects#Platform_Content_Area wiki]

|  

| XBL2 still in design

|-

| Browser/Content

| May be transitioned to reftest or MochiTest, as appropriate.

| [[Custom Test Executables|custom executable/C++]]

|  

|-

| Embedding

| Tests seem fairly current but not building, why?

| [[Custom Test Executables|custom executable/C++]]

|

|-

| Extensions

|  

| [[Custom Test Executables|custom executable/C++]]

|  

|-

| Java Applets

| list suggested by Java team at Apple

| browser and eyeballs, [http://wiki.mozilla.org/MozillaQualityAssurance:Test_JavaApplets wiki]

| Just a list. No automation is currently in process.

|-

| I18N

|  

| [[Custom Test Executables|custom executable/C++]]

|  

|-

| Graphics

|  

| [[Custom Test Executables|custom executable/C++]]

|  

|-

| HTML Parsing

| Will need to deal with WHATWG/html5lib tests [https://bugzilla.mozilla.org/show_bug.cgi?id=366936 366936]. The htmlparser/tests/html test vs a baseline, but a baseline has not been stored.

| [[Custom Test Executables|custom executable/C++]]  

|

|-

| Acid2

| compliance tests from outside standards "watcher"

| manually executed, [http://www.webstandards.org/action/acid2/ Acid2 Browser Test]

| currently passing

|-

| XPConnect

| TestXPC -> "FAILED to get a JSRuntime"

| [[Custom Test Executables|custom executable/C++]]

|  

|-

| Layout/HTML

| 1800+ files. These can be transitioned, probably to reftest. HTML file groups: CellMapTest, FrameSetNav, SlideShow, attributes, block, css, formctls, frameset, printer, style, table, tags, xbl.

| "the old regression test suite" and reftest. Older tests will go to reftest.

|  

|-

| Math/ML

| Do mathematical glyphs display correctly on all platforms?

| "browsers and eyeballs"

| also using w3c.org site content

|-

| Plugins

|

|

| Not known

|-

| Java Plugin

|  

|  

| not current, dev may feel out of loop

|-

| Networking

| 25 test executables, but all must be given parameters

| [[Custom Test Executables|custom executable/C++]]

|  

|-

| NSPR

| approx 150 test executables, most pass.

| custom executable/c

|  

|-

| RDF

|  

| [[Custom Test Executables|custom executable/C++]]

| dsds (not being built), rdfcat, rdfpoll, triplescat (doc for invoke?)

|-

| Security/NSS

| 26 tests, but not runnable without info

| [[Custom Test Executables|custom executable/C++]] and "make check"

|

|-

| XPCOM

| 28 custom executables, most pass, can be better documented, xpcom/tests/unit/test_pipe.js (xpcshell)

| [[Custom Test Executables|custom executable/C++]]

|  

|-

| XPFE

| winopen.xul, runs but does not show final results

|  

|

|-

| XPInstall

| pre_checkin_trigger.html, runs in browser, passes

|  

|  

|}