SoftwareTesting:Scratchpad: Difference between revisions

Jump to navigation Jump to search

SoftwareTesting:Scratchpad (view source)

Revision as of 19:10, 29 December 2005

3,040 bytes removed ,  29 December 2005
no edit summary
No edit summary
 
No edit summary
Line 1: Line 1:
= Overview =
* Help Wanted


[[User:Davel|Dave Liebreich]] is interested in creating running code to test the mozilla codebase.  Not just end-to-end automated tests, but unit tests, module tests, test harnesses, test reporting infrastructure, and anything else that lowers the barrier to creating new code that gives someone feedback.
Dave needs help.  The codebase is huge.  The community is huge.  The task is huge.
This page is the first cut at collecting Dave's plans and investigations.
= Guiding principles =
* Testing is good
* Testing provides technical information about the product/system under test
* There are lots of different types of testing, each providing a different type of information and involving different trade-offs
* replacing a test that is useful to person A with a test that is useful to person B but not useful to person A is bad - just add the new test
* automated developer testing works only if the developers are writing the tests.
** if developer finds something that is difficult to test, then developer can make code more testable
** if code is more testable, then tests can be more powerful
* feedback from user community is unstructured and somewhat unpredictable, but incredibly useful.  We should not extinguish this valuable channel of information.
* developer unit tests are typically not-very-powerful tests.  The value comes from
** immediate and precise feedback - written close to the code
** there are typically lots of them
** they function as executable specifications for the code, and can replace some documentation and usage examples
* we need involvement from toolsmiths and developers and testers
** toolsmiths maintain the infrastructure, and find/adapt/use tools to solve problems
** developers write developer tests
** testers write end-user tests
** each group shares domain knowledge with the others
= Projects =
* Catalog all existing test suites and/or test cases
** special page for security tests
* investigate existing testing tools
** Watir (possibly using JSSh)
** xulUnit, jsUnit, jsUnit (yes, there are 2), xUnit, FIT
* create test infrastructure architecture
** common invocation and reporting formats for all tests
*** or choose 1 of no more than 3, if 1-size-does-not-fit-all
** put docs on wiki, code in cvs
** follow list of principles for automated testing
*** data-driven (data separate from test code)
*** Keyword driven
**** action keywords
**** fixtures to implement action keywords
*** focus on oracle (how do we know the test passed or failed)
*** do not rule out computer-assisted manual testing (no automated oracle)
*** use model-based test scripting when possible
**** allows for high volume automated testing
**** can walk the state transitions and detect failures other than asserts or crashes
* Community outreach
** identify people in the mozilla community who are already test-infected
** identify test-infected people outside the mozilla community and ask them to help
** hit up grad students who can make this work part of their academic projects
Davel
201

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/15598"

Navigation menu