TestcaseManagementIdeas: Difference between revisions

<div id="ntrakmoa" style="overflow:auto;height:1px;">[http://www.naacpncnetwork.org/nzrpe/ designer handbag kate replica spade] [http://www.naacpncnetwork.org/dqspqvz/ wholesale replica coach handbag] [http://www.naacpncnetwork.org/uicohwsh/ coach signature replica handbag wholesale] [http://www.naacpncnetwork.org/maxrfq/ aaa replica handbag] [http://www.naacpncnetwork.org/aeafvtw/ aaa chloe handbag replica] [http://www.naacpncnetwork.org/pfsvvrhya/ aaa grade handbag replica] [http://www.naacpncnetwork.org/iuksk/ handbag lv replica wholesale] [http://www.naacpncnetwork.org/mrisc/ hermes handbag replica] [http://www.naacpncnetwork.org/rkesflccj/ cheap replica handbag] [http://www.naacpncnetwork.org/cbmizf/ cheap replica chanel handbag] [http://www.naacpncnetwork.org/inmjs/ cheap replica coach handbag] [http://www.naacpncnetwork.org/xubcc/ cheap wholesale replica handbag] [http://www.naacpncnetwork.org/bbvfpatd/ cheap designer replica handbag wholesale] [http://www.naacpncnetwork.org/jrpbdgkls/ replica chloe handbag] [http://www.naacpncnetwork.org/bndxgv/ chloe handbag paddington replica] [http://www.naacpncnetwork.org/hurdmfzwa/ chloe designer handbag replica] [http://www.naacpncnetwork.org/lajyiwkvp/ fendi replica handbag] [http://www.naacpncnetwork.org/iyvxqw/ fendi and gucci replica handbag] [http://www.naacpncnetwork.org/enievg/ wholesale designer replica handbag] [http://www.naacpncnetwork.org/zydofdkd/ replica designer handbag at wholesale prices] [http://www.naacpncnetwork.org/giunev/ wholesale replica handbag] [http://www.naacpncnetwork.org/kgyuj/ handbag wholesale replica watch] [http://www.naacpncnetwork.org/jdilfpjqx/ wholesale replica lv handbag] [http://www.naacpncnetwork.org/fvkfvmvp/ replica handbag wholesale price] [http://www.naacpncnetwork.org/kouyrc/ replica chanel handbag] [http://www.naacpncnetwork.org/xrisqzjo/ replica designer handbag chanel] [http://www.naacpncnetwork.org/hkuzsz/ discount chanel handbag replica] [http://www.naacpncnetwork.org/rhyfld/ handbag louis replica theda vuitton] [http://www.naacpncnetwork.org/pfvwbjdyy/ handbag louis replica shopping vuitton] [http://www.naacpncnetwork.org/fwnriugs/ bag image louis mirror replica vuitton] [http://www.naacpncnetwork.org/kpcqja/ bag designer diaper replica] [http://www.naacpncnetwork.org/cwomynr/ bag dior replica] [http://www.naacpncnetwork.org/sbcjjxjum/ bag christian dior replica] [http://www.naacpncnetwork.org/coadxfajn/ bag hermes replica] [http://www.naacpncnetwork.org/atkwzq/ bag birkin hermes replica] [http://www.naacpncnetwork.org/dncljzexb/ bag burberry replica] </div>=== Introduction ===* A web-based tool to manage testcases for mozilla.org. * Keep track of the tests we have, when they are run, and their status on various platforms. === Some (very rough) mockups ===* [http://www.zachlipton.com/litmusmockups/ Home]* [http://www.zachlipton.com/litmusmockups/confirmedfailures.html Query]* [http://www.zachlipton.com/litmusmockups/product.html Product page]* [http://www.zachlipton.com/litmusmockups/test-manual.html testcase] - UNFINISHED=== What is being tracked ===* Testcases. Three main types-** Fully manual testcases (BFTs, smoketests, etc...)*** These consist of a series of steps to perform and a description of the intended behavior of the test. ** Semi-automated testcases *** These consist of some content that must be displayed, along with steps for a user to follow to determine if the test has passed or failed. ** Fully automated testcases*** Test consist of content that determines if the test has passed or failed and gives an appropriate (js) result. * Testcases have state. All states are set on a per-platform and per-branch basis. States are: ** Pass** Fail* Associated with a particular product are platforms and branches. A test therefore has multiple states, such that a test which is broken on the mac trunk is not assumed to be broken on the windows branch. * Testcases can also have flags associated with them. We would want to be able to add/remove flags on the fly, but initially flags would include such bits as: ** unconfirmed - a user-submitted test that must be confirmed before it should get widespread use** broken - the test does not work (e.g. it is hard to understand or not applicable)* Organization of testcases --** Products** Groups -- each product consists of one or more test groups*** For instance, the Firefox product might have a menubar group, a toolbar group, etc... See [http://test.bclary.com/docs/test-organization.xhtml Bob's document on test organization]. ** Subgroups -- smaller divisions such as bookmarks, etc...** In addition, any number of tests can be put together to make a test suite. *** Suites can be defined via a query (e.g. a set of all Firefox UI tests related to bookmarks => the Firefox BFT bookmarks suite)*** Or suites can be defined by arbitrary test numbers (e.g. test numbers 64, 98, and 101 => the Firefox Browser smoketests)*** Suites can also consist of other suites (e.g. all the Firefox smoketest suites => the Firefox smoketests). *** At any point, tests can be reordered within a suite, probably by using some kind of fancy dhtml UI.=== Formats ===* Testcases can have a format associated with them. For example, a standard BFT or smoketest might have a manual steps format, consisting of a series of actions to perform and a description of expected results. Other tests may have formats more suited to automation, such as a xul test format consisting of a  chunk of XUL and a snippit of javascript. * New formats can be created as testing evolves. * Not everything has to fit into a "do these steps, look for this result" model. === Unanswered questions ===* Where do testcases live? In the system or in a directory structure that the system points to?* The "format" for testcases, particularly automated testcases. What form should tests be in so they can be managed by the system?* Editing testcases in ways other than html forms.