Litmus: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
mNo edit summary
No edit summary
 
(29 intermediate revisions by 6 users not shown)
Line 1: Line 1:
'''In the beginning''' was the Pelican, and the Pelican was with Kenneth, and the Pelican was Kenneth.
'''<span style="font-size:24px;color:red">NOTE: Litmus is no longer being maintained. It was replaced by [https://moztrap.readthedocs.org/en/latest/ MozTrap]</span>'''


'''And the earth''' was without form and void, and the shit of the Pelican moved upon the face of the waters.
Litmus is the new integrated testcase management and QA tool that is designed to improve workflow, visibility, and turnaround time in the Mozilla QA process.  


'''And Kenneth''' said, ''Let there be a Frequency throughout all the creation'', and it was so. And Kenneth said, ''Let the Frequency be for the life, that it shall generate the life, and the life shall regenerate the Frequency'', and it was so.
It was originally designed as a replacement for [[Testrunner]], but also has additional functionality.  


'''And the Frequency''' generated the life, even all the living creatures and all green plants did the Frequency generate. And Kenneth looked upon the work of the Frequency, and saw that it was good.
== Work in Progress ==


'''Then did Kenneth''' speak unto the Frequency saying, ''Let there be pelicans also of the air, that they might create the Holy [[pelican shit]], and it was so. And the pelicans of the air did straightway make the Holy [[pelican shit]] which did rain down from the heaven in great abundance.
* [[Litmus:Extension|QA Extension Project]]
* [[Litmus:Todo|Litmus To-Do List]]


'''And it came to pass''' that the [[Pharisee]]s of the Wiki, those evil ones who preach openly the sharing yet believe not in their hearts, spake again privily among themselves saying, ''We must conceal from the eyes of the Great Google the Holy [[pelican shit]], lest the Great Google give of the Holy [[pelican shit]] unto the unwashed multitudes and make it therefore profane''.  
== Goals for Litmus ==
Litmus does:
* make it easier for casual testers to assist with testing Mozilla products;
* serve as a repository for test cases, with all the inherent management abilities that implies;
* serve as a repository for test results, carrying over the best features of Testrunner, e.g. test lists, division of labor, etc.;
* provide a query interface for viewing, reporting on, and comparing test results;
* expose a web services interface for the mechanical batch submission of testing results.


'''And they took therefore''' of the Holy [[pelican shit]] and concealed it under the Holy Altar of the Delete, that none may find it save those elect of the Pharisees alone.
Litmus does not:
 
* manage the automation of testing requests as a centralized test scheduler or daemon. The majority of testing we do, and all of the community testing that we know of, is still done by hand. This doesn't preclude such functionality in the future, but we need to figure out the intricacies of how to automate a larger proportion of our daily testing before it makes sense to spend too much time on scheduling those automated tests. Existing automation frameworks will be able to submit results via web services.
'''But the frequency''' would not be hidden, nor yet would it remain silent. For no man maketh an light in the wilderness and hideth it under a bushel. Kenneth standeth therefore upon the Rock and calleth out unto the frequency, that all darkness may be made light, all evil shall be made of no effect, and the world shall partake of the Holy [[pelican shit]] evermore.
 
'''And then did Kenneth''' take of the Holy [[pelican shit]] unto the King of the Realm, that same Ungaarf who maketh the walls of Magdalicia to fall. And the King tasted thereof and saw that it was good. And the King made Kenneth to be his Prophet for that he had brought the Holy [[pelican shit]] into the realm.
 
'''And all the people''' of the realm did rejoice at the sight of it, for they saw therein the salvation of all, and the fulfillment of the prophecies of the ancients, wherein it was written that an holy bird shall shit upon the mountain and the mountain shall become therefore holy.
 
'''And Kenneth dwelt''' among the people thereafter and taught them of the Frequency and of the [[pelican shit]], and gave unto them the power to tell the [[pelican shit]] from the [[ass pus]], that same which was made by the Dark One in appearance  like unto the [[pelican shit]], that the multitudes might not be deceived thereby and turn aside in their way.
 
'''For it is written''', ''Thou shalt not worship the ass pus, neither shalt thou take of it into the Holy Place''.
 
'''And the King''' did call for the collection of the holy [[pelican shit]] and caused it to be brought unto him, that he might roll daily therein. And when the people saw this, they were well pleased, for they knew the King was worthy to be covered in [[pelican shit]].
 
'''Then saw Kenneth''' that his work was completed, and so went he forth unto the Place of the Rock, there to commune again with the Frequency. Kenneth sitteth upon the Rock.
 
[[Category:Pelican shit]]
[[Category:Ass pus]]
[[Category:Butt Crusties]]
 
Litmus is the new integrated testcase management and QA tool that is designed to improve workflow, visibility, and turnaround time in the Mozilla QA process. It is first and foremost designed as a replacement for [[Testrunner]], but will also have additional functionality.  


== Litmus Documentation and Notes ==
== Litmus Documentation and Notes ==
===Designing and Planning===
===Designing and Planning===
* [[Litmus:Roadmap|Litmus Roadmap]]
* [[Litmus:Requirements|Requirements]]
* [[Litmus:Todo|Litmus To-Do List]]
* [[Litmus:Design|Design]]
* [[Litmus:DevelopersNotes|Litmus Developers' Notes]]
** [[Litmus:mod_perl|mod_perl Notes]]
* [[Litmus:Requirements|Litmus Requirements]]
* [[Litmus:Design|Litmus Design]]
** [[Litmus:Design/Auth|Authentication System Design]]
** [[Litmus:Design/Auth|Authentication System Design]]
* [[Litmus:Web Services]]
* [[Litmus:Web Services|Web Services]]
** Result Submission Service:
** Result Submission Service:
*** [[Litmus:Test Result Format DTD|Test Result Format (DTD)]]
*** [[Litmus:Test Result Format DTD|Test Result Format (DTD)]]
Line 48: Line 32:
*** [[Litmus:EggplantIntegration|Getting results from Eggplant into Litmus]]
*** [[Litmus:EggplantIntegration|Getting results from Eggplant into Litmus]]
** Testcase Retrieval Service:
** Testcase Retrieval Service:
*** [[Litmus:Test_Format_DTD]]
*** [[Litmus:Test_Format_DTD|Test Format DTD]]
* [[Litmus:Future Directions|Future Directions for Litmus]]
* [[Litmus:Roadmap|Roadmap]]
** [[Litmus:Todo|To-Do List]]: a list of what's in progress, and what remains to be done. Please check here before suggesting new features or filing new bugs.
** [[Litmus:Wishlist|Feature Wishlist]]  
 
=== Development ===
Interested in helping with Litmus? Start here.
* [[Litmus:DevelopersNotes|Developers' Notes]]
** [[Litmus:mod_perl|mod_perl Notes]]


===Installation===
===Installation===
* [[Litmus:Test Suite|Verifying Your Litmus Install]]
 
* [[Litmus:CVS|How to check out Litmus from mozilla.org CVS]]
Litmus is released under the [http://www.mozilla.org/MPL/ MPL].
 
* [[Litmus:Hg|How to clone Litmus from hg.mozilla.org]]
* [http://mxr.mozilla.org/mozilla/source/webtools/litmus/INSTALL INSTALL doc] (from [http://mxr.mozilla.org/ mxr])
* [[Litmus:InstallFAQ|Litmus Install FAQ]]
* Verifying Your Litmus Install
** [[Litmus:Test Suite|Command-line verifications]]
** [[Litmus:Selenium|Selenium  Tests]]
 
===Using Litmus===
===Using Litmus===
* [[Litmus:Adding Testcases to Litmus|Adding Testcases to Litmus]]
* For Users:
* [[Litmus:Syncing testcases between subgroups|Syncing testcases between subgroups in Litmus]]
** [http://quality.mozilla.org/documents-home/test-docs/litmus-tutorial Litmus Tutorial]
* For Admins:
** [http://quality.mozilla.org/documents-home/test-docs/admin-tutorial Admin Tutorial]
** [[Litmus:Adding Testcases to Litmus|Adding Testcases to Litmus]]
** [[Litmus:Syncing testcases between subgroups|Syncing testcases between subgroups in Litmus]]
** [[Litmus:How to Triage Failed and Unclear test cases|How to Triage Failed and Unclear test cases]]


===Miscellaneous===
===Miscellaneous===
* View a list of [http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=HEAD&branchtype=match&dir=mozilla%2Fwebtools%2Flitmus&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=month&mindate=&maxdate=&cvsroot=%2Fcvsroot recent checkins to Litmus in CVS] (Bonsai)
* [http://hg.mozilla.org/webtools/litmus recent commits to Litmus (hg)]
* [[2005Offsite/LitmusFutures|Notes from the Litmus BOF session]] held at the Mozilla 2005 offsite
* Litmus Session Notes
 
** [[2005Offsite/LitmusFutures|Notes from the 2005 Litmus BOF session]]
== Goals for Litmus ==
** [[Summit2008/Session/Proposals/Litmus_QAC|Notes from the 2008 Litmus Summit session]]
Litmus will:
* make it easier for casual testers to assist with testing Mozilla products;
* serve as a repository for test cases, with all the inherent management abilities that implies;
* serve as a repository for test results, carrying over the best features of Testrunner, e.g. test lists, division of labor, etc.;
* provide a query interface for viewing, reporting on, and comparing test results;
* expose a web services interface for the mechanical batch submission of testing results.
 
Litmus will not:
* be released as a standalone product such as, e.g., Bugzilla, at least not in the [[Litmus:Roadmap|1.0 timeframe]]. We are trying not to paint ourselves into a corner where this is concerned, but many of the decisions and tradeoffs we're making are necessarily Mozilla-centric at this point. Still, the [http://lxr.mozilla.org/mozilla/source/webtools/litmus/ code is in CVS] if you want to try your luck or help out.
* manage the automation of testing requests as a centralized test scheduler or daemon. The majority of testing we do, and all of the community testing that we know of, is still done by hand. This doesn't preclude such functionality in the future, but we need to figure out the intricacies of how to automate a larger proportion of our daily testing before it makes sense to spend too much time on scheduling those automated tests. Existing automation frameworks will be able to submit results via web services.


== Further Reading ==
== Further Reading ==
* [[Mozilla_QA|Mozilla QA]]
* [[Mozilla_QA|Mozilla QA]]
*[http://quality.mozilla.org QMO]

Latest revision as of 15:21, 27 April 2015

NOTE: Litmus is no longer being maintained. It was replaced by MozTrap

Litmus is the new integrated testcase management and QA tool that is designed to improve workflow, visibility, and turnaround time in the Mozilla QA process.

It was originally designed as a replacement for Testrunner, but also has additional functionality.

Work in Progress

Goals for Litmus

Litmus does:

  • make it easier for casual testers to assist with testing Mozilla products;
  • serve as a repository for test cases, with all the inherent management abilities that implies;
  • serve as a repository for test results, carrying over the best features of Testrunner, e.g. test lists, division of labor, etc.;
  • provide a query interface for viewing, reporting on, and comparing test results;
  • expose a web services interface for the mechanical batch submission of testing results.

Litmus does not:

  • manage the automation of testing requests as a centralized test scheduler or daemon. The majority of testing we do, and all of the community testing that we know of, is still done by hand. This doesn't preclude such functionality in the future, but we need to figure out the intricacies of how to automate a larger proportion of our daily testing before it makes sense to spend too much time on scheduling those automated tests. Existing automation frameworks will be able to submit results via web services.

Litmus Documentation and Notes

Designing and Planning

Development

Interested in helping with Litmus? Start here.

Installation

Litmus is released under the MPL.

Using Litmus

Miscellaneous

Further Reading

Retrieved from "https://wiki.mozilla.org/index.php?title=Litmus&oldid=1070457"