Testing Tools Survey

From MozillaWiki
Jump to: navigation, search

List of Modules

  • Module: None
    • Source Dirs: db, ipc, extensions/irc, extensions/metrics, extensions/schema-validation, extensions/universalchardet, extensions/webdav, extensions/webservices
    • Owners(s):
    • Test Files:
  • Module: Application Suite
    • Source Dirs: extensions/help, suite
    • Owners(s): Neil Rashbrook
    • Test Files: 0
    • No response from e-mail.
  • Module: BeOS-based gfx and widget
    • Source Dirs: gfx/src/beos, widget/src/beos
    • Owners(s): Paul Ashford
    • Test Files: 0
    • Owner disavows association with project.
  • Module: Berkeley DB
    • Source Dirs: dbm
    • Owners(s): Bob Relyea
    • Test Files: 1
    • No response from e-mail.
  • Module: Bugzilla
    • Source Dirs: webtools/bugzilla
    • Owners(s): Dave Miller
    • Test Files: 0
    • Does not have "user-visible" testing, but could. Has not felt to be needed.
  • Module: Build Config
    • Source Dirs: build, config
    • Owners(s): Benjamin Smedberg
    • Test Files: 0
    • The build team is fully engaged in enabling testing.
  • Module: Composer
    • Source Dirs: editor
    • Owners(s): Daniel Glazman
    • Test Files: 1
    • No response from e-mail.
  • Module: Cookies and Permissions
    • Source Dirs: extensions/cookie
    • Owners(s): Darin Fisher
    • Test Files: 0
    • No response from e-mail.
  • Module: DOM Inspector
    • Source Dirs: extensions/inspector, layout/inspector
    • Owners(s): Christopher Aillon
    • Test Files: 1
    • No response from e-mail.
  • Module: Directory SDK
    • Source Dirs: directory
    • Owners(s): Mark C. Smith
    • Test Files: 1
    • An automatable test suite is kept in the repository but not checked out by default. Automated testing would need an LDAP server, such as can be set up with OpenSSL on Mac or Linux.
  • Module: Document Object Model
    • Source Dirs: content/base, content/events, content/html/content, content/html/document, dom
    • Owners(s): Peter Van der Beken, Johnny Stenback
    • Test Files: 41
    • No response from e-mail.
  • Module: Embedding
    • Source Dirs: embedding
    • Owners(s): Benjamin Smedberg
    • Test Files: 319
  • Module: Find As You Type
    • Source Dirs: extensions/typeaheadfind
    • Owners(s): None
    • Test Files: 0
    • No response from e-mail.
  • Module: Firefox
    • Source Dirs: browser, other-licenses/branding/firefox, browser/components/places/tests, browser/components/safebrowsing/tests, browser/components/sessionstore/test
    • Owners(s): Michael Connor
    • Test Files: 6
  • Module: GFX and Widget - General
    • Source Dirs: gfx, gfx/public, gfx/src, gfx/src/shared, widget, widget/public, widget/src, widget/src/xpwidgets
    • Owners(s): Robert O'Callahan
    • Test Files: 5
    • No response from e-mail.
  • Module: GFX and Widget - Mac
    • Source Dirs: gfx/src/mac, widget/src/cocoa, widget/src/mac
    • Owners(s): Mark Mentovai, Josh Aas
    • Test Files: 0
    • No response from e-mail.
  • Module: GTK
    • Source Dirs: gfx/src/gtk, widget/src/gtk, widget/src/gtk2, widget/src/gtksuperwin, widget/src/gtkxtbin
    • Owners(s): Robert O'Callahan, Stuart Parmenter
    • Test Files: 0
    • No response from e-mail.
  • Module: GTK Embedding Widget
    • Source Dirs: ()
    • Owners(s): Josh Soref
    • Test Files: 0
    • No response from e-mail.
  • Module: Global Key Bindings
    • Source Dirs: content/xbl/builtin
    • Owners(s): Aaron Leventhal
    • Test Files: 0
    • No response from e-mail.
  • Module: Graphics - Cairo and Thebes
    • Source Dirs: gfx/cairo, gfx/src/thebes, gfx/thebes
    • Owners(s): Vladimir Vukicevic, Stuart Parmenter
    • Test Files: 0
    • No response from e-mail.
  • Module: Grendel
    • Source Dirs: grendel
    • Owners(s): R.J. Keller
    • Test Files: 0
    • No response from e-mail.
  • Module: HTML Parser
    • Source Dirs: parser/htmlparser
    • Owners(s): Blake Kaplan
    • Test Files: 328
    • Tests are based on perl and configured to run easily on Windows only. Sayres is working on, or has worked on, new tests for WHATWG.
  • Module: HTML to Text/PostScript Translation
    • Source Dirs: gfx/src/ps, gfx/src/psshared
    • Owners(s): Kenneth Herron
    • Test Files: 0
    • No response from e-mail.
  • Module: I18N Library
    • Source Dirs: intl
    • Owners(s): Simon Montagu, Jungshik Shin
    • Test Files: 35
    • No response from e-mail.
  • Module: ImageLib
    • Source Dirs: jpeg, modules/libimg/png, modules/libpr0n, modules/zlib
    • Owners(s): Stuart Parmenter
    • Test Files: 0
    • No response from e-mail.
  • Module: Java APIs for DOM
    • Source Dirs: java/dom
    • Owners(s): Ashutosh Kulkarni
    • Test Files: 0
    • No response from e-mail.
  • Module: Java APIs to WebShell
    • Source Dirs: java/webclient
    • Owners(s): Edward J. Burns
    • Test Files: 0
    • No response from e-mail.
  • Module: Java Stubs
    • Source Dirs: modules/oji, nav-java, sun-java
    • Owners(s): Kyle Yuan
    • Test Files: 778
    • No response from e-mail.
  • Module: Java Utility Classes
    • Source Dirs: java/util
    • Owners(s): Edward J. Burns
    • Test Files: 0
    • No response from e-mail.
  • Module: Java and JS Capability-Based Security
    • Source Dirs: caps, caps/tests
    • Owners(s): Christopher Aillon
    • Test Files: 0
    • No response from e-mail.
  • Module: Java to XPCOM Bridge
    • Source Dirs: extensions/java
    • Owners(s): Javier Pedemonte
    • Test Files: 10
    • Recently fixed to be runnable from "make check" but no results are reported anywhere in particular.
  • Module: Java-Implemented Plugins
    • Source Dirs: java/plugins
    • Owners(s): Igor Kushnirskiy
    • Test Files: 0
    • No response from e-mail.
  • Module: JavaScript
    • Source Dirs: js/src, js/src/config, js/src/editline, js/src/fdlibm
    • Owners(s): Brendan Eich
    • Test Files: 2540
    • Tests are runnable in jssh or in browser. Tests are currently being maintained and are working.
  • Module: JavaScript Debugger Backend
    • Source Dirs: js/jsd
    • Owners(s): Josh Soref
    • Test Files: 0
    • No response from e-mail.
  • Module: Layout Engine
    • Source Dirs: layout, layout/base, layout/build, layout/doc, layout/forms, layout/generic, layout/html, layout/macbuild, layout/printing, layout/tables, layout/tools
    • Owners(s): L. David Baron
    • Test Files: 2733
    • Reftest is current and working. There would be significant value in:
      • converting testcases from fixed bugs into formats that can be run in these suites
      • writing testcases for these test suites based on the specifications being implemented or based on testing in other browsers
      • converting existing test suites for specifications into formats usable by these harnesses
  • Module: LiveConnect
    • Source Dirs: js/src/liveconnect
    • Owners(s): Alfred Peng
    • Test Files: 0
    • Owner is enthusiastic about getting help with collecting LC tests and making sure they are running.
  • Module: MIME
    • Source Dirs: mailnews/mime
    • Owners(s): David Bienvenu, Scott MacGregor
    • Test Files: 0
    • No response from e-mail.
  • Module: Mail and News Backend
    • Source Dirs: mailnews
    • Owners(s): David Bienvenu, Scott MacGregor
    • Test Files: 0
    • No response from e-mail.
  • Module: MathML
    • Source Dirs: layout/mathml
    • Owners(s): Roger B. Sidje
    • Test Files: 2
    • Would be interested in automation of visual checks.
  • Module: Movemail
    • Source Dirs: mailnews/movemail
    • Owners(s): Philip K. Warren
    • Test Files: 0
    • No response from e-mail.
  • Module: Mozilla Tools
    • Source Dirs: tools
    • Owners(s): L. David Baron
    • Test Files: 0
  • Module: Mstone
    • Source Dirs: mstone
    • Owners(s): Dan Christian
    • Test Files: 0
    • No response from e-mail.
  • Module: NSPR
    • Source Dirs: nsprpub
    • Owners(s): Wan-Teh Chang, Nelson Bolyard, Darin Fisher
    • Test Files: 183
    • No response from e-mail.
  • Module: NetLib
    • Source Dirs: netwerk
    • Owners(s): Darin Fisher
    • Test Files: 11
    • No response from e-mail.
  • Module: P3P
    • Source Dirs: extensions/p3p
    • Owners(s): Harish Dhurvasula
    • Test Files: 0
    • No response from e-mail.
  • Module: PerlConnect
    • Source Dirs: js/src/perlconnect
    • Owners(s): Vladimir Livshits
    • Test Files: 0
    • No response from e-mail.
  • Module: Photon
    • Source Dirs: gfx/src/photon, widget/src/photon
    • Owners(s): Adrian Mardare
    • Test Files: 0
    • No response from e-mail.
  • Module: Plugins
    • Source Dirs: modules/plugin
    • Owners(s): Johnny Stenback
    • Test Files: 0
    • No response from e-mail.
  • Module: Preferences
    • Source Dirs: modules/libpref
    • Owners(s):
    • Test Files: 0
    • No response from e-mail.
  • Module: Profile Manager
    • Source Dirs: profile
    • Owners(s):
    • Test Files: 0
    • No response from e-mail.
  • Module: Qt-based gfx and widget
    • Source Dirs: gfx/src/qt, widget/src/qt
    • Owners(s): Zack Rusin
    • Test Files: 0
    • No response from e-mail.
  • Module: RDF
    • Source Dirs: rdf
    • Owners(s): Axel Hecht
    • Test Files: 10
    • Axel is not anguine about the health of the testing available for RDF, but it sounds as though there might be doubt about the longevity of the module.
  • Module: Registry
    • Source Dirs: modules/libreg
    • Owners(s): Dan Veditz
    • Test Files: 2
    • No response from e-mail.
  • Module: Rhino
    • Source Dirs: js/rhino
    • Owners(s): Attila Szegedi
    • Test Files: 0
    • No response from e-mail.
  • Module: SVG
    • Source Dirs: content/svg, layout/svg
    • Owners(s): Tim Rowley
    • Test Files: 0
    • No response from e-mail.
  • Module: Security - Mozilla PSM Glue
    • Source Dirs: security/manager
    • Owners(s): Kai Engert, John G. Myers
    • Test Files: 0
    • No response from e-mail.
  • Module: Standalone Composer
    • Source Dirs: composer
    • Owners(s): Daniel Glazman
    • Test Files: 0
    • No response from e-mail.
  • Module: String
    • Source Dirs: string, xpcom/string
    • Owners(s): Darin Fisher
    • Test Files: 0
    • No response from e-mail.
  • Module: Style System
    • Source Dirs: layout/style
    • Owners(s): L. David Baron
    • Test Files: 0
  • Module: Tamarin
    • Source Dirs: js/tamarin
    • Owners(s): Dan Smith
    • Test Files: 0
    • Dan is willing to see what can be done for testing Tamarin for FF4.
  • Module: Testing Infrastructure
    • Source Dirs: testing
    • Owners(s): Rob Campbell
    • Test Files: 101
  • Module: Thunderbird
    • Source Dirs: mail, other-licenses/branding/thunderbird
    • Owners(s): Scott MacGregor, David Bienvenu
    • Test Files: 0
    • No response from e-mail.
  • Module: Update Service
    • Source Dirs: webtools/addons, webtools/update
    • Owners(s): Mike Morgan
    • Test Files: 0
    • No response from e-mail.
  • Module: Venkman
    • Source Dirs: extensions/venkman
    • Owners(s): Robert Ginda
    • Test Files: 4
    • No response from e-mail.
  • Module: View System
    • Source Dirs: view
    • Owners(s): Robert O'Callahan
    • Test Files: 0
    • No response from e-mail.
  • Module: Win32-Based GFX and Widget
    • Source Dirs: gfx/src/windows, widget/src/windows
    • Owners(s): Ere Maijala
    • Test Files: 0
    • The current testing is ad hoc and not complete enough.
  • Module: XBL
    • Source Dirs: content/xbl, content/xbl/public, content/xbl/src
    • Owners(s): Jonas Sicking, Boris Zbarsky
    • Test Files: 0
    • No tests of XBL exist, but both owners want more testing.
  • Module: XML
    • Source Dirs: content/xml, extensions/xmlextras, parser/expat
    • Owners(s): Peter Van der Beken
    • Test Files: 57
    • No response from e-mail.
  • Module: XPApps
    • Source Dirs: xpfe
    • Owners(s): Neil Rashbrook
    • Test Files: 0
    • No response from e-mail.
  • Module: XPCOM
    • Source Dirs: xpcom, xpcom/base, xpcom/build, xpcom/components, xpcom/doc, xpcom/ds, xpcom/glue, xpcom/MoreFiles, xpcom/obsolete, xpcom/proxy, xpcom/sample, xpcom/stub, xpcom/tests, xpcom/threads, xpcom/tools, xpcom/windbgdlg
    • Owners(s): Doug Turner
    • Test Files: 96
    • No response from e-mail.
  • Module: XPConnect
    • Source Dirs: js/src/xpconnect
    • Owners(s): David Bradley
    • Test Files: 112
    • No response from e-mail.
  • Module: XPIDL
    • Source Dirs: xpcom/typelib
    • Owners(s): David Bradley
    • Test Files: 4
    • No response from e-mail.
  • Module: XPInstall
    • Source Dirs: xpinstall
    • Owners(s): Dan Veditz
    • Test Files: 0
    • No response from e-mail.
  • Module: XPToolkit
    • Source Dirs: content/xul, layout/xul
    • Owners(s):
    • Test Files: 0
    • No response from e-mail.
  • Module: XPrint
    • Source Dirs: gfx/src/xprint
    • Owners(s): Masaki Katakai
    • Test Files: 0
    • No response from e-mail.
  • Module: XSLT Processor
    • Source Dirs: content/xslt, extensions/transformiix
    • Owners(s): Peter Van der Beken
    • Test Files: 24
    • No response from e-mail.
  • Module: XTF
    • Source Dirs: content/xtf, layout/xtf
    • Owners(s):
    • Test Files: 0
    • No response from e-mail.
  • Module: Xlib-based gfx + widget
    • Source Dirs: gfx/src/xlib, widget/src/xlib, widget/src/xlibxtbin
    • Owners(s): Christopher Blizzard
    • Test Files: 0
  • Module: accessibility
    • Source Dirs: accessibility
    • Owners(s): Aaron Leventhal
    • Test Files: 0
    • No response from e-mail.
  • Module: calendar
    • Source Dirs: calendar
    • Owners(s): Michiel van Leeuwen
    • Test Files: 0
    • No response from e-mail.
  • Module: docshell
    • Source Dirs: docshell, uriloader, webshell
    • Owners(s): Benjamin Smedberg
    • Test Files: 0
  • Module: ef
    • Source Dirs: ef
    • Owners(s): Scott Furman
    • Test Files: 0
    • No response from e-mail.
  • Module: js-tests
    • Source Dirs: js/tests
    • Owners(s): Bob Clary
    • Test Files: 0
  • Module: mozilla-toplevel
    • Source Dirs: mozilla
    • Owners(s): Brendan Eich
    • Test Files: 0
    • No response from e-mail.
  • Module: security
    • Source Dirs: security/coreconf, security/dbm, security/jss, security/nss
    • Owners(s): Wan-Teh Chang, Bob Relyea, Nelson Bolyard, Julien Pierre
    • Test Files: 287
    • Tests run on tinderbox. There is a full-time person at Sun who works on NSS. Tests should be current and working.
  • Module: toolkit
    • Source Dirs: chrome, storage, toolkit, xulrunner
    • Owners(s): Benjamin Smedberg
    • Test Files: 2
  • Module: xptcall
    • Source Dirs: xpcom/reflect/xptcall
    • Owners(s): Josh Soref
    • Test Files: 15
    • No response from e-mail.

Categories of Information

  • Test suite/test procedure name
    • Any useful name that will uniquely identify the set of tests
  • Brief description of the test suite/procedure
    • The purpose and goals for the test. What is the scope of the test? Examples: DOM parser, Layout engine, key bindings. Are there any major areas known to be missing from the test?
  • Test Harness
    • What framework or context does the test suite run in? Examples: none, custom script, mochikit, XPCshell, jsunit, litmus, silk
  • Status of automation
    • Examples: manual procedure, computer guided, semi-automated (some manual sets with setup, launching, verifying results), fully-automated
  • When is the test run?
    • Examples: every integration, nightly, every milestone, as needed/desired, never