Calendar:QA Improvements

From MozillaWiki
Jump to: navigation, search

There are no bad ideas. So, if you have an idea for improving the type of QA done on Lightning and Sunbird, please list it here. Feel free to expand on ideas, add to existing ones, and start your own idea sections. This is all free-form. Go. Have fun.

If you format new sections like this, the table of contents will make us a nifty outline for the page:

Idea Title

Idea description, feel free to format it however you like

Some starter ideas:

Measure Code Coverage of Litmus test cases

  • We need to measure the code coverage of each Litmus section to determine whether the cases we have there are good enough in each section.
  • Perhaps we can do this objectively using the Venkman profiler?
    • How could we do this and not take a ton of time in doing it?
  • As a result we have to revise the Litmus testcases and organize them better.

Improve attendance at Test Days

  • Pass out virtual beer on IRC :-)
  • We announce on: blog, newsgroup, mozillazine...where else can we go?

Automate testing

  • Use XPCShell to automate low level interface testing across the interfaces for the calendar components (code coverage!).
  • Create unit tests for bugs which get fixed and have appropriate testcases for effective regression testing.
  • ICS file pool with automated standard tests.
  • Introduce in-testsuite flag (like Toolkit).
  • Enable test on tinderboxen (with reporting TUnit).
  • Can we turn on libical tests? Would that help or just clutter things?
  • MochiTest is probably not going to help us with testing, but the Eggplant UI automation tool might. (MochiTest/Browser Chrome Test Framework only on Trunk) There is the brand new MozMill.

Performance testing

  • Startup & loading time of (remote) calendars
  • Loading of calendar views (provider performance)

Compatibility testing

  • For calendar application inter-compatibility with other calendar applications and (calendar) servers is crucial.
  • Define a set of applications and servers.
    • Server
      • CalDAV
        • Apple iCal
        • RSCDS
        • Cosmo
        • Apache with mod_caldav
      • WebDAV
        • Apache with mod_webdav
        • Microsoft WebDav
        • Open Exchange WebDav
      • FTP
      • SMB share
    • Applications (for Invitation Handling (iTIP))
      • Outlook flavors
      • Windows Calendar
      • Apple iCal
      • Evolution
      • KOrganizer
  • Create testcases for ICS files from different applications (ICS file pool).
  • Set up (calendar) servers and associated Litmus testcases. (no automation possible?)