« QA/Firefox3/TestPlan

Overview

This Testplan will focus on the rework of Bookmarks and History as they are transformed into the Places component for Firefox 3. From the Places feature plan:

The primary objectives of this project are to:

   * Improve access to History and Bookmarks
   * Make it easier for people to Bookmark pages 

Secondary objectives include:

   * Consolidating user data formats
   * Providing a platform for using interesting per-URL metadata
   * Improving the capabilities of Live Bookmarks
   * Providing a solid architecture for bookmark sync and remote storage

Scope of planned testing

Coverage includes:

• Basic Functional tests will cover the most common user based functions of Bookmark and History (litmus BFT)
• Full Functional test will cover the BFT's and a much deeper level of functional testing. (litmus FFT)

• Unit tests are run with every tinderbox cycle.

Platform and Configurations

Places must work on all platforms Firefox 3.0 will support.

Major Test Areas

• Functionality: All areas of Bookmarks, History, Tagging and to some extent RSS and Microsummaries
  • Pull test cases from the bugs filed on this component and convert them to Litmus test cases. Especially for the P1 and P2 items of the Places PRD.
    • Bugs with P1/P2 in-litmus? requests
    • All other bugs with in-litmus requests
    • Bugs with test cases in-litmus+
  • Also expand around those test cases in areas that appear slim in Litmus
  • Write new Litmus test cases and rework old ones where applicable for the new Places UI

• Performance:
  • Alice used Talos to test builds during Places, Places + bookmarks and no Places
    • those results can be found on the graph server
  • Bugs Tracking various performance issues:
    • Open issues:
      • perf bugs
      • slow bugs
    • Fixed issues:
      • perf bugs
      • slow bugs

• Extensibility:
  • Extension developers will have to stay on top of changes affecting their extensions.

Schedule/milestones

Milestones: The milestone schedule continues to be in flux, but this is how it roughly lays out for the next couple of milestones: (using M series where M1 was A1.. M9 should be B1)

• M10: more UI and perf tweaks
• M9: UI and perf tweaks
• M8: Places front end to land
• M7: performance tweaks and push to parity with 2.0 with protection against data loss
• M5-M6: backend of bookmarks and history landed

Testing Schedule:

• Cyclic Plan:
  • Meet with the Places dev team a week or so before each milestone freeze to plan out the testing strategy for that milestone (what needs specific attention, big landings to prep for, etc..)
  • Since so much is changing with Places, I think it makes sense to run FFT's as often as is reasonable. This will help us catch bugs early in each milestone cycle. An FFT every 4 weeks (even if it's spread out over a few days).
  • BFT's for post 3.0.0.x stable testing

regular testing:

• bug fix verifications
• daily smoketests ( a very small set of Places related tests)
• Unit tests are run with every tinderbox cycle.

References

• Places 3.0 feature plan Chock full of info and links to more info.
• Places section of the PRD
• Places Task List (dev) Kind of old but has useful info and links.
• Places 2006 summit notes; a good quick look at what is expected of Places.
• Devs and QA folk for Places hang out in #Places on irc.
• This page linked from Firefox 3.0 test plan