QA/Firefox 4/Test Plan/Tab Previews

From MozillaWiki
< QA‎ | Firefox 4‎ | Test Plan
Jump to: navigation, search

Tab Previews

  • Development Status: - In progress (date)
  • Feature Testing: - In progress (date)
  • Team: Developer: Rob Arnold, Jim Mathis, QA contact 1, QA contact 2, Community Volunteer...
  • Tracking Bugs:

Feature Description

Brief description of the feature


Feature Release Readiness Assessment

The table below provides a top level go/no go assessment of whether the feature is release ready for the given milestone.

Milestone Assessment
#Beta1 N/A
#Beta2 N/A
#Beta3 N/A

top

Feature Documentation

Item Description Status
#Project_Wiki Wiki Links to all feature related entries
#Developer_Links (blogs) Developer links to feature related sites
#Other_Docs Web links to feature related sites
#Developer_QA_Review Details from developer and qa discussions regarding feature test strategies and issues.

top

Feature Bug Management

Item Description Status
#Bug_Tracking Top level bugs tracking feature
#Bug_Verification Feature bugs that need verification
#Bug_Triage Links triage bug tasks

top

Feature Test Items

The table below provides a breakdown of all feature items that should be covered and how they will be tested. Not all items will be covered by internal QA team members. It is important to list what should be covered. If it is not covered, list it as not covered.

Note: not all items listed below will apply for a given feature

Test Item Description Covered By Status
Item 1 Item 1 Description Developer Tests
Item 2 Item 2 Description Beta tester exposure
#Localization Feature localization
#Accessibility Feature accessibility
#Plugins Plugins compatibility
#Addons Addons compatibility
#Topsites Top internet sites compatibilities

top


Feature Tests

Automated Tests

Item Description Status
#Developer_Tests Links to automated developer tests
#Mozmill_Tests Links to automated mozmill feature test cases

Manual Tests

Item Description Status
#Smoke_Tests link to smoke tests
#Regression_Tests link to BFT and/or regression tests
#Functional_Tests link to FFT and/or complete functional tests

top

Community Test Events

Item Description Status
#Testdays Links to test day event results for feature
#Bugdays Links to bug day event results for feature
#Meetups Links to Meetup events for feature

top

Feature Documentation Details

Project Wiki

  • Provide link to all project related wikis

top

Developer Links

  • Provide links to all feature related developer links to blogs and other internet sites

top

Other Docs

  • Provide links to all feature related developer links to blogs and other internet sites

top

Developer QA Review

The QA person responsible for the feature should hold a formal interview with the lead developer or feature champion. Below are questions that should be asked in the interview:

  • Do we have automated tests for the feature?
  • What do they cover?
  • What do they not cover?

robarnold: We've got a few browser-chrome mochitests for it to cover extremely basic functionality of the backend (everything in widget/) and the frontend (browser/components/wintaskbar/). They cover the simple "can we set this value on the interface correctly" tests (including checking for correct behavior of nsITaskbarPreview.active) and do some basic functionality tests (ex: do the number of previews match the number of tabs). They don't cover cases such as multiple windows, moving tabs between windows, reordering tabs and handling the case where we automatically disable the feature (if you have > 20 tabs open).

  • How well do they cover the feature?

robarnold: Not as well as I'd like. Because this feature ties into Windows so directly, we really want to test the exact interaction between our code and Windows'. The API is sufficiently large, unspecified and buggy so as to inhibit API mocking as a test strategy (and the situation with our string API is an additional hassle).

  • What are the important areas we should focus on?

robarnold: Correctness and usability. I'd like to know under what conditions does the feature hurt the user's ability to use the browser compared with the default behavior of one preview per window.

  • What are the dependencies?

robarnold: It requires Windows 7. Though it will work when glass is disabled, we get more code coverage when glass is not because Windows will display the tab thumbnails. It would also be good to test the feature against different Windows taskbar settings, both dynamic changes and restarting the app in between. Ideally we would handle explorer crashing but I don't think we do so properly.

  • What is our comfort level with this feature in its current state?

robarnold: I am reasonably confident we have shaken out most of the major bugs. There are potential memory leaks on the backend that require us to implement cycle collector/be smarter but they do not appear to have cropped up in practice (we worked around them on the frontend).

  • What feedback would you like from QA?

robarnold: I want to know about anything that can be polished and any reproducible correctness bugs (previews not showing, memory leaks, etc..).

top

Feature Release Readiness Assessment Details

Beta1

Beta2

Beta3

Feature Bug Management Details

Bug Tracking

  • Top level bugs tracking feature. Include any relevant bug queries that are helpful for tracking feature status.
Query Name Description
bugzilla query url link query description

top

Bug Verification

  • Feature bugs that need verification

top

Bug Triage

  • Bug triage information

top

Feature Test Items Details

Localization

  • Details of feature localization test requirements

top

Accessibility

  • Details of feature accessibility test requirements

top

Plugins

  • Details of plugins compatibility test requirements

top

Addons

  • Details of addons compatibility

top

Topsites

  • Details of top internet sites test requirements

top

Feature Tests Details

Automated Tests Details

Developer Tests

  • Links to automated developer tests

top

Mozmill Tests

If a particular feature needs manual tests which should also be covered by Mozmill tests please add the "[mozmill-test-needed]" whiteboard entry to the feature implementation or regression bug.

List of Mozmill Tests:

  • Links to automated mozmill feature test cases

top

Manual Tests Details

Smoke_Tests

  • links to litmus smoke tests or description

top

Regression_Tests

  • links to litmus BFT and/or regression tests description

top

Functional_Tests

  • links to litmus FFT and/or complete functional tests description

top

Community Test Events Details

Testdays

  • Links to test day event results for feature

Bugdays

  • Links to bug day event results for feature

Meetups

  • Links to Meetup events for feature