QA/Firefox 4/Test Plan/Tab Previews
Contents
- 1 Tab Previews
- 2 Feature Release Readiness Assessment
- 3 Feature Documentation
- 4 Feature Bug Management
- 5 Feature Test Items
- 6 Feature Tests
- 7 Community Test Events
- 8 Feature Documentation Details
- 9 Feature Release Readiness Assessment Details
- 10 Feature Bug Management Details
- 11 Feature Test Items Details
- 12 Feature Tests Details
- 13 Community Test Events Details
Tab Previews
- Development Status: - In progress (date)
- Feature Testing: - In progress (date)
- Team: Developer: Rob Arnold, Jim Mathis, QA contact 1, QA contact 2, Community Volunteer...
- Tracking Bugs:
Feature Description
Brief description of the feature
Feature Release Readiness Assessment
The table below provides a top level go/no go assessment of whether the feature is release ready for the given milestone.
Milestone | Assessment |
---|---|
#Beta1 | N/A |
#Beta2 | N/A |
#Beta3 | N/A |
Feature Documentation
Item | Description | Status |
---|---|---|
#Project_Wiki | Wiki Links to all feature related entries | |
#Developer_Links (blogs) | Developer links to feature related sites | |
#Other_Docs | Web links to feature related sites | |
#Developer_QA_Review | Details from developer and qa discussions regarding feature test strategies and issues. |
Feature Bug Management
Item | Description | Status |
---|---|---|
#Bug_Tracking | Top level bugs tracking feature | |
#Bug_Verification | Feature bugs that need verification | |
#Bug_Triage | Links triage bug tasks |
Feature Test Items
The table below provides a breakdown of all feature items that should be covered and how they will be tested. Not all items will be covered by internal QA team members. It is important to list what should be covered. If it is not covered, list it as not covered.
Note: not all items listed below will apply for a given feature
Test Item | Description | Covered By | Status |
---|---|---|---|
Item 1 | Item 1 Description | Developer Tests | |
Item 2 | Item 2 Description | Beta tester exposure | |
#Localization | Feature localization | ||
#Accessibility | Feature accessibility | ||
#Plugins | Plugins compatibility | ||
#Addons | Addons compatibility | ||
#Topsites | Top internet sites compatibilities |
Feature Tests
Automated Tests
Item | Description | Status |
---|---|---|
#Developer_Tests | Links to automated developer tests | |
#Mozmill_Tests | Links to automated mozmill feature test cases |
Manual Tests
Item | Description | Status |
---|---|---|
#Smoke_Tests | link to smoke tests | |
#Regression_Tests | link to BFT and/or regression tests | |
#Functional_Tests | link to FFT and/or complete functional tests |
Community Test Events
Item | Description | Status |
---|---|---|
#Testdays | Links to test day event results for feature | |
#Bugdays | Links to bug day event results for feature | |
#Meetups | Links to Meetup events for feature |
Feature Documentation Details
Project Wiki
- Provide link to all project related wikis
Developer Links
- Provide links to all feature related developer links to blogs and other internet sites
Other Docs
- Provide links to all feature related developer links to blogs and other internet sites
Developer QA Review
The QA person responsible for the feature should hold a formal interview with the lead developer or feature champion. Below are questions that should be asked in the interview:
- Do we have automated tests for the feature?
- What do they cover?
- What do they not cover?
robarnold: We've got a few browser-chrome mochitests for it to cover extremely basic functionality of the backend (everything in widget/) and the frontend (browser/components/wintaskbar/). They cover the simple "can we set this value on the interface correctly" tests (including checking for correct behavior of nsITaskbarPreview.active) and do some basic functionality tests (ex: do the number of previews match the number of tabs). They don't cover cases such as multiple windows, moving tabs between windows, reordering tabs and handling the case where we automatically disable the feature (if you have > 20 tabs open).
- How well do they cover the feature?
robarnold: Not as well as I'd like. Because this feature ties into Windows so directly, we really want to test the exact interaction between our code and Windows'. The API is sufficiently large, unspecified and buggy so as to inhibit API mocking as a test strategy (and the situation with our string API is an additional hassle).
- What are the important areas we should focus on?
robarnold: Correctness and usability. I'd like to know under what conditions does the feature hurt the user's ability to use the browser compared with the default behavior of one preview per window.
- What are the dependencies?
robarnold: It requires Windows 7. Though it will work when glass is disabled, we get more code coverage when glass is not because Windows will display the tab thumbnails. It would also be good to test the feature against different Windows taskbar settings, both dynamic changes and restarting the app in between. Ideally we would handle explorer crashing but I don't think we do so properly.
- What is our comfort level with this feature in its current state?
robarnold: I am reasonably confident we have shaken out most of the major bugs. There are potential memory leaks on the backend that require us to implement cycle collector/be smarter but they do not appear to have cropped up in practice (we worked around them on the frontend).
- What feedback would you like from QA?
robarnold: I want to know about anything that can be polished and any reproducible correctness bugs (previews not showing, memory leaks, etc..).
Feature Release Readiness Assessment Details
Beta1
Beta2
Beta3
Feature Bug Management Details
Bug Tracking
- Top level bugs tracking feature. Include any relevant bug queries that are helpful for tracking feature status.
Query Name | Description |
---|---|
bugzilla query url link | query description |
Bug Verification
- Feature bugs that need verification
Bug Triage
- Bug triage information
Feature Test Items Details
Localization
- Details of feature localization test requirements
Accessibility
- Details of feature accessibility test requirements
Plugins
- Details of plugins compatibility test requirements
Addons
- Details of addons compatibility
Topsites
- Details of top internet sites test requirements
Feature Tests Details
Automated Tests Details
Developer Tests
- Links to automated developer tests
Mozmill Tests
If a particular feature needs manual tests which should also be covered by Mozmill tests please add the "[mozmill-test-needed]" whiteboard entry to the feature implementation or regression bug.
List of Mozmill Tests:
- Links to automated mozmill feature test cases
Manual Tests Details
Smoke_Tests
- links to litmus smoke tests or description
Regression_Tests
- links to litmus BFT and/or regression tests description
Functional_Tests
- links to litmus FFT and/or complete functional tests description
Community Test Events Details
Testdays
- Links to test day event results for feature
Bugdays
- Links to bug day event results for feature
Meetups
- Links to Meetup events for feature