QA/Firefox 3.next/Test Plan:AddonsManagerRedesign: Difference between revisions

From MozillaWiki
< QA
Jump to navigation Jump to search
 
(28 intermediate revisions by 3 users not shown)
Line 1: Line 1:
<small>[[QA/Firefox_3.next/Test_Plan|&laquo; Firefox 3.next TestPlan]]</small>
<small>[[QA/Firefox_4/Test_Plan|&laquo; Firefox 4 TestPlan]]</small>


= Add-ons Manager Redesign =
= Add-ons Manager Redesign =
Line 16: Line 16:
| By group:
| By group:
* Jennifer Boriss (UX)
* Jennifer Boriss (UX)
* Dave Townsend, Blair McBride, Robert Strong, Ben Parr (Add-ons Manager)
* Dave Townsend, Blair McBride, Robert Strong, Ben Parr, Benedict Hsieh (Add-ons Manager)
* Justin Scott, Jeff Balogh (Discovery Pane)
* Justin Scott, Jeff Balogh (Discovery Pane)
* Brian King, David McNamara (Add-on Compatibility Reporter)
* Brian King, David McNamara (Add-on Compatibility Reporter)
Line 26: Line 26:
* [https://bugzilla.mozilla.org/show_bug.cgi?id=461973 Extension Manager API Rewrite]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=461973 Extension Manager API Rewrite]
* [https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&component=Discovery%20Pane&product=addons.mozilla.org Discovery Pane]
* [https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&component=Discovery%20Pane&product=addons.mozilla.org Discovery Pane]
* [http://bugzilla.mozilla.org/show_bug.cgi?id=564870 Add-on Compatibility Reporter]
* [http://bugzilla.mozilla.org/show_bug.cgi?id=565930 Add-on Compatibility Reporter]
* [http://bugzilla.mozilla.org/show_bug.cgi?id=569667 Pull More AMO Data]
* [http://bugzilla.mozilla.org/show_bug.cgi?id=569667 Pull More AMO Data]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=563909 Accessibility]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=563909 Accessibility]
Line 34: Line 34:


=== Feature Description  ===
=== Feature Description  ===
This redesign will substantially improve the basic functionality of the a add-ons manager, such as installation and configuration of add-ons, as well as incorporate additional features and functionality, such as use of Personas and Jetpacks. The design will also remove parts of the manager that interrupt and distract user tasks.  
This redesign will substantially improve the basic functionality of the a add-ons manager, such as installation and configuration of add-ons, as well as incorporate additional features and functionality, such as use of Personas and Jetpacks. The design will also remove parts of the manager that interrupt and distract user tasks.
 


= [[#Feature Release Readiness Assessment_Details|Feature Release Readiness Assessment]] =
= [[#Feature Release Readiness Assessment_Details|Feature Release Readiness Assessment]] =
Line 47: Line 46:
|-
|-
| [[#Beta1]]
| [[#Beta1]]
| N/A
| Ready to Ship
|-
|-
| [[#Beta2]]
| [[#Beta2]]
Line 231: Line 230:


== Developer Links ==
== Developer Links ==
* [10/05/27] [http://www.oxymoronical.com/blog/2010/05/Support-for-dropping-XPI-files-into-the-extension-install-locations Support for dropping XPI files into the extension install locations might be going away]
* [10/05/10] [http://theunfocused.net/2010/05/10/status-update-extension-manager-ui-app-tabs/ Frontend Updates]
* [10/04/29] [http://www.oxymoronical.com/blog/2010/04/The-new-add-ons-manager-is-here The new Add-ons Manager is here]
* [10/04/27] [http://theunfocused.net/2010/04/27/status-update-extension-manager-ui/ Frontend Updates]
* [10/04/13] [http://theunfocused.net/2010/04/13/status-update-extension-manager-ui-tab-matches/ Frontend Updates]
* [10/04/01] [http://jboriss.wordpress.com/2010/04/01/add-ons-manager-redesign-update/ Add-ons Manager Redesign Update]
* [10/03/24] [http://blog.fligtar.com/2010/03/24/a-new-front-door/ A New Front Door]
* [10/03/22] [http://theunfocused.net/2010/03/23/status-update-extension-manager-ui-redesign/ Frontend Updates]
* [10/03/19] [http://www.oxymoronical.com/blog/2010/03/How-were-breaking-some-extensions-in-the-near-future How we’re breaking some extensions in the near future]
* [10/03/14] [http://www.oxymoronical.com/blog/2010/03/Look-Ma-no-restarts Look Ma, no restarts!]
* [10/03/12] [http://www.oxymoronical.com/blog/2010/03/Simplifying Simplifying]
* [10/01/07] [http://www.oxymoronical.com/blog/2010/01/Do-we-need-extension-dependencies Do we need extension dependencies?]
* [09/12/03] [http://jboriss.wordpress.com/2009/12/03/relocating-firefox%E2%80%99s-add-ons-manager/ Relocating Firefox’s Add-ons Manager]
* [09/11/23] [http://jboriss.wordpress.com/2009/11/23/redesigning-firefoxs-addons-manager/ Redesigning Firefox’s Addons Manager]


[[#Feature_Documentation|top]]
[[#Feature_Documentation|top]]


== Other Docs ==
== Other Docs ==
* Provide links to all feature related developer links to blogs and other internet sites
* [https://wiki.mozilla.org/AMO:AOM_Meeting AMO Meeting notes]


[[#Feature_Documentation|top]]
[[#Feature_Documentation|top]]
Line 271: Line 284:
! Description
! Description
|-
|-
|bugzilla query url link
| Add-ons Manager (interface and backend):
|query description
* Blocker: [http://tinyurl.com/24858er alpha5]
* Blocker: [http://tinyurl.com/2fjgzl9 beta1]
* Blocker: [http://tinyurl.com/2fn33bb beta2]
* Blocker: [http://tinyurl.com/2b45cr7 final]
* Triage: [http://tinyurl.com/235oa2m uiwanted]
* Triage: [http://tinyurl.com/2blqg7h all]
* Triage: [http://tinyurl.com/242j28v open]
* Triage: [http://tinyurl.com/2496tsm unconfirmed]
* Tests: [http://tinyurl.com/2fgn8y3 flags missing]
* Tests: [http://tinyurl.com/28wrv8o in-testsuite?]
* Tests: [http://tinyurl.com/2843lm8 in-testsuite+]
* Tests: [http://tinyurl.com/27acxzj in-litmus?]
* Tests: [http://tinyurl.com/29jh5b5 in-litmus+]
|
|-
| Discovery Pane:
|
|-
| Add-ons Compatibility Reporter:
* Triage: [http://tinyurl.com/2ftkw4u all]
* Triage: [http://tinyurl.com/28bvk7l open]
* Triage: [http://tinyurl.com/2ap8hgm unconfirmed]
|
|-
| Pull More AMO Data into the Add-ons Manager:
|
|-
| Top Issues:
* {{bug|565930}} - ACR inclusion?
* {{bug|520124}} - Personas on top of custom themes
* {{bug|553455}} - Missing notification box
* {{bug|553493}} - UI for restartless add-ons
* {{bug|562622}} - Update design mockups
* {{bug|562902}} - Detail pane mockups
| Open bugs which blocks testing of a specific area.
|}
|}
    
    
Line 278: Line 325:


== Bug Verification ==
== Bug Verification ==
* Feature bugs that need verification
* [https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&field0-0-0=blocked&bug_status=RESOLVED&type0-0-1=substring&value0-0-1=AddonsRewrite&type0-0-0=anywordssubstr&field0-0-1=status_whiteboard&value0-0-0=461973%2C550048%2C553169%2C558289%2C558289 Fixed bugs] (need verification).
* [https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&field0-0-0=blocked&bug_status=VERIFIED&type0-0-1=substring&value0-0-1=AddonsRewrite&type0-0-0=anywordssubstr&field0-0-1=status_whiteboard&value0-0-0=461973%2C550048%2C553169%2C558289%2C558289 Verified bugs]


[[#Feature_Bug_Management|top]]
[[#Feature_Bug_Management|top]]
Line 290: Line 338:


== Localization ==
== Localization ==
* Details of feature localization test requirements  
* Details of feature localization test requirements  


Line 295: Line 344:


== Accessibility ==   
== Accessibility ==   
* Details of feature accessibility test requirements
* Keyboard accessibility: Test that all actions can be performed through the keyboard, without the use of a mouse. Items can either be interacted with directly (buttons to perform actions, for example), or be accessed through a popup/context menu of any given item. Success is met when all interactive items can be tabbed to and all features are reachable through either an interactive control or the context menu of an item.
* Themeing testing: Is the UI still fully readable if the operating system's theme is changed to high-contrast, inverted colors etc? Success is met when all controls are still readable in interactive and noninteractive states when changing the operating system's color scheme.
* Screen reader exposure: Are all controls and items exposed with all available information to screen readers? Success is met when all controls speak with their name and control type (like button etc.).


[[#Feature_Test_Items|top]]
[[#Feature_Test_Items|top]]
Line 315: Line 366:


= [[#Feature_Tests|Feature Tests Details]] =
= [[#Feature_Tests|Feature Tests Details]] =
To test all of the new features which will come with Firefox 4 we have setup a test plan which can be found on [http://spreadsheets.google.com/ccc?key=txGbPoaoGuoJW2P9cBdWL8g Google Spreadsheets].
== [[#Automated_Tests|Automated Tests Details]] ==
== [[#Automated_Tests|Automated Tests Details]] ==


Line 331: Line 384:


== [[#Manual_Tests|Manual Tests Details]] ==
== [[#Manual_Tests|Manual Tests Details]] ==
Areas which cannot be tested by our automated test harnesses will need a good set of manual tests on Litmus. We cover those tests in a [http://spreadsheets.google.com/ccc?key=0AlroormSQgiRdDZhVEJqcF9ITFNLLW5zMURzMUROQXc&hl=en#gid=0 Google Spreadsheet].


=== Smoke_Tests ===
=== Smoke_Tests ===
Line 349: Line 403:
= [[#Community_Test_Events|Community Test Events Details]] =
= [[#Community_Test_Events|Community Test Events Details]] =
== Testdays ==
== Testdays ==
* Links to test day event results for feature
* [10/04/30] [http://quality.mozilla.org/qmo-news/2010/results-of-add-ons-manager-testday/ Results] of the [http://www.hskupin.info/2010/04/24/testday-exploratory-testing-the-new-add-ons-manager/ first Add-ons Manager testday]
* [10/06/11] [http://www.hskupin.info/2010/06/09/second-add-ons-manager-redesign-testday-this-friday/ Second Add-ons Manager testday]


== Bugdays ==
== Bugdays ==
* Links to bug day event results for feature
None


== Meetups ==
== Meetups ==
* Links to Meetup events for feature
None

Latest revision as of 21:47, 30 June 2010

« Firefox 4 TestPlan

Add-ons Manager Redesign

Area Details
Development Status: In progress (10/06/09)
Feature Testing: In progress (10/06/09)
Team: By group:
  • Jennifer Boriss (UX)
  • Dave Townsend, Blair McBride, Robert Strong, Ben Parr, Benedict Hsieh (Add-ons Manager)
  • Justin Scott, Jeff Balogh (Discovery Pane)
  • Brian King, David McNamara (Add-on Compatibility Reporter)
  • Henrik Skupin, Marco Zehe, Raymond Etornam (QA)
Tracking Bugs By sub-project:


Feature Description

This redesign will substantially improve the basic functionality of the a add-ons manager, such as installation and configuration of add-ons, as well as incorporate additional features and functionality, such as use of Personas and Jetpacks. The design will also remove parts of the manager that interrupt and distract user tasks.

Feature Release Readiness Assessment

The table below provides a top level go/no go assessment of whether the feature is release ready for the given milestone. Each milestone link references a section below that discusses the criteria and evaluation that went into the QA go/no-go decision.

Milestone Assessment
#Beta1 Ready to Ship
#Beta2 N/A
#Beta3 N/A

top

Feature Documentation

Item Description Status
#Project_Wiki Wiki Links to all feature related entries
#Developer_Links (blogs) Developer links to feature related sites
#Other_Docs Web links to feature related sites
#Developer_QA_Review Details from developer and qa discussions regarding feature test strategies and issues.

top

Feature Bug Management

Item Description Status
#Bug_Tracking Top level bugs tracking feature
#Bug_Verification Feature bugs that need verification
#Bug_Triage Links triage bug tasks

top

Feature Test Items

The table below provides a breakdown of all feature items that should be covered and how they will be tested. Not all items will be covered by internal QA team members. It is important to list what should be covered. If it is not covered, list it as not covered.

Note: not all items listed below will apply for a given feature

Test Item Description Covered By Status
Item 1 Item 1 Description Developer Tests
Item 2 Item 2 Description Beta tester exposure
#Localization Feature localization
#Accessibility Feature accessibility
#Plugins Plugins compatibility
#Addons Addons compatibility
#Topsites Top internet sites compatibilities

top


Feature Tests

Automated Tests

Item Description Status
#Developer_Tests Links to automated developer tests
#Mozmill_Tests Links to automated mozmill feature test cases

Manual Tests

Item Description Status
#Smoke_Tests link to smoke tests
#Regression_Tests link to BFT and/or regression tests
#Functional_Tests link to FFT and/or complete functional tests

top

Community Test Events

Item Description Status
#Testdays Links to test day event results for feature
#Bugdays Links to bug day event results for feature
#Meetups Links to Meetup events for feature

top

Feature Documentation Details

Project Wiki

top

Developer Links

top

Other Docs

top

Developer QA Review

The QA person responsible for the feature should hold a formal interview with the lead developer or feature champion. Below are questions that should be asked in the interview:

  • Do we have automated tests for the feature?
  • What do they cover?
  • What do they not cover?
  • How well do they cover the feature?
  • What are the important areas we should focus on?
  • What are the dependencies?
  • What is our comfort level with this feature in its current state?
  • What feedback would you like from QA?

top

Feature Release Readiness Assessment Details

Beta1

Beta2

Beta3

Feature Bug Management Details

Bug Tracking

  • Top level bugs tracking feature. Include any relevant bug queries that are helpful for tracking feature status.
Query Name Description
Add-ons Manager (interface and backend):
Discovery Pane:
Add-ons Compatibility Reporter:
Pull More AMO Data into the Add-ons Manager:
Top Issues: Open bugs which blocks testing of a specific area.

top

Bug Verification

top

Bug Triage

  • Bug triage information

top

Feature Test Items Details

Localization

  • Details of feature localization test requirements

top

Accessibility

  • Keyboard accessibility: Test that all actions can be performed through the keyboard, without the use of a mouse. Items can either be interacted with directly (buttons to perform actions, for example), or be accessed through a popup/context menu of any given item. Success is met when all interactive items can be tabbed to and all features are reachable through either an interactive control or the context menu of an item.
  • Themeing testing: Is the UI still fully readable if the operating system's theme is changed to high-contrast, inverted colors etc? Success is met when all controls are still readable in interactive and noninteractive states when changing the operating system's color scheme.
  • Screen reader exposure: Are all controls and items exposed with all available information to screen readers? Success is met when all controls speak with their name and control type (like button etc.).

top

Plugins

  • Details of plugins compatibility test requirements

top

Addons

  • Details of addons compatibility

top

Topsites

  • Details of top internet sites test requirements

top

Feature Tests Details

To test all of the new features which will come with Firefox 4 we have setup a test plan which can be found on Google Spreadsheets.

Automated Tests Details

Developer Tests

  • Links to automated developer tests

top

Mozmill Tests

If a particular feature needs manual tests which should also be covered by Mozmill tests please add the "[mozmill-test-needed]" whiteboard entry to the feature implementation or regression bug.

List of Mozmill Tests:

  • Links to automated mozmill feature test cases

top

Manual Tests Details

Areas which cannot be tested by our automated test harnesses will need a good set of manual tests on Litmus. We cover those tests in a Google Spreadsheet.

Smoke_Tests

  • links to litmus smoke tests or description

top

Regression_Tests

  • links to litmus BFT and/or regression tests description

top

Functional_Tests

  • links to litmus FFT and/or complete functional tests description

top

Community Test Events Details

Testdays

Bugdays

None

Meetups

None

Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Firefox_3.next/Test_Plan:AddonsManagerRedesign&oldid=235545"