MozillaWiki

Features/Add-ons/Add-ons Default to Compatible/TestPlan: Difference between revisions

Page Discussion

Features/Add-ons/Add-ons Default to Compatible/TestPlan (view source)

Revision as of 16:01, 31 January 2012

8,515 bytes added ,  31 January 2012
no edit summary
No edit summary
 
(62 intermediate revisions by the same user not shown)
Line 4: Line 4:
|-
|-
| Feature  
| Feature  
| Lead engineer
| Status  
| Status  
| Direct Responsible
| QA Lead  
| QA Lead  
| Status
| Status
|-
|-
| Add-ons Default To Compatible  
| Add-ons Default To Compatible  
| {{StatusHealthy|status=In Progress}}  
| Blair McBride
| Lawrence Mandel
| {{StatusHealthy|status=Complete}}  
| Virgil Dicu  
| Virgil Dicu  
|  
| {{StatusHealthy|status=Signed Off}}
|}
|}


== Summary  ==
== Summary  ==


*Firefox will mark addons compatible with new versions of Firefox by default, except for the following:
*As majority of add-ons work from one version to the next without the need for developer maintenance, Firefox will mark addons compatible with new versions of Firefox by default, except for the following:
**All binary add-ons
**All binary add-ons
**All strict incompatible add-ons
**All strict incompatible add-ons
Line 24: Line 24:


*The Feature is planned for Firefox 10.
*The Feature is planned for Firefox 10.
The feature is enabled by default in Firefox 10. The feature can be disabled by setting the "extensions.strictCompatibility" preference to true in about:config.


== Reference  ==
== Reference  ==
Line 29: Line 31:
*Feature wiki page - [https://wiki.mozilla.org/Features/Add-ons/Add-ons_Default_to_Compatible Link]
*Feature wiki page - [https://wiki.mozilla.org/Features/Add-ons/Add-ons_Default_to_Compatible Link]


*There are no RESOLVED bugs on Add-on Manager side. All bugs are still New or Assigned.
*Bug Tracking:
**Add-on Manager: [https://bugzilla.mozilla.org/show_bug.cgi?id=692664 Bug 692664]
**AMO: [https://bugzilla.mozilla.org/show_bug.cgi?id=691834 Bug 691834]
 
== Non-Goals  ==
 
*Will not be tested:
**Add-ons individual functionality after being enabled to default.
 
== Fixed bugs verification  ==
 
*Verified: [https://bugzilla.mozilla.org/show_bug.cgi?id=693897 bug 693897], [https://bugzilla.mozilla.org/show_bug.cgi?id=693899 bug 693899], [https://bugzilla.mozilla.org/show_bug.cgi?id=702868 bug 702868], [https://bugzilla.mozilla.org/show_bug.cgi?id=700201 bug 700201], [https://bugzilla.mozilla.org/show_bug.cgi?id=693901 bug 693901], [https://bugzilla.mozilla.org/show_bug.cgi?id=698653 bug 698653], [https://bugzilla.mozilla.org/show_bug.cgi?id=695977 bug 695977], [https://bugzilla.mozilla.org/show_bug.cgi?id=527141 bug 527141]
 
== Status per weeks ==
=== Status-week 11/7/2011-11/14/2011 ===
 
Implementation work has started on Add-on manager side starting with the following bugs:
*[https://bugzilla.mozilla.org/show_bug.cgi?id=693897 Allow add-ons to opt-in to strict compatibility checking]
*[https://bugzilla.mozilla.org/show_bug.cgi?id=693901 Add preference to globally (re)enable strict compatibility checks]
*[https://bugzilla.mozilla.org/show_bug.cgi?id=693899 Support detecting binary components, and enable strict compatibility checking when found]
 
TO DO list:
 
*create 75% basic functionality test cases
*run feature basic functionality test cases on all platforms (Windows XP, Windows 7, Mac OS 10.6, Ubuntu 11.10)
*run feature basic functionality test cases on Firefox 7, 8 and 9, while upgrading from each one to Nightly.
*create test cases for edge areas and different add-on states.
 
=== Status-week 11/14/2011-11/21/2011 ===
 
Latest feature implementation:
*[https://bugzilla.mozilla.org/show_bug.cgi?id=695977 bug 695977]
*[https://bugzilla.mozilla.org/show_bug.cgi?id=700201 bug 700201]
 
DONE list:
 
*created 75% basic functionality test cases
*ran feature basic functionality test cases on all platforms (Windows XP, Windows 7, Mac OS 10.6, Ubuntu 11.10)
*ran feature basic functionality test cases on Firefox 7, 8 and 9, while upgrading from each one to Nightly.
*created test cases for edge areas and different add-on states.
 
TO DO list:
*create and run test cases for recently implemented bugs
*provide help for 18 November test day exploratory testing around Add-ons Default to compatible:[https://quality.mozilla.org/2011/11/39099/ Testday]
*add feature test cases in Litmus
 
=== Status-week 11/21/2011-11/28/2011 ===
 
DONE list:
*started running a weekly exploratory testing on Windows XP
*started adding feature test cases in Litmus
*create and ran test cases for recently implemented bugs
*started verifying fixed bugs-two bugs remaining to be verified: [https://bugzilla.mozilla.org/show_bug.cgi?id=695977 bug 695977] and [https://bugzilla.mozilla.org/show_bug.cgi?id=698653 bug 698653]
 
TO DO list for next week:
*run the weekly exploratory on Windows XP
*finish off adding test cases in litmus
*upgrade with heavy add-on profile (more than 30)
*upgrade from lower versions than Firefox 7.0.1
 
=== Status-week 11/28/2011-12/5/2011 ===
 
DONE list:
*ran the weekly exploratory on Windows XP
*ran basic test cases on all platforms: Windows XP, Windows 7 (x86, x86_64), Ubuntu 11.10 (x86, x86_64), Mac OS 10.6, 10.7, Windows Vista, Windows 2000
*ran other test scenarios (upgrading from lower versions of Firefox, heavy add-on profiles)
 
TO DO list for next week:
*verify recently fixed bugs
*run Litmus test cases on 4 platforms: Ubuntu 11.10, Mac OS 10.6, Windows XP, Windows 7 x86
 
=== Status-week 12/5/2011-12/12/2011 ===
 
DONE:
*modified some test cases in Litmus-added indication to report Firefox issues due to add-ons marked as incompatible in Bugzilla (Detection and Mitigation Document)
*ran basic test cases on Windows XP and Ubuntu 11.10
 
TO DO:
*verify remaining fixed bugs
*run test cases on Mac OS 10.6 and Windows 7
*insist on dark matter add-ons through testing on Windows 7
 
=== Status-week 12/12/2011-12/19/2011 ===
 
DONE:
*verified [https://bugzilla.mozilla.org/show_bug.cgi?id=527141 bug 527141]
*ran exploratory testing across 4 platforms (Windows XP, 7, Mac OS 10.6, Ubuntu 11.10)[https://docs.google.com/spreadsheet/ccc?key=0Amty6gb0hLqVdDR6NlMxYWJoSzZSbEx3VTNnWk02Vmc&hl=ro&pli=1#gid=1 Document]
 
TO DO:
*to verify bugs 693906 and 705530
*run a weekly exploratory on Windows XP
 
=== Status-week 12/19/2011-12/26/2011 ===
 
DONE:
*ran exploratory testing across 3 platforms (Windows XP, Mac OS 10.6, Ubuntu 11.10)[https://docs.google.com/spreadsheet/ccc?key=0Amty6gb0hLqVdDR6NlMxYWJoSzZSbEx3VTNnWk02Vmc&hl=ro&pli=1#gid=2 Document]
 
TO DO:
*run a weekly exploratory on Windows XP
 
=== Status-week 01/04/2012-01/11/2012 ===
 
DONE:
*ran exploratory testing across 2 platforms (Windows XP, Ubuntu 11.10) [https://docs.google.com/spreadsheet/ccc?key=0Amty6gb0hLqVdDR6NlMxYWJoSzZSbEx3VTNnWk02Vmc&hl=ro&pli=1#gid=4 Document]
*test day preparation: created test scenarios based on Marcia's information-[https://wiki.mozilla.org/Features/Add-ons/Add-ons_Default_to_Compatible/TestdayScenarios Link]
*run the Litmus test cases (BFTs) for Firefox 10 Beta 4
*created test plan for community involvement:http://mzl.la/xrsNr6


== Use Cases to Test  ==
TO DO:
*run the weekly exploratory on at least 3 platforms
*run the Litmus test cases(BFTs) for Firefox 10 Beta 4
*Friday testday involvement: https://etherpad.mozilla.org/testday-20120113


*Install several add-ons (binary component, blacklisted) on Firefox 9
=== Status-week 01/11/2012-01/18/2012 ===
*Upgrade To Firefox 10.
**Blacklisted and Binary components Add-ons should be set to incompatible in Firefox 10.


*Install several add-ons (binary component, blacklisted) on Firefox 4
DONE:
*Manually Upgrade successively with 1 unit,from Firefox 4 up to Firefox 10 (going through F5,6,7,8,9)
*Friday add-ons compatibile by default test day involvement: https://etherpad.mozilla.org/testday-20120113
**Blacklisted and Binary components Add-ons should be set to incompatible in Firefox 10.
**verified add-ons provided by Jorge from the list: http://bit.ly/vZvg1b
**this has been tested mostly on Linux during the testday with community help. Going to verify Windows platform more thoroughly in the next days (with Beta 5 too)
*Fixed bugs: logged bug https://bugzilla.mozilla.org/show_bug.cgi?id=718630
*Ran the weekly exploratory on 3 platforms (Windows XP, Windows 7, Linux)
*Kept track of https://bugzilla.mozilla.org/show_bug.cgi?id=711275
**The update window now lists all add-ons that would stop working with Strict Compatibility enabled for the Firefox version to which the update is made (10 and 11 as well given the workaround fix for this)


== Test Cases  ==
TO DO:
*Finish off testing as many add-ons from the list http://bit.ly/vZvg1b on Windows and Mac OS 10.6 on Beta5. *Started this together with colleagues in Romania in the following etherpad: https://mozqa.etherpad.mozilla.org/ADT
*Run the weekly tests for this feature on Beta5
*Run an all platforms check before the Beta Release candidate next week.


=== Status-week 01/18/2012-01/25/2012 ===


== Important bugs  ==
DONE:


*Tracking bugs:  
*executed tests on popular add-ons from the [https://docs.google.com/spreadsheet/ccc?key=0AmkRt0ylPb8zdFBXUkRXVmJIdzgzYTVQWTM2QUtDLUE&hl=en_US#gid=8 Spreadsheet]
**Add-on Manager: [https://bugzilla.mozilla.org/show_bug.cgi?id=692664 Bug 692664]
*all platforms have been checked using the add-ons listed there.
**AMO: [https://bugzilla.mozilla.org/show_bug.cgi?id=691834 Bug 691834]
*verified bugs: {{bug|693906}} and {{bug|692664}} ({{bug|705530}} remaining-no examples yet)
 
== Use Cases to Test ==
 
*An add-on can have different states in a Firefox version: compatible-incompatible, enabled/disabled, pending restart, updated/in need of update.
*User installs several types of add-ons on a Firefox version with one or more of the above mentioned states
*When upgrading to a new version, Firefox will automatically set his add-ons to compatible with the new Firefox version, unless the installed add-ons are binary, blacklisted or strictly set to incompatible with the Firefox version.
 
== Edge cases ==
 
*Upgrade and verify that installed third party and restartless add-ons are default to compatible or incompatible if binary/blacklisted/strictCompatibility listed
*Upgrade and check behavior for add-ons installed on older Firefox versions (<Firefox 7)
*Upgrade from Firefox4 successively to Firefox 5,6,7,8,9 and 10 and verify that all add-ons are listed as expected.
 
== Test Cases ==
 
Test cases based on this feature are listed in the following spreadsheet: [https://docs.google.com/spreadsheet/ccc?key=0Amty6gb0hLqVdDR6NlMxYWJoSzZSbEx3VTNnWk02Vmc&hl=ro#gid=0 Link]
 
== Signed-off for Aurora ==
Signed off for Aurora:
*No major or critical bugs found
*Feature's basic functionality test cases work as expected
*Add-on are enabled and working after update
 
== Signed-off for Beta ==
Signed off for Beta:
*The feature has landed and is now set as resolved fix in tracking bug: [https://bugzilla.mozilla.org/show_bug.cgi?id=692664 Bug 692664]
*Feature's basic functionality test cases work as expected
*Feature has been tested on 4 platforms: Ubuntu 11.10, Mac OS 10.6, Windows XP and Windows 7.
 
 
== Sign-off Criteria for Release ==
 
*No unresolved major or critical bugs
*Feature's basic functionality test cases work as expected
*Ensure users can upgrade their add-ons successively from one Firefox version to another without any unnecessary change of states (e. g. from disabled to enabled)
*No other issues are triggered by this feature implementation
VirgilD
938

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/360336...392202"