TestEngineering/UI/AMO/AddonCompatibleByDefault: Difference between revisions

Latest revision as of 20:19, 26 August 2016

Compatible by default

The vast majority of add-ons work from one version of Firefox to the next without the need for developer maintenance, but under the current system, compatibility information must be updated in order for Firefox to enable the add-on for use. For add-ons hosted on AMO, this is done automatically. However, 75% of add-ons in use are not hosted on AMO, and are therefore a major compatibility obstacle for our users. All of the compatibility effort put into each release is simply because Firefox still assumes add-ons will be incompatible between versions, when they usually aren't.

We should change Firefox's assumption to be that add-ons are compatible, with a few exceptions. Binary add-ons are never compatible between releases and are also the highest risk of negative side effects. Firefox should automatically enable low-risk (non-binary) add-ons in new versions of Firefox, and check AMO for additional compatibility information.

When users upgrade to a new version of Firefox, only the add-ons that are actually incompatible should be disabled, and the rest are assumed to be compatible. Because Nightly, Aurora, and Beta users will test out the add-ons for weeks before stable users, we should be able to identify and blacklist incompatible add-ons before stable users would be affected by a truly incompatible add-on.

Status

Feature	Status	Lead engineer	QA Lead	Status
Add-ons Default To Compatible	In Progress	Rob Hudson	Krupa Raj	In Progress

Tracking bug

bug 694658

Testing Matrix

browsers: Firefox 3.6-10 platforms: WinXP, Windows 7, Windows Vista, MAC OS X, Linux

Settings

In your about:config settings, hange the following setting in your about:config
extensions.update.url to

Settings for Firefox:

https://addons-dev.allizom.org/update/VersionCheck.php?reqVersion=%REQ_VERSION%&id=%ITEM_ID%&version=%ITEM_VERSION%&maxAppVersion=%ITEM_MAXAPPVERSION%&status=%ITEM_STATUS%&appID=%APP_ID%&appVersion=%APP_VERSION%&appOS=%APP_OS%&appABI=%APP_ABI%&locale=%APP_LOCALE%&currentAppVersion=%CURRENT_APP_VERSION%&updateType=%UPDATE_TYPE%

Settings for Fennec:

https://versioncheck.addons.mozilla.org/update/VersionCheck.php?reqVersion=%REQ_VERSION%&id=%ITEM_ID%&version=%ITEM_VERSION%&maxAppVersion=%ITEM_MAXAPPVERSION%&status=%ITEM_STATUS%&appID=%APP_ID%&appVersion=%APP_VERSION%&appOS=%APP_OS%&appABI=%APP_ABI%&locale=%APP_LOCALE%&currentAppVersion=%CURRENT_APP_VERSION%&updateType=%UPDATE_TYPE%

Usecase Scenarios:

Update Check

DiscoveryPane

Install Buttons

API Search

AMO Search and Listing Pages

References

Link Project plan
Description of compat modes
Test plan for Client-side testing
Feature implementation on Add-on Manager side: bug 693901, bug 693899, bug 693897, bug 695977, bug 700201

@@ Line 1: / Line 1: @@
 === Compatible by default  ===
-*As majority of add-ons work from one version to the next without the need for developer maintenance, Firefox will mark addons compatible with new versions of Firefox by default, except for the following:
+The vast majority of add-ons work from one version of Firefox to the next without the need for developer maintenance, but under the current system, compatibility information must be updated in order for Firefox to enable the add-on for use. For add-ons hosted on AMO, this is done automatically. However, 75% of add-ons in use are not hosted on AMO, and are therefore a major compatibility obstacle for our users. All of the compatibility effort put into each release is simply because Firefox still assumes add-ons will be incompatible between versions, when they usually aren't.
-**All binary add-ons
-**All strict incompatible add-ons
-**All blacklisted add-ons
-*The Feature is planned for Firefox 10. Implementation has started on Firefox 10
+We should change Firefox's assumption to be that add-ons are compatible, with a few exceptions. Binary add-ons are never compatible between releases and are also the highest risk of negative side effects. Firefox should automatically enable low-risk (non-binary) add-ons in new versions of Firefox, and check AMO for additional compatibility information.
-The feature can be enabled by setting the "extensions.strictCompatibility" preference to false in about:config.<br>
+When users upgrade to a new version of Firefox, only the add-ons that are actually incompatible should be disabled, and the rest are assumed to be compatible. Because Nightly, Aurora, and Beta users will test out the add-ons for weeks before stable users, we should be able to identify and blacklist incompatible add-ons before stable users would be affected by a truly incompatible add-on.
 <br>
-=== Settings<br>  ===
-To test the updates on the staging server we need to tweak the settings.
-'''<u>Settings for Firefox:</u>'''
-https://addons.allizom.org/update/VersionCheck.php?reqVersion=%REQ_VERSION%&amp;id=%ITEM_ID%&amp;version=%ITEM_VERSION%&amp;maxAppVersion=%ITEM_MAXAPPVERSION%&amp;status=%ITEM_STATUS%&amp;appID=%APP_ID%&amp;appVersion=%APP_VERSION%&amp;appOS=%APP_OS%&amp;appABI=%APP_ABI%&amp;locale=%APP_LOCALE%&amp;currentAppVersion=%CURRENT_APP_VERSION%&amp;updateType=%UPDATE_TYPE%<br> <br>'''<u>Settings for Fennec:</u>'''
-<br> To test update on the staging server, change the following setting in your about:config <br>extensions.update.url to<br>
-<br>
-https://addons.allizom.org/update/VersionCheck.php?reqVersion=%REQ_VERSION%&amp;id=%ITEM_ID%&amp;version=%ITEM_VERSION%&amp;maxAppVersion=%ITEM_MAXAPPVERSION%&amp;status=%ITEM_STATUS%&amp;appID=%APP_ID%&amp;appVersion=%APP_VERSION%&amp;appOS=%APP_OS%&amp;appABI=%APP_ABI%&amp;locale=%APP_LOCALE%&amp;currentAppVersion=%CURRENT_APP_VERSION%&amp;updateType=%UPDATE_TYPE%
 === Status ===
@@ Line 42: / Line 25: @@
 |}
+=== Tracking bug ===
+[https://bugzilla.mozilla.org/show_bug.cgi?id=694658 bug 694658]
+<br>
-== Usecase Scenarios:  ==
+=== Testing Matrix ===
+browsers: Firefox 3.6-10
-'''testcase 1: Check that add-ons are compatible by default while upgrading: Firefox 3.6 to Firefox 9 '''
+platforms: WinXP, Windows 7, Windows Vista, MAC OS X, Linux
-'''testcase 2: Check that add-ons are compatible by default while upgrading: Firefox 4.0.1 to Firefox 9 '''
+=== Settings<br>  ===
+In your about:config settings, hange the following setting in your about:config <br>extensions.update.url to<br>
-'''testcase 3: Check that add-ons are compatible by default while upgrading: Firefox 5 to Firefox 9 '''
+'''<u>Settings for Firefox:</u>'''
-'''testcase 4: Check that add-ons are compatible by default while upgrading: Firefox 6 to Firefox 9 '''
+https://addons-dev.allizom.org/update/VersionCheck.php?reqVersion=%REQ_VERSION%&amp;id=%ITEM_ID%&amp;version=%ITEM_VERSION%&amp;maxAppVersion=%ITEM_MAXAPPVERSION%&amp;status=%ITEM_STATUS%&amp;appID=%APP_ID%&amp;appVersion=%APP_VERSION%&amp;appOS=%APP_OS%&amp;appABI=%APP_ABI%&amp;locale=%APP_LOCALE%&amp;currentAppVersion=%CURRENT_APP_VERSION%&amp;updateType=%UPDATE_TYPE%<br> <br>'''<u>Settings for Fennec:</u>'''
-'''testcase 5: Check that add-ons are compatible by default while upgrading: Firefox 7 to Firefox 9 '''
+https://versioncheck.addons.mozilla.org/update/VersionCheck.php?reqVersion=%REQ_VERSION%&id=%ITEM_ID%&version=%ITEM_VERSION%&maxAppVersion=%ITEM_MAXAPPVERSION%&status=%ITEM_STATUS%&appID=%APP_ID%&appVersion=%APP_VERSION%&appOS=%APP_OS%&appABI=%APP_ABI%&locale=%APP_LOCALE%&currentAppVersion=%CURRENT_APP_VERSION%&updateType=%UPDATE_TYPE%
-'''testcase 6: Check that add-ons are compatible by default while upgrading: Firefox 8 to Firefox 9 '''
-'''testcase 7: Check that compatible by default feature is not applicable to binary add-ons(extension)  '''
+<br>
-'''testcase 8: Check that compatible by default feature is not applicable to binary add-ons(theme)  '''
+== Usecase Scenarios:  ==
+[https://wiki.mozilla.org/QA/Execution/Web_Testing/AMO/AddonCompatibleByDefault/TestCases/UpdateCheck Update Check]
-'''testcase 9: Check compatible by default feature for a theme while upgrading to Firefox 9 '''
-'''testcase 10: Check compatible by default feature for a dictionary while upgrading to Firefox 9 '''
-'''testcase 11: Check compatible by default feature for a language tool while upgrading to Firefox 9 '''
-'''testcase 12: Check compatible by default feature doesn't affect version check for searchtools while upgrading to Firefox 9 '''
+[https://wiki.mozilla.org/QA/Execution/Web_Testing/AMO/AddonCompatibleByDefault/TestCases/DiscoveryPane DiscoveryPane]
-'''testcase 13: Check compatible by default feature for a multi-package extension while upgrading to Firefox 9 '''
+=== Install Buttons===
-'''testcase 14: Check compatible by default feature for a lite-review add-on while upgrading to Firefox 9 '''
+===API Search===
-'''testcase 15: Check compatible by default feature for an unreviewed add-on while upgrading to Firefox 9 '''
+===AMO Search and Listing Pages===
-== Reference  ==
+== References  ==
 *[https://wiki.mozilla.org/Features/Add-ons/Add-ons_Default_to_Compatible Link Project plan]
+*[https://bugzilla.mozilla.org/show_bug.cgi?id=698355#c0 Description of compat modes]
 *[https://wiki.mozilla.org/index.php?title=Features/Add-ons/Add-ons_Default_to_Compatible/TestPlan Test plan for Client-side testing]
 *Feature implementation on Add-on Manager side: [https://bugzilla.mozilla.org/show_bug.cgi?id=693901 bug 693901], [https://bugzilla.mozilla.org/show_bug.cgi?id=693899 bug 693899], [https://bugzilla.mozilla.org/show_bug.cgi?id=693897 bug 693897], [https://bugzilla.mozilla.org/show_bug.cgi?id=695977 bug 695977], [https://bugzilla.mozilla.org/show_bug.cgi?id=700201 bug 700201]