User:Ashughes/Test Plans/CTP Blocklisting

From MozillaWiki
Jump to: navigation, search

Summary

The following is a test plan for testing the CTP Block for Firefox 17.

Instructions

1. Go to about:config, change the following prefs and restart Firefox

  • find the extensions.blocklist.url pref and change the addons.mozilla.org part of the value to addons-dev.allizom.org
  • find the app.update.interval pref and change the value to 60
  • find the extensions.blocklist.interval pref and change the value to 60
  • find the app.update.lastUpdateTime.blocklist-background-update-timer and reset it

2. Run the following in Error Console:

Components.classes["@mozilla.org/extensions/blocklist;1"].getService(Components.interfaces.nsITimerCallback).notify(null);

3. Wait a minute then try to load some plugin content

4. If the block does not appear to be working, verify your %appdata%/Mozilla/Firefox/Profiles/%profile%/blocklist.xml file contains the correct blocklist URL

Downloads

Results

Staged Blocks

Firefox 16

The expectation with Firefox 16 is that an info-bar block is deployed.

  • Notification bar appears when plugin content is loaded
  • Plugin content continues to display
  • Dismissing the info-bar and reloading the page redisplays the info-bar
  • Clicking update from the info-bar opens the PFS page enabling the user to update their plugin
  • The info-bar no longer appears after updating the plugin

Adobe Flash

Plugin Version Windows XP Windows 7
Adobe Flash 11.4.402.278 (current release, should not be blocked) PASS - 2012-10-04 PASS - 2012-10-04
Adobe Flash 11.3.300.273 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Flash 11.2.202.235 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Flash 11.1.102.63 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Flash 11.0.1.152 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Flash 10.3.183.18 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Flash 10.3.181.34 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Flash 10.2.152.26 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Flash 10.1.85.3 PASS - 2012-10-04 PASS - 2012-10-04

Adobe Reader

Plugin Version Windows XP Windows 7
Adobe Reader 10.1.4.38 (current release, should not be blocked) PASS - 2012-10-04 PASS - 2012-10-04
Adobe Reader 10.1.3.23 FAIL - 2012-10-04 FAIL - 2012-10-04
Adobe Reader 10.1.0.534 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Reader 10.0.0.396 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Reader 9.4.0.195 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Reader 9.1.0.163 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Reader 9.0.0.332 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Reader 8.1.0.137 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Reader 8.0.0.456 PASS - 2012-10-04 PASS - 2012-10-04

Microsoft Silverlight

Plugin Version Windows XP Windows 7
Silverlight 5.1.10411.0 (current release, should not be blocked) PASS - 2012-10-04 PASS - 2012-10-04
Silverlight 5.0.61118.0 PASS - 2012-10-04 PASS - 2012-10-04
Silverlight 4.0.60531.0 PASS - 2012-10-04 PASS - 2012-10-04
Silverlight 3.0.40624.0 PASS - 2012-10-04 PASS - 2012-10-04

Firefox 17

The expectation with Firefox 17 is that a Click-to-Play block is deployed.

  • An overlay appears blocking the content (see below)
  • Clicking the overlay displays the plugin content
  • Reloading the page redisplays the blocking overlay
  • Clicking update from the overlay opens the PFS page enabling the user to update their plugin
  • The overlay no longer appears after updating the plugin

Overlay:

This plugin is vulnerable and should be updated. 
__Check for updates__...
Click here to activate the <plugin_name> plugin.

Adobe Flash

Plugin Version Windows XP Windows 7
Adobe Flash 11.4.402.278 (current release, should not be blocked) PASS - 2012-10-04 PASS - 2012-10-04
Adobe Flash 11.3.300.271 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Flash 11.2.202.235 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Flash 11.1.102.63 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Flash 11.0.1.152 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Flash 10.3.183.18 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Flash 10.3.181.34 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Flash 10.2.152.26 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Flash 10.1.85.3 PASS - 2012-10-04 PASS - 2012-10-04

Adobe Reader

Plugin Version Windows XP Windows 7
Adobe Reader 10.1.4.38 (current release, should not be blocked) PASS - 2012-10-04 PASS - 2012-10-04
Adobe Reader 10.1.3.23 FAIL - 2012-10-04 FAIL - 2012-10-04
Adobe Reader 10.1.0.534 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Reader 10.0.0.396 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Reader 9.4.0.195 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Reader 9.1.0.163 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Reader 9.0.0.332 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Reader 8.1.0.137 PASS - 2012-10-04 PASS - 2012-10-04
Adobe Reader 8.0.0.456 PASS - 2012-10-04 PASS - 2012-10-04

Microsoft Silverlight

Plugin Version Windows XP Windows 7
Silverlight 5.1.10411.0 (current release, should not be blocked) PASS - 2012-10-04 PASS - 2012-10-04
Silverlight 5.0.61118.0 PASS - 2012-10-04 PASS - 2012-10-04
Silverlight 4.0.60531.0 PASS - 2012-10-04 PASS - 2012-10-04
Silverlight 3.0.40624.0 PASS - 2012-10-04 PASS - 2012-10-04

Live Push

Once the staged blocks are pushed live, we will test a subset to ensure they are working correctly.

Adobe Flash 10.2 and lower

Plugin Version Firefox 16 (info-bar) Firefox 17 (click-to-play)
Adobe Flash 11.4.402.278 (current release, should not be blocked) PASS - 2012-10-05 PASS - 2012-10-05
Adobe Flash 10.3.183.18 (should not be blocked) PASS - 2012-10-05 PASS - 2012-10-05
Adobe Flash 10.2.159.1 PASS - 2012-10-05 PASS - 2012-10-05
Adobe Flash 10.1.85.3 PASS - 2012-10-05 PASS - 2012-10-05
Issues
  • FIXED Info-bar block shown on Firefox 17 instead of CTP block

Adobe Reader 9.5.1 and lower

Plugin Version Firefox 16 (info-bar) Firefox 17 (click-to-play)
Adobe Reader 9.4.0 PASS - 2012-10-05 PASS - 2012-10-05
Adobe Reader 9.1.0 PASS - 2012-10-05 PASS - 2012-10-05
Adobe Reader 8.0 PASS - 2012-10-05 PASS - 2012-10-05

Adobe Reader 10.0 to 10.1.3

Plugin Version Firefox 16 (info-bar) Firefox 17 (click-to-play)
Adobe Reader 10.1.4.38 (should not be blocked) PASS - 2012-10-05 PASS - 2012-10-05
Adobe Reader 10.1.3.23 (expected not to be blocked) PASS - 2012-10-05 PASS - 2012-10-05
Adobe Reader 10.0 PASS - 2012-10-05 PASS - 2012-10-05

Silverlight 4.1.10329.0 and lower

Plugin Version Firefox 16 (info-bar) Firefox 17 (click-to-play)
Microsoft Silverlight 4.0.60129.0 PASS - 2012-10-05 PASS - 2012-10-05
Microsoft Silverlight 3.0 PASS - 2012-10-05 PASS - 2012-10-05
Microsoft Silverlight 2.0 PASS - 2012-10-05 PASS - 2012-10-05

Silverlight 5.0 to 5.1.10410.0

Plugin Version Firefox 16 (info-bar) Firefox 17 (click-to-play)
Microsoft Silverlight 5.1.10411.0 (should not be blocked) PASS - 2012-10-05 PASS - 2012-10-05
Microsoft Silverlight 5.0.61118.0 PASS - 2012-10-05 PASS - 2012-10-05

Round #2: Flash and Java

bug 803152 Please CTP block all versions of Java, and Flash versions 10.3 through (and including) 11.3.*

Expectations:

  • Firefox 16: plugin content should load normally
  • Firefox 17: click-to-play block overlay appears over loaded plugin content

Staging

Flash

Plugin Version Windows 7 Mac OSX 10.7
Adobe Flash Player 11.3.300.271 PASS - 2012-10-29 PASS - 2012-10-29
Adobe Flash Player 11.2.202.235 PASS - 2012-10-29 PASS - 2012-10-29
Adobe Flash Player 11.1.102.63 PASS - 2012-10-29 PASS - 2012-10-29
Adobe Flash Player 10.3.183.18 PASS - 2012-10-29 PASS - 2012-10-29
Adobe Flash Player 10.3.181.34 PASS - 2012-10-29 PASS - 2012-10-29
Adobe Flash Player 10.2.159.1 PASS - 2012-10-29 PASS - 2012-10-29
Adobe Flash Player 10.1.85.3 PASS - 2012-10-29 PASS - 2012-10-29
Not Blocked - The following plugins should not be blocked
Adobe Flash Player 11.4.402.287 PASS - 2012-10-29 PASS - 2012-10-29
Adobe Flash Player 10.3.183.19 PASS - 2012-10-29 PASS - 2012-10-29

Java

Plugin Version Windows 7 Ubuntu 12.04
Java 7u8 PASS - 2012-10-29 result - YYYY-MM-DD
Java 7u7 PASS - 2012-10-29 PASS - 2012-10-30
Java 6u34 PASS - 2012-10-29 PASS - 2012-10-29
Java 6 PASS - 2012-10-29 PASS - 2012-10-29
Not Blocked - The following plugins should not be blocked
Java 7u9 PASS - 2012-10-29 PASS - 2012-10-30
Java 6u37 PASS - 2012-10-29 PASS - 2012-10-30

Production

Flash

Plugin Version Windows 7 Mac OSX 10.7
Adobe Flash Player 11.3.300.271 PASS - 2012-10-30 PASS - 2012-10-30
Adobe Flash Player 10.3.183.18 PASS - 2012-10-30 PASS - 2012-10-30
Not Blocked - The following plugins should not be blocked
Adobe Flash Player 11.4.402.287 PASS - 2012-10-30 PASS - 2012-10-30
Adobe Flash Player 10.3.183.29 PASS - 2012-10-30 PASS - 2012-10-30

Java

Plugin Version Windows 7 Ubuntu 12.04
Java 7u7 PASS - 2012-10-30 PASS - 2012-10-30
Java 6u34 PASS - 2012-10-30 PASS - 2012-10-30
Not Blocked - The following plugins should not be blocked
Java 7u9 PASS - 2012-10-30 PASS - 2012-10-30
Java 6u37 PASS - 2012-10-30 PASS - 2012-10-30

Final CTP Fx17 Changes

bug 812936 Final CTP Blocklist Changes for FF17 Launch

  • Production
    • Flash 10: CTP blocked on Firefox <=16 for Flash 10.2 and earlier
    • Java 6: CTP blocked on Firefox >=18 for Java 6u36 and earlier
    • Java 7: CTP blocked on Firefox >=18 for Java 7u7 and 7u8
  • Staging
    • Reader 10: CTP blocked on Firefox <=16 for Reader 10.1.3 and earlier
    • Reader 9: CTP blocked on Firefox <=16 for Reader 9.5.1 and earlier
    • Silverlight 5: CTP blocked on Firefox <=16 for Silverlight 5.1.10410.0 and earlier
    • Silverlight 4: CTP blocked on Firefox <=16 for Silverlight 4.1.10328.0 and earlier

Results

Java 6 (production)
  • [PASS] Java 6u33, Windows 7, Firefox 18.0a2
  • [N/A] Java 6u35, Mac OSX, Firefox 19.0a1
    • Java 6u35 Mac download could not be found
  • [PASS] Java 6u34, Ubuntu 12.04, Firefox 18.0a2
Java 7 (production)
  • [PASS] Java 7u7, Windows 7, Firefox 19.0a1
  • [PASS] Java 7u7, Mac OSX 10.7, Firefox 18.0a2
  • [PASS] Java 7u7, Ubuntu 12.04, Firefox 19.0a1
Flash 10.2 (staging)
  • [FAIL] Flash 10.2.version, Windows 7, Firefox 16.0.2
    • Out-of-date plugin notification, but content loads anyway with no CTP UI
  • [PASS] Flash 10.2.version, Win7, Firefox 16.0.2 - I get a notification bar indicating my Flash version is out of date and an option to "Update Plugins."
Reader 10 (staging)
  • [FAIL] Reader 10.1.3, Windows 7, Firefox 16.0.2
    • Content fails to load, and no CTP UI is displayed
Reader 9 (staging)
  • [FAIL] Reader 9.5, Windows 7, Firefox 16.0.1
    • Content fails to load, and no CTP UI is displayed
Silverlight (staging)
  • [PASS] Silverlight 5.1.10410.0, Windows 7, Firefox 16.0.2
    • Notification bar comes up. You can play content or you can upgrade.
  • [PASS] Silverlight 4.1.10328.0, Windows 7, Firefox 16.0.1

Issues Found