Features/Desktop/Add-on hotfix/Test Plan: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
Line 29: Line 29:
*We want to speak directly to our users with a one-time marketing message / news flash. We ship an update to the add-on that pops up an infobar with the message and then never shows again
*We want to speak directly to our users with a one-time marketing message / news flash. We ship an update to the add-on that pops up an infobar with the message and then never shows again
*We want to A/B test a UI pref on beta. We ship the add-on to x% of our population or ship it to all Beta users and have the add-on enable functionality x% of the time
*We want to A/B test a UI pref on beta. We ship the add-on to x% of our population or ship it to all Beta users and have the add-on enable functionality x% of the time
== Test Case 1 ==
=== Test Case 1 - installing the add-on ===
====Steps to reproduce====
====Steps to reproduce====
* Start Firefox 10 beta 4 with a new profile
* Start Firefox with a new profile
* In about:config set the following prefs:
* In about:config set the following prefs:
** set '''extensions.hotfix.id''' to '''test-hotfix@mozilla.org'''
** set '''extensions.hotfix.id''' to '''test-hotfix@mozilla.org'''
Line 47: Line 47:
*{{done|}} Windows XP - '''PASS''' - 2012-01-13
*{{done|}} Windows XP - '''PASS''' - 2012-01-13


== Test Case 2 ==
=== Test Case 2 ===
*This test checks that none of the four test add-ons are installed.
*This test checks that none of the four test add-ons are installed.
*You will have to run the steps from below for four times, changing the extension's ID accordingly:
*You will have to run the steps from below for four times, changing the extension's ID accordingly:
Line 56: Line 56:
*Do not install manually.
*Do not install manually.
==== Steps to reproduce====
==== Steps to reproduce====
* Start Firefox 10 beta 4 with a new profile
* Start Firefox with a new profile
* In about:config set the following prefs:
* In about:config set the following prefs:
** set '''extensions.hotfix.id''' to '''test-hotfix-"extension's ID"@mozilla.org'''
** set '''extensions.hotfix.id''' to '''test-hotfix-"extension's ID"@mozilla.org'''
Line 71: Line 71:
*{{done|}} Ubuntu 11.10 x64 - '''PASS''' - 2012-01-16  
*{{done|}} Ubuntu 11.10 x64 - '''PASS''' - 2012-01-16  
*{{done|}} Windows XP - '''PASS''' - 2012-01-16
*{{done|}} Windows XP - '''PASS''' - 2012-01-16
=== Test Case 3 - removing and reinstalling the add-on  ===
==== Steps to reproduce 1 ====
* Start Firefox with a new profile
* In about:config set the following prefs:
** set '''extensions.hotfix.id''' to '''test-hotfix@mozilla.org'''
** set '''extensions.update.interval''' to 10
* Restart Firefox
* After Firefox Hotfix Test 1 is installed, remove it from the Add-ons Manager. Do not restart the browser yet.
====Expected Results====
*The Add-on should get removed with the "undo" function active.
**Pressing the "undo" should keep the add-on installed.
*After Firefox restarts, the add-on should be deleted.
==== Steps to reproduce 2 ====
* Start Firefox with a new profile
* In about:config set the following prefs:
** set '''extensions.hotfix.id''' to '''test-hotfix@mozilla.org'''
** set '''extensions.update.interval''' to 10
* Restart Firefox
* After Firefox Hotfix Test 1 is installed, remove it from the Add-ons Manager and restart Firefox.
* In about:config, reset the pref '''extensions.hotfix.lastVersion'''
* Restart Firefox
====Expected Results====
* After restart, navigate randomly over 10 seconds
* Open Add-ons Manager and '''Firefox Hotfix Test 1''' should be installed

Revision as of 16:54, 16 January 2012

Add-on hotfix

Release Target Status Lead Engineer QA Lead Status Health
Firefox 10 Complete Christian Legnitto VladG Landed OK

Summary

Firefox will have a hardcoded add-on id. When checking for add-on updates (every 24 hours), it will always check for a specific add-on id and install it if an update is found.

Use Cases to Test

  • We ship a new feature (like http:// hiding) and the user response is extremely negative. We issue an update to the add-on that flips a pref back to old behavior
  • We need to distrust a CA. We ship an update to the add-on that makes Firefox distrust it and users are silently updated within 24 hours
  • We have a problem with add-ons that require a rebuild of add-on metadata. We issue an empty add-on
  • We have an issue that only affects Italian builds. Rather than issue a new Firefox update to everyone, we ship an update to the add-on which only acts when the locale is set to Italian
  • We want to blocklist a widely used plugin. We turn on crude click-to-play and whitelist it for known good websites
  • We know that Netflix/Hulu/etc does bad user sniffing for a particular Firefox version. We issue an update that drops down a infobar telling users what is happening and to contact the site, or we ship the add-on that changes the user agent on the site while they work on a fix
  • We want to speak directly to our users with a one-time marketing message / news flash. We ship an update to the add-on that pops up an infobar with the message and then never shows again
  • We want to A/B test a UI pref on beta. We ship the add-on to x% of our population or ship it to all Beta users and have the add-on enable functionality x% of the time

Test Case 1 - installing the add-on

Steps to reproduce

  • Start Firefox with a new profile
  • In about:config set the following prefs:
    • set extensions.hotfix.id to test-hotfix@mozilla.org
    • set extensions.update.interval to 10
  • Restart Firefox

Expected Results

  • After restart, navigate randomly over 10 seconds
  • Open Add-ons Manager and Firefox Hotfix Test 1 should be installed
Firefox 10 Beta 4
  • [DONE] MacOS X 10.6.7 - PASS - 2012-01-13
  • [DONE] Windows 7 x86 - PASS - 2012-01-13
  • [DONE] Ubuntu 11.10 x64 - PASS - 2012-01-13
  • [DONE] Windows XP - PASS - 2012-01-13

Test Case 2

Steps to reproduce

  • Start Firefox with a new profile
  • In about:config set the following prefs:
    • set extensions.hotfix.id to test-hotfix-"extension's ID"@mozilla.org
    • set extensions.update.interval to 10
  • Restart Firefox

Expected Results

  • After restart, navigate randomly over 10 seconds
  • None of the add-on should be installed
Firefox 10 Beta 4
  • [DONE] MacOS X 10.6.7 - PASS - 2012-01-16
  • [DONE] Windows 7 x86 - PASS - 2012-01-16
  • [DONE] Ubuntu 11.10 x64 - PASS - 2012-01-16
  • [DONE] Windows XP - PASS - 2012-01-16

Test Case 3 - removing and reinstalling the add-on

Steps to reproduce 1

  • Start Firefox with a new profile
  • In about:config set the following prefs:
    • set extensions.hotfix.id to test-hotfix@mozilla.org
    • set extensions.update.interval to 10
  • Restart Firefox
  • After Firefox Hotfix Test 1 is installed, remove it from the Add-ons Manager. Do not restart the browser yet.

Expected Results

  • The Add-on should get removed with the "undo" function active.
    • Pressing the "undo" should keep the add-on installed.
  • After Firefox restarts, the add-on should be deleted.

Steps to reproduce 2

  • Start Firefox with a new profile
  • In about:config set the following prefs:
    • set extensions.hotfix.id to test-hotfix@mozilla.org
    • set extensions.update.interval to 10
  • Restart Firefox
  • After Firefox Hotfix Test 1 is installed, remove it from the Add-ons Manager and restart Firefox.
  • In about:config, reset the pref extensions.hotfix.lastVersion
  • Restart Firefox

Expected Results

  • After restart, navigate randomly over 10 seconds
  • Open Add-ons Manager and Firefox Hotfix Test 1 should be installed
Retrieved from "https://wiki.mozilla.org/index.php?title=Features/Desktop/Add-on_hotfix/Test_Plan&oldid=387325"