QA/Desupport Hotfix Testing: Difference between revisions
< QA
Jump to navigation
Jump to search
| Line 34: | Line 34: | ||
| class="header" style="width:20%" | Tester | | class="header" style="width:20%" | Tester | ||
|- | |- | ||
| Windows XP | | Windows XP SP3 | ||
| Hotfix downloaded | | Hotfix downloaded | ||
| | | | ||
| | | Tracy | ||
|- | |- | ||
| Windows 7 | | Windows 7 | ||
Revision as of 18:28, 19 April 2012
De-support Hotfix Testing
Known issues: https://etherpad.mozilla.org/arC2CZqj6y
Timeline
Targeting QA testing and sign-off on April 16, 2012
Focus Areas
- The hotfix is only downloaded on Windows
- The hotfix is installed automatically on Windows 2000 and Windows XP <SP2, and is uninstalled automatically when we're not on those versions of Windows
- the proper localization comes up in the billboard, depending on the version of Firefox
- when we set the clock forward (10 days), the user is re-prompted again at startup
PLACEHOLDER for instructions to test a hotfix
Installing Hotfix
- Start Firefox with a new profile
- In about:config set the following prefs:
- make sure extensions.hotfix.id is set to firefox-hotfix@mozilla.org (should be by default)
- set extensions.update.interval to 10
- In extensions.update.url replace versioncheck.addons.mozilla.org from the string with addons-dev.allizom.org (only do this when testing the staged add-on)
- Restart Firefox
- Open the add-ons manager
- Wait for Addon Hotfix to appear
Staging Server Tests
- WHAT: Testing with latest Firefox Beta on the staging server
- WHEN: April 16, 2012
Test #1: Hotfix only downloads on Windows
For this test we need to make sure that the hotfix is only downloaded on Windows platforms.
| Platform | Expected | Result | Tester |
| Windows XP SP3 | Hotfix downloaded | Tracy | |
| Windows 7 | Hotfix downloaded | Tracy | |
| Mac OSX 10.6 | Hotfix NOT downloaded | Tracy | |
| Ubuntu 11.10 | Hotfix NOT downloaded |
Test #2: Hotfix uninstalled if Windows XP SP2 or newer
For this test we need to make sure that the hotfix automatically removed for users running Windows XP SP2 or later.
| Platform | Expected | Result | Tester |
| Windows 2000 | Hotfix NOT removed | Tracy | |
| Windows XP | Hotfix NOT removed | ||
| Windows XP SP1 | Hotfix NOT removed | ||
| Windows XP SP2 | Hotfix removed | ||
| Windows XP SP3 | Hotfix removed | Tracy | |
| Windows Vista | Hotfix removed | ||
| Windows 7 | Hotfix removed | Tracy |
Test #3: Billboard Prompting
For this test we need to make sure the billboard only appears on Windows <= XP SP1
| Platform | Expected | Result | Tester |
| Windows 2000 | Billboard appears | Tracy | |
| Windows XP | Billboard appears | ||
| Windows XP SP1 | Billboard appears | ||
| Windows XP SP2 | Billboard DOES NOT appear | ||
| Windows XP SP3 | Billboard DOES NOT appear | Tracy | |
| Windows Vista | Billboard DOES NOT appear | ||
| Windows 7 | Billboard DOES NOT appear | Tracy | |
| Mac OSX 10.6 | Billboard DOES NOT appear | Tracy | |
| Mac OSX 10.7 | Billboard DOES NOT appear | ||
| Ubuntu 11.10 | Billboard DOES NOT appear |
Test #4: Billboard Localization
For this test we need to make sure that the billboard is properly localized and is appropriate for the Windows version.
- Windows 2000 (tester - Tracy)
Expected first bullet in the en-US billboard: "Upgrading your operating system". "Learn More" links to Windows 2000 page on SUMO.
- en-US: result
- de: result
- pt-BR: result
- ru: result
- zh-TW: result
- ja-JP: result
- Windows XP (tester - Tracy)
Expected first bullet in the en-US billboard: "Installing a free upgrade to your operating system". "Learn More" links to Windows XP page on SUMO.
- en-US: result
- es-ES: result
- fr: result
- it: result
- ko: result
Test #5: Billboard Re-prompting
For this test we need to make sure the billboard reprompts when the clock is moved forward.
- Install Firefox 12 Beta
- Get the hotfix to install
- Wait for the billboard to prompt and dismiss it
- Set your system clock forward 10 days
- Restart Firefox and verify the billboard prompts again
- Windows 2000 (tester - Tracy)
| Platform | Expected | Result |
| Windows 2000 en-US | Billboard reprompts | |
| Windows 2000 de | Billboard reprompts localized | |
| Windows 2000 es-ES | Billboard reprompts localized | |
| Windows 2000 zh-TW | Billboard reprompts localized | |
| Windows 2000 pt-BR | Billboard reprompts localized |
Test #6: Spotcheck Updating Operating System
For this test we need to make sure updating to a supported OS works as expected.
- Start with Firefox 12 Beta on Windows XP SP1
- Verify the hotfix gets installed
- Verify the billboard appears
- Update to SP2 or newer
- Verify the hotfix gets removed
- Verify the billboard does not re-prompt
- RESULT:
Beta Push (Apr 17th, 2012)
- Firefox 10: no hotfix or billboard on any platform
- Firefox 11: no hotfix or billboard on any platform
- Firefox 12: hotfix and billboard ONLY on Windows 2000, XP, XP SP1
- Billboard should be localized
| Platform | Firefox 10 | Firefox 11 | Firefox 12 |
| Windows 2000 | en-US: de: |
en-US: es_ES: |
en-US: pt-BR: |
| Windows XP SP2 | en-US: de: |
en-US: fr: |
en-US: zh-TW: |
| Windows 7 | en-US: it: |
en-US: ro: |
en-US: ja-JP: |
| Mac OSX 10.6 | en-US: de: |
en-US: ko: |
en-US: ru: |
| Ubuntu 11.10 | en-US: de: |
en-US: es_ES: |
en-US: pt-BR: |
Release Push (Apr 23, 2012)
- Firefox 10 ESR: no hotfix or billboard on any platform TODO: add to table
- Firefox 10: no hotfix or billboard on any platform
- Firefox 11: no hotfix or billboard on any platform
- Firefox 12: hotfix and billboard ONLY on Windows 2000, XP, XP SP1
- Billboard should be localized
| Platform | Firefox 10 | Firefox 11 | Firefox 12 |
| Windows 2000 | en-US: de: |
en-US: es_ES: |
en-US: pt-BR: |
| Windows XP SP2 | en-US: de: |
en-US: fr: |
en-US: zh-TW: |
| Windows 7 | en-US: it: |
en-US: ro: |
en-US: ja-JP: |
| Mac OSX 10.6 | en-US: de: |
en-US: ko: |
en-US: ru: |
| Ubuntu 11.10 | en-US: de: |
en-US: es_ES: |
en-US: pt-BR: |