QA/Firefox3/TestPlan/MajorUpdate

Major Update Test Plan: Fx 2.0x - Fx 3.0x

Overview

We will be offering major updates to Firefox 2.0.0.x users so they can upgrade to Firefox 3, and we need to test this functionality before Firefox 3 goes out.

Scope

We are going to test basic major update functionality in some P1 locales, CJKT locales, and partner builds. We want to make sure basic funtionality in the resulting build is the same as a fresh Fx3 install, and we must ensure user preferences are preserved.

Most of this testing will be manually executed. However the partner distribution builds will, in addition, be tested with some automation. The major update test plan for partner distributions is here.

We will need to test MU for CJKT builds as these have special preferences that need to be preserved (?) after the major update. For refence see: CJKT Matrix for Fx1.5x - Fx2 MU.

Basic Tests

At a basic level we should make sure:

• Updates are presented
• In-line content for Details and EULA is localized (not sure if this ready yet)
• Update does not break basic functionality for the user
• User data and preferences are preserved

Extended Tests

The extended tests take into account rich user profiles. For this, we will start with an Fx2 build with bookmarks, history, saved form data, saved passwords, feeds, extensions and themes (mix of compatible and incompatible with Fx3), plugins, default search engine preferences, and any other customizations a user may have for a typical Fx200x installation.

During the major update the extensions and themes should be checked for compatibility.

After going through the major update all (most) of the user's preferences should be preserved.

Instructions

• Download a released Firefox 2.0.0.11 build from here: http://releases.mozilla.org/pub/mozilla.org/firefox/releases/2.0.0.11/
• Install it and change the update channel to betatest
• Restart the browser and select "Check for Updates" from the Help menu
• NOTE: This is for MU testing purposes only. Please do not modify any files if you are not actively testing with the Mozilla QA team. Thanks!

Results

We will track the fx2-fx3 major update testing results in the Main Results Page. For specific iterations of MU testing results see the following links as they are produced:

• Fx2.0.0.11-Fx3beta3 MU results on betatest

Reference

A detailed list of scenarios QA must watch out for is here: http://wiki.mozilla.org/Firefox3/UpgradePolicy

For the 1.5x to 2.0x major updates testing we created a number of tests in Litmus. They are in the 1.5x - 2.0x Major Update testgroup and FX 1.5-2.0 Major Update - Basic Tests and FX 1.5-2.0 Major Update - Extended Tests subgroups in the Firefox 2.0 Catchall test run.

The major update dialog will have an explanatory page, and it will be localized. For example, the 1.5x - 2.0x MU had dialogs that look like this: http://www.chevrel.org/mozco/20migration/display.php

The following is a list of tests taken from the "1.5x - 2.0x Major Update" test group in Litmus, as explained above.

check Details and EULA description
check that you can only install if you accept license
check eulas and descriptions are localized (Paul K., send a list of those that are ready)
check major update works on different locales
check interface elements work, buttons, links, text selection, scrolling, closing
check "fallback" major updates (should there be a failure in the update download for example)
check uninstalling and going back to previous build
check only one type of update is presented at a time minor/major
check major updates if you have multiple profiles
check extension compatibility check
check theme compatibility check
check what tabs are default after you update.
check these start pages are localized as before.
check Windows bits are signed after major update.
post: check minors are presented to the upgraded 2.0.0.1 users
post: check major updates are presented only to those using the latest 150x
post: check minor updates are presented for those users who said never
post: check 200x users are not presented a major
post: check majors a presented when user had said never, then manually checked for updates again (assuming major are once again available)
check to see what happens when one type of update has been downloaded and not applied, and another type of update is made available.
check to see if it is possible to download and say later and then regret and then install something better, like a major update.
check what is left over after a "diff" (currently some crumbs are left for Win and Mac and it has been captured in a bug)
check the state of the registry after a major update (and uninstall)

Test Coverage

We will run the Firefox 2.0->3.0 Major Update Test Suite in litmus across all platforms. Results will be tracked in litmus.

We are also adding FFT test cases in the Firefox 3.0 Catch-All Test Run for one locale, for some of the selected features. These include:

• Location Bar
• Autocomplete
• History
• Bookmarks

Task Breakdown

• Windows XP: Tracy - result
  • en-US (include selected FFTs)
  • it
  • de

• Mac Tiger: Tony - result
  • en-US (include selected FFTs)
  • ja-JP
  • fr

• linux: Juanb - result
  • en-US (include selected FFTs)
  • nl
  • ko

• Vista and Leopard: pri 2, lets hit these after we're done with the above.

Note: bug 394046 tracks generating these updates and serving them. The Litmus migration from past 1.5->2.0.x testcases are tracked in bug 411750

Detailed Results

tchung results

• bug 412348 – MU: Titles are missing in history dropdown for urls w/out www. Filed and nominated for blocking.
• bug 412372 – Software Update dialog window needs text update for Fx2->Fx3. Filed for tracking.