QA/Firefox3/TestPlan/MajorUpdate

Major Update Test Plan: Fx 2.0x - Fx 3.0x

Overview

We will be offering major updates to Firefox 2.0.0.x users so they can upgrade to Firefox 3, and we need to test this functionality before Firefox 3 goes out.

Scope

We are going to test basic major update functionality in some P1 locales, CJKT locales, and partner builds. We want to make sure basic funtionality in the resulting build is the same as a fresh Fx3 install, and we must ensure user preferences are preserved.

Most of this testing will be manually executed. However the partner distribution builds will, in addition, be tested with some automation. The major update test plan for partner distributions is here.

We will need to test MU for CJKT builds as these have special preferences that need to be preserved (?) after the major update. For refence see: CJKT Matrix for Fx1.5x - Fx2 MU.

Schedule

There are no dates for a final release, yet. Sometime after Firefox 3.0.2 is released is, however, the best estimate.

We will run a few iterations of MU on testing channels. A major update release to all users depends on the timing of Fx2.x and Fx3.x releases.

As of 7.18.2008 we need to test the major update scenario in the majortest channel to find as many issues as possible so they can be fixed perhaps by the time Fx3.0.2 is released.

Basic Tests

At a basic level we should make sure:

• Updates are presented
• In-line content for Details and EULA is localized
• Update does not break basic functionality for the user
• User data and preferences are preserved

Extended Tests

The extended tests take into account rich user profiles. For this, we will start with an Fx2 build with bookmarks, history, saved form data, saved passwords, feeds, extensions and themes (mix of compatible and incompatible with Fx3), plugins, default search engine preferences, and any other customizations a user may have for a typical Fx200x installation.

During the major update the extensions and themes should be checked for compatibility.

After going through the major update all (most) of the user's preferences should be preserved.

Instructions

• Download the latest Fx2.x version for which a major update is offered, see bug 394046
• Install it and change the update channel to majortest
• Restart the browser and select "Check for Updates" from the Help menu
• NOTE: This is for MU testing purposes only. Please do not modify any files if you are not actively testing with the Mozilla QA team. Thanks!

Results

We will track the fx2-fx3 major update testing results in the Main Results Page. For specific iterations of MU testing results see the following links as they are produced:

• Fx2.0.0.16-Fx3.0.1 MU results on releasetest and betatest - as of 8.21.2008
• Fx2.0.0.16-Fx3.0.1 MU results on majortest - Current iteration as of 8.20.2008
• Fx2.0.0.14-Fx3.0 MU results on majortest - as of 7.16.2008
• Fx2.0.0.11-Fx3beta3 MU results on betatest - Previous iterations

Reference (legacy)

A detailed list of scenarios QA must watch out for is here: http://wiki.mozilla.org/Firefox3/UpgradePolicy

For the 1.5x to 2.0x major updates testing we created a number of tests in Litmus. They are in the 1.5x - 2.0x Major Update testgroup and FX 1.5-2.0 Major Update - Basic Tests and FX 1.5-2.0 Major Update - Extended Tests subgroups in the Firefox 2.0 Catchall test run.

The major update dialog will have an explanatory page, and it will be localized. For example, the 1.5x - 2.0x MU had dialogs that look like this: http://www.chevrel.org/mozco/20migration/display.php

The following is a list of tests taken from the "1.5x - 2.0x Major Update" test group in Litmus, as explained above.

check Details and EULA description
check that you can only install if you accept license
check eulas and descriptions are localized (Paul K., send a list of those that are ready)
check major update works on different locales
check interface elements work, buttons, links, text selection, scrolling, closing
check "fallback" major updates (should there be a failure in the update download for example)
check uninstalling and going back to previous build
check only one type of update is presented at a time minor/major
check major updates if you have multiple profiles
check extension compatibility check
check theme compatibility check
check what tabs are default after you update.
check these start pages are localized as before.
check Windows bits are signed after major update.
post: check minors are presented to the upgraded 2.0.0.1 users
post: check major updates are presented only to those using the latest 150x
post: check minor updates are presented for those users who said never
post: check 200x users are not presented a major
post: check majors a presented when user had said never, then manually checked for updates again (assuming major are once again available)
check to see what happens when one type of update has been downloaded and not applied, and another type of update is made available.
check to see if it is possible to download and say later and then regret and then install something better, like a major update.
check what is left over after a "diff" (currently some crumbs are left for Win and Mac and it has been captured in a bug)
check the state of the registry after a major update (and uninstall)