Firefox:1.5.0.x:TBMajor Update 1.5.0.x to 2.0.0.x:Test Results:15012

Testing results tracking page for major updates. We have tested major updates in the past by following the major software update test cases in Litmus, but currently we can do this type of test on "releasetest" going from 1.5.0.12->2.0.0.6. Bear in mind that the testing environment is temporary.

Use the major software update testing strategy page as reference. In-line Details should be translated for all locales, although it is unclear if we will have these ready for Thunderbird major updates this time around. EULAs will default to the en-US locale. These will not be translated.

Test on betatest or releasetest.

Mac (Intel) - juanb

en-US

• include information regarding types of profile data you testested with
• buildId's

Mac (PPC) - marcia

en-US

• Scenario 1 = PASS
• G5 PPC Mac running 10.4.10
• testing with a 1.5.0.x profile created on PPC 10.3.9
• betatest and releastest
• mozilla.com, IMAP, POP3, newsgroup and blog account
• From version 1.5.0.12 (20070509) to version 2.0.0.6 (20070728)
• Configured to automatically check for updates. Hit "Later."
• Installed a theme as was compatible with 1.5 but not with 2.0.0.6 and installed it.
• Check for updates manually and get the update.
• Bugs noted during testing: Bug 349594 - "back/go back" button disabled in software update wizard on the license page for major updates (and elsewhere)

• Scenario 2 - PASS
• Lab PPC Mac running 10.3.9
• testing with a 1.5.0.x profile created on PPC 10.3.9
• releasetest
• mozilla.com, IMAP, POP3, newsgroup and blog account
• From version 1.5.0.12 (20070509) to version 2.0.0.6 (20070728)

Windows XP - Tomcat, stephend, marcia, TimK

stephend test results

en-US - stephend

• Using mscott's XML file for Win, I tested the migration of:
  • labels->tags (changed their default colors and names)
  • filters
  • saved searches (both IMAP/POP3)
  • address books
    • cards/mailing lists
  • column ordering (drag and drop)
  • newsgroups
    • regular and SSL w/certificates
• Tested the suggested 1.5-only/1.5/2 and popular add-ons, and the 1.5-only ones weren't upgraded (as expected). The others were.
• NOTE: Update history didn't work, but hopefully that's because I'm using mscott's file
• 1.5.0.12 (2007050903) -> 2.0.0.6 (2007072817)

 * Haven't yet run the BFT, or even the smoketests, but did a lot of ad-hoc; sorry, I'm away at the baseball game, but will resume testing when I return

timk test results

• En-us (timk)
• Build ID: version 2.0.0.6 (20070728)
• Basic upgrade test passed
  • Later, never
  • Extensions, 1.5 only and 2.0
  • Themes upgraded successfully, though I can't be 100% certain everything looked as it should
  • Feeds
  • Address book
  • Pop-imap-newsgroup accounts
  • Prefs

es-es (timk)

• Build-id: Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.6) Gecko/2007072817 Thunderbird/2.0.0.6
• Same tests as en-us, passed
• The detail pane wasn't localized, but apparently that's not scheduled to happen immediately.

marcia test results

• en-US (marcia)
• Build-id: Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.6) Gecko/2007072817 Thunderbird/2.0.0.6
• 1.5 profile with mozilla.com IMAP
• Tested composing mail to send offline. Don't send it, but wait until after upgrade to send. I did get the dialog prompt to send the mail, but I get a mail relay error when the mail tries to be sent.

Windows Vista - abillings, marcia

en-US

• Tested under various user scenarios, results are available at http://quality.mozilla.org/vista+update+testing
  • NOTE: If you launch Firefox from the shortcut (without "Run as administrator"), software update will fail. You will be able to check for updates, download the patch, and restart... but it will not be applied.)

Windows 2K marcia

en-US

• Scenario 1 - PASS
• testing with an existing POP profile stored on the lab machine. No extensions installed.
• releasetest
• mozilla.com POP3
• From version 1.5.0.12 (20070509) to version 2.0.0.6 (20070728)

• Scenario 2 - PASS
• testing with an existing POP profile stored on the lab machine. No extensions installed.
• betatest
• fail the update and fallover to a full update
• mozilla.com POP3
• From version 1.5.0.12 (20070509) to version 2.0.0.6 (20070728)

Windows 98 marcia

en-US

• Pending reinstallation of OS on lab machine

Linux - ashughes

Profile Details

• Newsgroups: news@mozilla, news@motzerella (SSL)
• IMAP: mozilla.com
• POP3: gmail.com
• Custom Toolbar:
  • Added Buttons: Prev, Next, File, Seperator
  • Moved Buttons: Address Book, Forward
  • Removed Buttons: Junk
  • Other Settings: Use small icons
• Search Folders:
  • IMAP - search for "tbird" from SUBJECT stored in "Inbox-TBird"
  • POP - search for "Armen" from FROM stored in "Inbox-Armen"
• Filters:
  • IMAP - "test"
  • POP - "test"
• Options Dialog: set to Composition>Addressing tab
• Cards/List - created 3 "test" cards and populated 1 "test" list
• Layout - set to vertical view (3-pane)
• Themes:
  • Pitch Dark - installed but not enabled
  • Black Japan - installed but not enabled
  • Vista Mail - installed but not enabled
  • Noia 2.0 eXtreme - installed but not enabled
• Extensions:
  • Contacts Sidebar - installed and enabled
  • NewsWorthy - installed and enabled
  • Adblock Plus - installed and enabled
  • FlashGot - installed and enabled

en-US

• Scenario 1:
  • 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
  • Update method: MScott's XML method
  • Date: 8 Aug 2007
  • Results: PASS
  • Notes:
    • Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found. This is expected behaviour.
    • Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
    • Everything else was migrated and appeared the same as in 1.5.0.12

Focused Testing

Let's try to go through all of the tests in the major updates reference page and test around different kinds of profiles, containing multiple extensions and themes, compatible and incompatible with 2.0x. Keep an eye out for data loss problems, such as mail, labels/tags/settings, and any other preferences that should be preserved after an update.