MozillaWiki

Firefox:1.5.0.x:TBMajor Update 1.5.0.x to 2.0.0.x:Test Results:15012: Difference between revisions

Page Discussion

Firefox:1.5.0.x:TBMajor Update 1.5.0.x to 2.0.0.x:Test Results:15012 (view source)

Revision as of 02:01, 11 August 2007

17,577 bytes added ,  11 August 2007
→‎Bugs Discovered during Testing
 
(39 intermediate revisions by 6 users not shown)
Line 1: Line 1:
Testing results tracking page for major updates. We have tested major updates in the past by following the [http://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=1&testgroup_id=38&subgroup_id=623 major software update test cases in Litmus], but currently we can do this type of test on "releasetest" going from 1.5.0.12->2.0.0.6. Bear in mind that the testing environment is temporary.
Testing results tracking page for major updates. We have tested major updates in the past by following the [http://litmus.mozilla.org/show_test.cgi?searchType=by_category&product_id=1&testgroup_id=38&subgroup_id=623 major software update test cases in Litmus], but currently we can do this type of test on "releasetest" going from 1.5.0.12->2.0.0.6. Bear in mind that the testing environment is temporary.


Use the major software update [http://wiki.mozilla.org/Firefox:1.5.0.x:Major_Update_1.5.0.x_to_2.0.0.x testing strategy page] as reference. In-line Details have been translated for all locales. Some EULAS may not be translated and they default to the en-US version.
Use the major software update [http://wiki.mozilla.org/Firefox:1.5.0.x:Major_Update_1.5.0.x_to_2.0.0.x testing strategy page] as reference. In-line Details should be translated for all locales, although it is unclear if we will have these ready for Thunderbird major updates this time around. EULAs will default to the en-US locale. These will not be translated.


Test on betatest or releasetest.
Test on betatest or releasetest.


= Mac (Intel) - juanb =
= Mac (Intel) - juanb =
'''en-US'''
'''en-US''' - '''PASS'''
* include information regarding types of profile data you testested with
* Thunderbird BuildIDs: 2007050903-2007072816
* buildId's
* Tested with a profile containing different accounts and settings:
** POP, IMAP, RSS Feeds, Newsgroups
** Toolbar customizations
** Saved searches
** Filters
** Tags, Flags
** Sorting
** Address book entries, mailing lists
** LDAP configuration
** Compatible and incompatible extensions/themes
 
* Accounts and settings were preserved
* Extensions and Themes were checked for compatibility
* Functionality remains OK.
* Software Update History showed successful update entry
* Fallback update (after a forced failure) worked as well.
 
'''es-ES''' - '''PASS'''
* Checked similar things and all seemed ok
* Major Software Update dialog box doesn't look as polished as en-US
 
'''ja-jp-mac''' - '''PASS'''
 
'''fr''' - '''PASS'''


= Mac (PPC) - marcia =
= Mac (PPC) - marcia =
'''en-US'''
'''en-US'''
* Scenario 1 = PASS
* Scenario 1 = PASS
* G5 PPC Mac running 10.4.10
* testing with a 1.5.0.x profile created on PPC 10.3.9
* testing with a 1.5.0.x profile created on PPC 10.3.9
* betatest and releastest
* betatest and releastest
* mozilla.com, IMAP, Pop3, newsgroup and blog account
* mozilla.com, IMAP, POP3, newsgroup and blog account
* From version 1.5.0.12 (20070509) to version 2.0.0.6 (20070728)
* From version 1.5.0.12 (20070509) to version 2.0.0.6 (20070728)
* Configured to automatically check for updates. Hit "Later."
* Configured to automatically check for updates. Hit "Later."
Line 22: Line 46:
* Bugs noted during testing: [https://bugzilla.mozilla.org/show_bug.cgi?id=349594 Bug 349594] - "back/go back" button disabled in software update wizard on the license page for major updates (and elsewhere)
* Bugs noted during testing: [https://bugzilla.mozilla.org/show_bug.cgi?id=349594 Bug 349594] - "back/go back" button disabled in software update wizard on the license page for major updates (and elsewhere)


= Windows XP - Tomcat, stephend, TimK =
* Scenario 2 - PASS
* Lab PPC Mac running 10.3.9
* testing with a 1.5.0.x profile created on PPC 10.3.9
* releasetest
* mozilla.com, IMAP, POP3, newsgroup and blog account
* From version 1.5.0.12 (20070509) to version 2.0.0.6 (20070728)
 
= Windows XP - Tomcat, stephend, marcia, TimK, whimboo =
==stephend test results==
'''en-US''' - '''stephend'''
'''en-US''' - '''stephend'''
* Using mscott's XML file for Win, I tested the migration of:
* Using mscott's XML file for Win, I tested the migration of:
Line 37: Line 69:
* 1.5.0.12 (2007050903) -> 2.0.0.6 (2007072817)
* 1.5.0.12 (2007050903) -> 2.0.0.6 (2007072817)
   * Haven't yet run the BFT, or even the smoketests, but did a lot of ad-hoc; sorry, I'm away at the baseball game, but will resume testing when I return
   * Haven't yet run the BFT, or even the smoketests, but did a lot of ad-hoc; sorry, I'm away at the baseball game, but will resume testing when I return
==timk test results==
*En-us (timk)
* Build ID: version 2.0.0.6 (20070728)
* Basic upgrade test passed
** Later, never
** Extensions, 1.5 only and 2.0
** Themes upgraded successfully, though I can't be 100% certain everything looked as it should
** Feeds
** Address book
** Pop-imap-newsgroup accounts
** Prefs
es-es (timk)
* Build-id: Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.1.6) Gecko/2007072817 Thunderbird/2.0.0.6
* Same tests as en-us, passed
* The detail pane wasn't localized, but apparently that's not scheduled to happen immediately.
==marcia test results==
*All '''PASS''' except for mismatch issue noted.
*en-US (marcia)
* Build-id:
* 1.5 profile with mozilla.com IMAP
* Tested composing mail to send offline. Don't send it, but wait until after upgrade to send. I did get the dialog prompt to send the mail, but I get a mail relay error when the mail tries to be sent.
*it (marcia)
* Build-id: versione 2.0.0.6 (20070728)
* 1.5 profile with mozilla.com IMAP
* Tested composing mail to send offline. Don't send it, but wait until after upgrade to send. I did get the dialog prompt to send the mail, but I get a mail relay error when the mail tries to be sent.
*fr (marcia)
* Build-id: versione 2.0.0.6 (20070728)
* 1.5 profile with mozilla.com IMAP and POP3
* Filed https://bugzilla.mozilla.org/show_bug.cgi?id=391785 during this test cycle
==whimboo test results==
en-US (whimboo)
* Build-id: Gecko/2007072817 Thunderbird/2.0.0.6
* Compared original 2.0.0.6 application folder with MU application folder - no missing or remaining files.
* tested update while files are in use. inifinitly restart: bug 340535 happens
== Tomcats Test Results ==
*en-US (tomcat)
* Build-id:
* 1.5 profile with mozilla.com IMAP and yahoo pop
* Tested some Extensions and Mail profiles with mails and newsgroups and all went as expected.
*de (tomcat)
* Build-id: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070728 Thunderbird/2.0.0.6 Mnenhy/0.7.5.0 ID:2007072817
* 1.5 profile with mozilla.com IMAP and yahoo pop
* Tested some Extensions and Mail profiles with mails and newsgroups and all went as expected.
*de (tomcat)
* Build-id: tested on mac - ppc
* 1.5 profile with mozilla.com IMAP and yahoo pop
* Tested some Extensions and Mail profiles with mails and newsgroups and all went as expected.


= Windows Vista - abillings, marcia =
= Windows Vista - abillings, marcia =
Line 42: Line 130:
* Tested under various user scenarios, results are available at http://quality.mozilla.org/vista+update+testing
* Tested under various user scenarios, results are available at http://quality.mozilla.org/vista+update+testing
** NOTE: If you launch Firefox from the shortcut (without "Run as administrator"), software update will fail.  You will be able to check for updates, download the patch, and restart... but it will not be applied.)
** NOTE: If you launch Firefox from the shortcut (without "Run as administrator"), software update will fail.  You will be able to check for updates, download the patch, and restart... but it will not be applied.)
= Windows 2K marcia =
'''en-US'''
*Scenario 1 - PASS
*testing with an existing POP profile stored on the lab machine. No extensions installed.
* releasetest
* mozilla.com POP3
* From version 1.5.0.12 (20070509) to version 2.0.0.6 (20070728)
*Scenario 2 - PASS
*testing with an existing POP profile stored on the lab machine. No extensions installed.
* betatest
* fail the update and fallover to a full update
* mozilla.com POP3
* From version 1.5.0.12 (20070509) to version 2.0.0.6 (20070728)
= Windows 98 marcia =
'''en-US'''
*Pending reinstallation of OS on lab machine


= Linux - ashughes =
= Linux - ashughes =
'''Profile Details'''
* Newsgroups:  news@mozilla, news@motzerella (SSL)
* IMAP:  mozilla.com
* POP3: gmail.com
* Custom Toolbar:
** Added Buttons: Prev, Next, File, Seperator
** Moved Buttons: Address Book, Forward
** Removed Buttons: Junk
** Other Settings: Use small icons
* Search Folders:
** IMAP - search for "tbird" from SUBJECT stored in "Inbox-TBird"
** POP - search for "Armen" from FROM stored in "Inbox-Armen"
* Filters:
** IMAP - "test"
** POP - "test"
* Options Dialog: set to Composition>Addressing tab
* Cards/List - created 3 "test" cards and populated 1 "test" list
* Layout - set to vertical view (3-pane)
* Themes:
** Pitch Dark - installed but not enabled
** Black Japan - installed but not enabled
** Vista Mail - installed but not enabled
** Noia 2.0 eXtreme - installed but not enabled
* Extensions:
** Contacts Sidebar - installed and enabled
** NewsWorthy - installed and enabled
** Adblock Plus - installed and enabled
** FlashGot - installed and enabled
* RSS Reader: digg
* Inbox column headings reordered to SENDER, DATE, SUBJECT
'''en-US'''
'''en-US'''
* include information describing profile characteristics
* Scenario 1:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: MScott's XML method
** Date: 8 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
 
* Scenario 2:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = releasetest, ran litmus testcases
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
*** Litmus testcases: test 12 failed due to above toolbar problem, test 13 unclear due to below Update History issue, all other tests PASS.
*** Build ID reported in Update History is 2007071612
 
* Scenario 3:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = releasetest, theme set to Black Japan (1.5 only)
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
*** Theme reverted to default theme with no warnings.  Reverting back to 1.5.0.12 restores Black Japan theme.
 
* Scenario 4:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = releasetest, theme set to Pitch Dark (1.5/2 compatible)
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
*** Theme Pitch Dark still enabled after update.
 
* Scenario 5:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = releasetest, theme set to Vista Mail (popular)
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
*** Theme Vista Mail still enabled after update.
 
* Scenario 6:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = releasetest, theme set to Noia 2.0 eXtreme (popular)
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
*** Theme Noia 2.0 eXtreme still enabled after update.
 
* Scenario 7:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = betatest
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
 
'''fr'''
* Scenario 1:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = releasetest
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
 
* Scenario 2:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = betatest
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
 
* Scenario 3:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = releasetest, set theme to 1.5 only theme
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
*** Theme reverted to default theme after update.  Downgrading to 1.5.0.12 reverts the theme to the 1.5 only theme.
 
* Scenario 4:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = betatest, set theme to 2.0 compatible theme
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
*** Custom theme still enabled after update.
 
* Scenario 5:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = releasetest, ran litmus tests
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
*** Custom theme still enabled after update.
*** Build ID reported in Update History is 2007071612
*** Litmus testcases: test 12 failed due to above toolbar problem, test 13 unclear due to Update History issue, all other tests PASS.
 
'''de'''
* Scenario 1:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = releasetest
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
 
* Scenario 2:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = betatest
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
 
* Scenario 3:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = releasetest, set theme to 1.5 only theme
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
*** Theme reverted to default theme after update.  Downgrading to 1.5.0.12 reverts the theme to the 1.5 only theme.
 
* Scenario 4:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = betatest, set theme to 2.0 compatible theme
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
*** Custom theme still enabled after update.
 
* Scenario 5:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = releasetest, ran litmus tests
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
*** Custom theme still enabled after update.
*** Build ID reported in Update History is 2007071612
*** Litmus testcases: test 12 failed due to above toolbar problem, test 13 unclear due to Update History issue, all other tests PASS.
 
'''jp'''
* Scenario 1:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = releasetest
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
 
* Scenario 2:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = betatest
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
 
* Scenario 3:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = releasetest, set theme to 1.5 only theme
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
*** Theme reverted to default theme after update.  Downgrading to 1.5.0.12 reverts the theme to the 1.5 only theme.
 
* Scenario 4:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = betatest, set theme to 2.0 compatible theme
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
*** Custom theme still enabled after update.
 
* Scenario 5:
** 1.5.0.12 (20070509) -> 2.0.0.6 (20070728)
** Update method: channel = releasetest, ran litmus tests
** Date: 10 Aug 2007
** Results: PASS
** Notes:
*** Themes and Extensions that did not have 2.0 compatible versions were checked for updates and none were found.  This is expected behaviour.
*** Buttons that were added to the custom toolbar remained, however all other changes were set to defaults.
*** Everything else was migrated and appeared the same as in 1.5.0.12
*** Custom theme still enabled after update.
*** Build ID reported in Update History is 2007071612
*** Litmus testcases: test 12 failed due to above toolbar problem, test 13 unclear due to Update History issue, all other tests PASS.


= Focused Testing =
= Focused Testing =
Let's try to go through all of the tests in the major updates reference page and test around different kinds of profiles, containing multiple extensions and themes, compatible and incompatible with 2.0x. Keep an eye out for data loss problems, such as mail, labels/tags/settings, and any other preferences that should be preserved after an update.
Let's try to go through all of the tests in the major updates reference page and test around different kinds of profiles, containing multiple extensions and themes, compatible and incompatible with 2.0x. Keep an eye out for data loss problems, such as mail, labels/tags/settings, and any other preferences that should be preserved after an update.
= Bugs Discovered during Testing =
*[https://bugzilla.mozilla.org/show_bug.cgi?id=391785 Bug 391785]
*[https://bugzilla.mozilla.org/show_bug.cgi?id=391792 Bug 391792]
Marcia
canmove, Confirmed users
7,108

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/65065...65178"