Firefox:1.5.0.12-2.0.0.4:Test Plan:BFT: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
 
(35 intermediate revisions by 3 users not shown)
Line 1: Line 1:
== RC2 Builds ==
=== BFT's ===
=== Smoketests  ===
There were very few changes between RC1, RC2, and RC3(Firefox 2.0.0.4), so the bft results of rc1 should remain valid. However we can use the test cases in the BFT to test around the bug fixes that went into RC2 and RC3 such as Bookmarks, and CJKT default search engines.
 
{| border="1" cellpadding="5" cellspacing="0"
|-
!
! style="background:#ffdead;" | FF2004
! style="background:#ffdead;" | FF15012
! style="background:#ffdead;" | TB15012
 
|-
| style="background:#ffdead;" | '''Windows'''
|  (tracy) '''tbd'''
|  (juanb) '''tbd'''
|  (marcia) '''tbd'''
|-
| style="background:#ffdead;" | '''Vista'''
|  (marcia, tomcat) '''-PASS, unsigned'''
|  (tomcat) '''tbd'''
|  (tomcat) '''tbd'''
|
|-
| style="background:#ffdead;" | '''Mac'''
|  (tracy) '''tbd'''
|  (juanb) '''tbd'''
|  (marcia) '''tbd'''
|-
| style="background:#ffdead;" | '''Linux'''
|  (tomcat, tracy) '''tbd'''
|  (juanb) '''tbd'''
|  (marcia) '''tbd'''
|-
|}
 
== RC1 Builds ==
=== Basic Functional Tests (BFTs) ===
 
{| border="1" cellpadding="5" cellspacing="0"
|-
!
! style="background:#ffdead;" | FF2004
! style="background:#ffdead;" | FF15012
|-
| style="background:#ffdead;" | '''Windows'''
|  (tomcat) '''PASS'''
|  (juanb) '''Pending'''
|-
| style="background:#ffdead;" | '''Mac'''
|
| optional
|-
| style="background:#ffdead;" | '''Linux'''
|
| optional
|-
|}


=== Smoketests  ===
=== Smoketests  ===
Line 61: Line 7:
|-
|-
!  
!  
! style="background:#ffdead;" | FF2004
! style="background:#ffdead;" | FF2004rc3
! style="background:#ffdead;" | FF15012
! style="background:#ffdead;" | FF15012rc2
! style="background:#ffdead;" | TB15012
! style="background:#ffdead;" | TB15012rc2


|-
|-
Line 69: Line 15:
|  (tracy) '''PASS'''
|  (tracy) '''PASS'''
|  (juanb) '''PASS'''
|  (juanb) '''PASS'''
|  (marcia) '''PASS'''
|  (marcia) '''opt'''
|-
|-
| style="background:#ffdead;" | '''Vista'''
| style="background:#ffdead;" | '''Vista'''
|  (marcia, tomcat) '''-PASS, unsigned'''
|  (marcia, tomcat) '''PASS'''
|  (tomcat) '''PASS'''
|  (tomcat) '''''PASS'''''
|  (tomcat) '''PASS'''
|  (marcia) '''PASS'''
|
|-
|-
| style="background:#ffdead;" | '''Mac'''
| style="background:#ffdead;" | '''Mac'''
Line 85: Line 30:
|  (tomcat, tracy) '''PASS'''
|  (tomcat, tracy) '''PASS'''
|  (juanb) '''PASS'''
|  (juanb) '''PASS'''
|  (marcia) '''PASS'''
|  (tomcat) '''PASS'''
|-
|-
|}
|}
Line 98: Line 43:
* Results
* Results


=== BFT 2004 Results ===
=== Smoketest Results ===


==== Firefox 2004 RC3 ====


* Windows Smoketests RC3 Signed Builds :Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.4) Gecko/2007051502 Firefox/2.0.0.4, tomcat - PASS
** Focus on Signed Files
** Focus on Top Site Compatibility
** Common Operations


=== BFT 15012 Results ===
* Windows XP, smoketest (using Eggplant), 2.0.0.4rc3, BuildID, tracy: '''PASS'''


* Mac, smoketest (using Eggplant), 2.0.0.4rc3, BuildID, tracy: '''PASS'''


* Linux, smoketest (using Eggplant), 2.0.0.4rc3, BuildID, tracy: '''PASS'''


=== Smoketest Results ===


==== Firefox 2.0.0.4 ====
==== Firefox 2004 RC2 ====
===== Windows =====
* Windows XP, smoketest (using Eggplant), 2.0.0.4rc2, BuildID, tracy: '''PASS'''
* Firefox 2.0.0.4 (rc1)
* smoketest
* Windows XP
* Build ID:  Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.4) Gecko/2007050106 Firefox/2.0.0.4
* Tracy on 5-1-07
* PASS


===== Vista =====
* Mac, smoketest (using Eggplant), 2.0.0.4rc2, BuildID, tracy: '''PASS'''
* Firefox 2.0.0.4 (rc1), unsigned build
* modified smoketest, I ran the Vista Suite specific test which covered default browser as well as the fun smoketest suite
* Win Vista Home Basic
* Build ID: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.4) Gecko/2007050106 Firefox/2.0.0.4
* marcia on 5-1-07
* '''PASS'''  
*Verified signed builds on 5-3-07, firefox.exe and dlls are signed, and listing in default programs is correct with digitally embedded URL.
* Issues Noted
**I had the latest version of Java installed, but on the Sun java test page it was not showing the version number. If I do a command line check it shows the correct version number. Java Plug-in 1.6.0_01 for Netscape Navigator (DLL Helper) is the version I show in about:plugins.
**While running a heavy flash demo the pepsigallery.com, I was unable to click in any other tab I had open. More investigation need. Flash version installed on the machine was Shockwave Flash 9.0 r28. Filed https://bugzilla.mozilla.org/show_bug.cgi?id=379530.


* Firefox 2.0.0.4 (rc1), unsigned build
* Linux, smoketest (using Eggplant), 2.0.0.4rc2, BuildID, tracy: '''PASS'''
* Smoketest + specific Vista Tests (default browser, OS Integreation)
* Win Vista Ulitmate VM
* Build ID: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.4) Gecko/2007050106 Firefox/2.0.0.4
* Tomcat on 5-1-07
* '''PASS'''  
* Issues Noted
**Problems to install Java 6.1 via Sun Download Site - Addon Error 203 - Need to investigate this and do a plugin test run


===== Linux =====
* Windows Vista, smoketest , 2.0.0.4rc2, BuildID Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv :1.8.1.4) Gecko/2007050903 Firefox/2.0.0.4 unsigned, tomcat: '''PASS'''
* Test: Smoketest
* Tester: Tomcat
* Tested: 1.May 2007
* Build ID:  Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.4) Gecko/2007050105 Firefox/2.0.0.4
* Status : '''PASS''' Results in Litmus


* Firefox 2.0.0.4 (rc1)
Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.4) Gecko/2007051502 Firefox/2.0.0.4
* smoketest
* Linux FC5
* Build ID:  Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.4) Gecko/2007050105 Firefox/2.0.0.4
* Tracy on 5-1-07
* PASS


===== Mac =====
*General Testing of Firefox 2.0.0.4 on Vista, emphasizing the following [marcia]
**Bookmarks (ran a regression suite)
**Common operations
**Updating all plugins


* Firefox 2.0.0.4 (rc1)
==== Firefox 15012 RC2 ====
* smoketest
* Windows XP, smoketests, 1.5.0.12rc2(unsigned), 2007050813, juanb, 5/09: '''PASS'''
* Mac (PPC) 10.4.8
* Build ID:  Mozilla/5.0 (Macintosh; U; Mac OS X Mach-O; en-US; rv:1.8.1.4) Gecko/2007050106 Firefox/2.0.0.4
* Tracy on 5-1-07
* PASS


==== Firefox 1.5.0.12 ====
* Linux Ubuntu (vm), smoketests, 1.5.0.12rc2, 2007050813, juanb, 5/09: '''PASS'''
===== Windows =====
* Test: 1.5.0.12 Smoketest on WinXP (vm)
* Tester: juanb
* Date: 5/1
* Version ID: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.12) Gecko 20070501 Firefox/1.5.0.12
* Build ID: 2007050107
* Status: PASS (results in Litmus)


===== Vista =====
* Windows Vista, smoketests, 1.5.0.12rc2(unsigned), Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.0.12) Gecko/20070508 , tomcat, 5/09: '''PASS'''
* Test: 1.5.0.12 Smoketest on Vista
* Tester: tomcat
* Date: 1. May
* Version ID: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.0.12) Gecko/20070501 Firefox/1.5.0.12
* Build ID: 2007050107
* Status: PASS (results in Litmus)
* Note: Setting as default browser only worked after 2nd restart of Firefox.


===== Linux =====
* Windows Vista, smoketests, 1.5.0.12 rc2(signed),Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.0.12) Gecko/20070508 , tomcat, 5/09: '''PASS'''
* Test: 1.5.0.12rc1 Smoketest on Fedora
* Tester: juanb
* Date: 5/1
* Version ID: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.12) Gecko/20070501 Firefox/1.5.0.12
* Build ID: 2007050107
* Status: PASS (results in Litmus)


===== Mac =====
* MacIntel, smoketests, 1.5.0.12rc2, 2007050815, juanb, 5/10: '''PASS'''
* Test: 1.5.0.12 Smoketest on MacIntel
* Tester: juanb
* Date: 5/1
* Version ID: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.0.12) Gecko/20070501 Firefox/1.5.0.12
* Build ID: 2007050107
* Status: PASS (results in Litmus)


==== Thunderbird 1.5.0.12 ====
==== Thunderbird 15012 RC2 ====


===== Windows =====
===== Windows =====
* Test: 1.5.0.12 Smoketest on Windows XP
* Test: 1.5.0.12 Smoketest on Win Vista Home Basic
* Tester: marcia
* Tester: marcia
* Date: 5/3
* Date: 5/9
* Version ID: version 1.5.0.12 (20070502)
* Version ID: version 1.5.0.12 (20070509)
* Build ID: 2007050212
* Build ID: 2007050903
* Status: '''PASS''' (results in Litmus)
* Status: '''PASS''' (results in Litmus)
* Minor nits
* Issues: Difficult importing mail from Windows Mail, it didn't like our IMAP configuration.
**Get ext/themes directs to Firefox page instead of Tbird page on addons
**If you set your inbox to one group by sort state and then switch to another folder, that state is not preserved. This behaved differently than Mac, it actually expanded the "yesterday" group when I did not have that expanded before switching folders.


===== Vista =====
===== Windows Vista =====
 
* Test: 1.5.0.12 Smoketest on Win Vista  
*No Smoketest was performed on Vista, since we don't support all the default email client stuff we thought it was better to concentrate on Win XP.
* Tester: tomcat
* Date: 19.May 2007
* Version ID: version 1.5.0.12 (20070509)
* Build ID: 2007050903
* Status: '''PASS''' (results in Litmus)


===== Mac =====
===== Mac =====
* Test: 1.5.0.12 Smoketest on MacIntel 10.4.9
* Test: 1.5.0.12 Smoketest on MacIntel
* Tester: marcia
* Tester: marcia
* Date: 5/3
* Date: 5/9
* Version ID: version 1.5.0.12 (20070502)
* Version ID: version 1.5.0.12 (20070509)
* Build ID: 2007050210
* Build ID: 2007050903,
* Status: PASS (results in Litmus)
* Status: '''PASS''' (results in Litmus)
* Minor nits
* Issues: I crashed once, in this stack:  PL_DHashTableOperate + 27. Looks to be the same stack as https://bugzilla.mozilla.org/show_bug.cgi?id=349463
**Get ext/themes directs to Firefox page instead of Tbird page on addons
**If you set your inbox to one group by sort state and then switch to another folder, that state is not preserved.


===== Linux =====
===== Linux =====
 
* Test: 1.5.0.12 Smoketest Fedora FC 5
* Test: 1.5.0.12 Smoketest on Fedora Core
* Tester: tomcat
* Tester: marcia
* Date: 5/10
* Date: 5/4
* Version ID: version 1.5.0.12 (20070509)
* Version ID: version 1.5.0.12 (20070502)
* Build ID: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.12)Gecko/20070509 Thunderbird/1.5.0.12 ID:2007050903
* Build ID: 2007050210
* Status: '''PASS''' (results in Litmus)
* Status: '''PASS''' (results in Litmus)
* Minor nits
* Issues: Some JS Errors on First Start with new Profile and new Account setup, gone after restart
**I crashed once in the compose window.


[http://wiki.mozilla.org/Firefox:1.5.0.11-2.0.0.3:Test_Plan Back to the Test Plan page]
=== Bugs Filed during RC2 testing ===
*https://bugzilla.mozilla.org/show_bug.cgi?id=380181


=== Bugs found during testing  ===
== RC1 Builds ==
* https://bugzilla.mozilla.org/show_bug.cgi?id=379648
Results moved to this page: [http://wiki.mozilla.org/Firefox:1.5.0.12-2.0.0.4:Test_Plan:BFT:RC1_Builds '''15012/2004 RC1 Results''']
* https://bugzilla.mozilla.org/show_bug.cgi?id=379530
 
=== Top Site Testing ===
 
 
 
* Detailed Results and tested Sites -> [http://wiki.mozilla.org/Firefox:1.5.0.12-2.0.0.4:Test_Plan:BFT:TopSite]
 
=== Addon - Testing ===
 
 
* Detailed Results and tested Addons -> [http://wiki.mozilla.org/Firefox:1.5.0.12-2.0.0.4:Test_Plan:BFT:Addon]

Latest revision as of 16:55, 21 May 2007

BFT's

There were very few changes between RC1, RC2, and RC3(Firefox 2.0.0.4), so the bft results of rc1 should remain valid. However we can use the test cases in the BFT to test around the bug fixes that went into RC2 and RC3 such as Bookmarks, and CJKT default search engines.

Smoketests

FF2004rc3 FF15012rc2 TB15012rc2
Windows (tracy) PASS (juanb) PASS (marcia) opt
Vista (marcia, tomcat) PASS (tomcat) PASS (marcia) PASS
Mac (tracy) PASS (juanb) PASS (marcia) PASS
Linux (tomcat, tracy) PASS (juanb) PASS (tomcat) PASS

Detailed Results

Any smoketest and/or BFTs assignments and detailed results should be tracked here. Please include the following information:

  • Product and release tested
  • Type of test (smoketest/BFT)
  • Platform (Windows, Linux, MacOS X), architecture for Mac (PPC or MacIntel)
  • Build ID (from master.ini or using Nightly Tester Tools extension)
  • Tester and date
  • Results

Smoketest Results

Firefox 2004 RC3

  • Windows Smoketests RC3 Signed Builds :Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.4) Gecko/2007051502 Firefox/2.0.0.4, tomcat - PASS
    • Focus on Signed Files
    • Focus on Top Site Compatibility
    • Common Operations
  • Windows XP, smoketest (using Eggplant), 2.0.0.4rc3, BuildID, tracy: PASS
  • Mac, smoketest (using Eggplant), 2.0.0.4rc3, BuildID, tracy: PASS
  • Linux, smoketest (using Eggplant), 2.0.0.4rc3, BuildID, tracy: PASS


Firefox 2004 RC2

  • Windows XP, smoketest (using Eggplant), 2.0.0.4rc2, BuildID, tracy: PASS
  • Mac, smoketest (using Eggplant), 2.0.0.4rc2, BuildID, tracy: PASS
  • Linux, smoketest (using Eggplant), 2.0.0.4rc2, BuildID, tracy: PASS
  • Windows Vista, smoketest , 2.0.0.4rc2, BuildID Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv :1.8.1.4) Gecko/2007050903 Firefox/2.0.0.4 unsigned, tomcat: PASS

Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.1.4) Gecko/2007051502 Firefox/2.0.0.4

  • General Testing of Firefox 2.0.0.4 on Vista, emphasizing the following [marcia]
    • Bookmarks (ran a regression suite)
    • Common operations
    • Updating all plugins

Firefox 15012 RC2

  • Windows XP, smoketests, 1.5.0.12rc2(unsigned), 2007050813, juanb, 5/09: PASS
  • Linux Ubuntu (vm), smoketests, 1.5.0.12rc2, 2007050813, juanb, 5/09: PASS
  • Windows Vista, smoketests, 1.5.0.12rc2(unsigned), Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.0.12) Gecko/20070508 , tomcat, 5/09: PASS
  • Windows Vista, smoketests, 1.5.0.12 rc2(signed),Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.0.12) Gecko/20070508 , tomcat, 5/09: PASS
  • MacIntel, smoketests, 1.5.0.12rc2, 2007050815, juanb, 5/10: PASS

Thunderbird 15012 RC2

Windows
  • Test: 1.5.0.12 Smoketest on Win Vista Home Basic
  • Tester: marcia
  • Date: 5/9
  • Version ID: version 1.5.0.12 (20070509)
  • Build ID: 2007050903
  • Status: PASS (results in Litmus)
  • Issues: Difficult importing mail from Windows Mail, it didn't like our IMAP configuration.
Windows Vista
  • Test: 1.5.0.12 Smoketest on Win Vista
  • Tester: tomcat
  • Date: 19.May 2007
  • Version ID: version 1.5.0.12 (20070509)
  • Build ID: 2007050903
  • Status: PASS (results in Litmus)
Mac
  • Test: 1.5.0.12 Smoketest on MacIntel
  • Tester: marcia
  • Date: 5/9
  • Version ID: version 1.5.0.12 (20070509)
  • Build ID: 2007050903,
  • Status: PASS (results in Litmus)
  • Issues: I crashed once, in this stack: PL_DHashTableOperate + 27. Looks to be the same stack as https://bugzilla.mozilla.org/show_bug.cgi?id=349463
Linux
  • Test: 1.5.0.12 Smoketest Fedora FC 5
  • Tester: tomcat
  • Date: 5/10
  • Version ID: version 1.5.0.12 (20070509)
  • Build ID: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.12)Gecko/20070509 Thunderbird/1.5.0.12 ID:2007050903
  • Status: PASS (results in Litmus)
  • Issues: Some JS Errors on First Start with new Profile and new Account setup, gone after restart

Bugs Filed during RC2 testing

RC1 Builds

Results moved to this page: 15012/2004 RC1 Results

Retrieved from "https://wiki.mozilla.org/index.php?title=Firefox:1.5.0.12-2.0.0.4:Test_Plan:BFT&oldid=57289"