QA/Thunderbird3/TestResults/Beta1: Difference between revisions

From MozillaWiki
< QA
Jump to navigation Jump to search
(→‎Testing Results Summary: Add search links for Litmus)
 
(12 intermediate revisions by 2 users not shown)
Line 27: Line 27:
== Release Notes ==
== Release Notes ==


'''Please read [http://stage.mozillamessaging.com/en-US/thunderbird/3.0b1/releasenotes/ draft release notes] before testing.'''
'''Please read [http://trunk.mozillamessaging.com/en-US/thunderbird/3.0b1/releasenotes/ draft release notes] before testing.'''
<!-- http://stage.mozillamessaging.com/en-US/thunderbird/3.0b1/releasenotes/ -->
<!-- * [http://www.mozillamessaging.com/en-US/thunderbird/3.0b1/releasenotes/ Final Beta 1 Release Notes] -->


Additional information:  
Additional information for testers:  
* [[Thunderbird:QA_TestDay:2008-11-14#Focus|Details of new features]], as documented in a previous testday for 3.0b1
* [[Thunderbird:QA_TestDay:2008-11-14#Focus|Some new features]], as documented in a previous testday for 3.0b1
<!-- * [http://www.mozillamessaging.com/en-US/thunderbird/3.0b1/releasenotes/ Final Beta 1 Release Notes] -->
* [http://tinyurl.com/5ko8vf open trunk regressions, confirmed]
* [http://tinyurl.com/5ko8vf trunk regressions, confirmed]
* [http://tinyurl.com/6mz5b9 open trunk regressions, unconfirmed]
* [http://tinyurl.com/6mz5b9 trunk regressions, unconfirmed]
* for bugs fixed, please see the release notes
* for bugs fixed, please see the release notes
* {{Bug|466054}} - Prepare Thunderbird 3 Beta 1 release notes
* {{Bug|466054}} - Prepare Thunderbird 3 Beta 1 release notes


== Schedule ==  
== Schedule ==  
;Builds handed to QA: 2008-12-02?
;Builds handed to QA: 2008-12-02
;QA signoff:
;QA signoff:
* Smoketests - 2008-12-??
* Smoketests - 2008-12-08
* Spotchecks - 2008-12-??
* Spotchecks - 2008-12-08
* BFTs - 2008-12-??
* BFTs - 2008-12-08


== Build Location and IDs ==
== Build Location and IDs ==
=== Build 2 ===
Build 2 incorporates bug fixes documented in Beta 1 respin {{Bug|467813}}.
Please do not test using old builds (prior alphas, betas, nightlies). Download candidates builds from http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/3.0b1-candidates/build2
en-US build ids:
* '''Windows''' - version 3.0b1 (20081204115318)
* '''Linux''' - version 3.0b1 (20081204114258)
* '''OS X [PPC Mac]''': version 3.0b1 (20081204113938)
* '''OS X [Intel Mac]''' - version 3.0b1 (20081204113938)
=== Build 1 ===
=== Build 1 ===


Please do not test using prior alphas, prior betas or a nightly. Download release candidates at [http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/3.0b1-candidates/build1/ http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/3.0b1-candidates/build1]
<strike>(build1 is obsolete)
 
Please do not test using prior alphas, prior betas or a nightly. Download candidate builds at [http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/3.0b1-candidates/build1/ http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/3.0b1-candidates/build1]


en-US build ids:
en-US build ids:
* '''Windows''' - version 3.0b1 (20081201164237)  
* '''Windows''' - version 3.0b1 (20081201164237)  
* '''Linux''' - version 3.0b1 (20081201164237)
* '''Linux''' - version 3.0b1 (20081201162550)
* '''OS X [PPC Mac]''': version 3.0b1 (20081201164237)  
* '''OS X [PPC Mac]''': version 3.0b1 (20081201162550)  
* '''OS X [Intel Mac]''' - version 3.0b1 (20081201164237)
* '''OS X [Intel Mac]''' - version 3.0b1 (20081201162550)
</strike>


== Test Coverage ==
== Test Coverage ==
Line 59: Line 76:
Objectives:  
Objectives:  
* smoketest - all 3 major platforms, and as many locales as possible
* smoketest - all 3 major platforms, and as many locales as possible
* BFT - at least one platform (BFT is divided into pieces based on tester's last name to reduce time required of each person -- so all tests get done, but not necessarily all of a test by one person).
* BFT - at least one platform (in "pieces", i.e. all not done by one person).


* Areas Covered (en-US):
* Areas Covered (en-US):
** Spotchecks
** Spotchecks
** Smoketests  
** Smoketests  
** BFTs
** BFTs - excluding themes, updates, addons
** L10n, spotcheck all shipping locales
** L10n, spotcheck all shipping locales
* Areas Not Covered:
* Areas Not Covered:
** FFTs
** FFTs
** Updates


Previous release (for reference purposes): Alpha 3 smoketested all 3 major platforms, full coverage of windows BFT, all shipping locales were spot checked
Previous release (for reference purposes): Alpha 3 smoketested all 3 major platforms, full coverage of windows BFT, all shipping locales spot checked


=== QA Summary ===
=== QA Summary ===
* Smoketests: 2008-12-?? - 2008-12-??
* Smoketests: 2008-12-02 - 2008-12-07
* Windows XP/Vista BFTs: 2008-12-?? - 2008-12-??
* Windows XP/Vista BFTs: 2008-12-02 - 2008-12-07
* Linux BFTs: 2008-12-?? - 2008-12-??
* Linux BFTs: 2008-12-02 - 2008-12-07
* Mac BFTs: -
* Mac BFTs: -
<!-- * Spot check of Mac and Linux Platforms:: 2008-12-?? to 2008-12-?? -->
<!-- * Spot check of Mac and Linux Platforms:: 2008-12-?? to 2008-12-?? -->
Line 83: Line 99:


'''Smoketests'''
'''Smoketests'''
* Windows XP SP1 (-who-) - 2008-12-??
* Windows XP SP1 (-who-) - 2008-12-?? [https://litmus.mozilla.org/advanced_search.cgi?test_run=37&platform=1&opsys=4&end_date=20081222&limit=120]
* Windows Vista SP1 (-who-) - 2008-12-??
* Windows Vista SP1 (-who-) - 2008-12-?? [https://litmus.mozilla.org/advanced_search.cgi?test_run=37&platform=1&opsys=32&end_date=20081222&limit=120]
* Mac OS X [PPC Mac] 10.4 (-who-) - 2008-12-??
* Mac OS X [PPC Mac] 10.4 (-who-) - 2008-12-?? [https://litmus.mozilla.org/advanced_search.cgi?test_run=37&platform=3&end_date=20081222&limit=120]
* Mac OS X [Intel Mac] 10.5 (-who-) - 2008-12-??
* Mac OS X [Intel Mac] 10.5 (-who-) - 2008-12-?? [https://litmus.mozilla.org/advanced_search.cgi?test_run=37&platform=13&end_date=20081222&limit=120]
* Linux (-who-) - 2008-12-??
* Linux x86_64 (henryne) - 2008-12-02 .de [https://litmus.mozilla.org/advanced_search.cgi?test_run=37&platform=2&end_date=20081222&limit=120]


'''BFTs'''
'''BFTs'''
* Windows XP SP2/Vista (-who-) - 2008-12-??
* Windows XP SP2 (-who-) - 2008-12-?? [https://litmus.mozilla.org/advanced_search.cgi?test_run=38&platform=1&opsys=4&end_date=20081222&limit=120]
* Windows XP Vista (-who-) - 2008-12-?? [https://litmus.mozilla.org/advanced_search.cgi?test_run=38&platform=1&opsys=32&end_date=20081222&limit=120]
* Mac OS X [PPC Mac] 10.4 (-who-) - 2008-12-?? [https://litmus.mozilla.org/advanced_search.cgi?test_run=38&platform=3&end_date=20081222&limit=120]
* Mac OS X [Intel Mac] 10.5 (-who-) - 2008-12-?? [https://litmus.mozilla.org/advanced_search.cgi?test_run=38&platform=13&end_date=20081222&limit=120]
* Linux i686,x86_64 (hhenkel,henryne) - 2008-12-02 [https://litmus.mozilla.org/advanced_search.cgi?test_run=38&platform=2&end_date=20081222&limit=120]


'''Spot checks'''
'''Spot checks'''
Line 97: Line 117:
* Mac OS X [Intel Mac] 10.5 - 2008-12-??
* Mac OS X [Intel Mac] 10.5 - 2008-12-??
* Linux (multiple) - 2008-12-??
* Linux (multiple) - 2008-12-??
'''locales:'''
* all PASS - except af which failed, and ja-JP which no one tested
'''crashes:'''
* newly found - {{bug|467840}}
* no topcrash/major worth fixing based on monitoring crash-stats the past week
* nothing major that wasn't fixed in build2
'''bugs found:'''
* 10 QAing candidate builds {{bug|463057}}, major issues fixed in build2
* [http://tinyurl.com/5sbqpo on trunk since 3.0b1 pull] - no show stoppers
'''bugs fixed:'''
* 8 fixed in build2 via {{bug|467813}}
'''postings:'''
* 7 Hendrix in mozilla.feedback.thunderbird.prerelease - no show stoppers
[https://litmus.mozilla.org/testday_report.cgi?start_timestamp=20081202000000&finish_timestamp=20081222235959&product=3&branch=21&testgroup=&build_id=&locale= litmus tests completed]


=== Testday ===
=== Testday ===
Line 126: Line 166:
* Bugs reported:  
* Bugs reported:  


Used wrong builds...
==== Windows XP Vista (-who-) ====
==== Windows XP Vista (-who-) ====
* Date: 2008-12-??
* Date: 2008-12-??
Line 140: Line 179:
* Results: '''????'''
* Results: '''????'''
* Bugs reported: ()
* Bugs reported: ()
==== Linux x86_64 (HenryNe) ====
* Date: 2008-12-03
* Build ID: version 3.0b1 (20081201162550)
* Test Type: Litmus Smoketests
* Results: '''PASS'''
* Locale: de
* Bugs reported: ([https://bugzilla.mozilla.org/show_bug.cgi?id=467841 467841] [https://bugzilla.mozilla.org/show_bug.cgi?id=467847 467847] [https://bugzilla.mozilla.org/show_bug.cgi?id=467860 467860])


=== BFT ===
=== BFT ===
Line 148: Line 195:
* Test Type: [https://litmus.mozilla.org/test_run_report.cgi?test_run_id=38 Litmus BFT]
* Test Type: [https://litmus.mozilla.org/test_run_report.cgi?test_run_id=38 Litmus BFT]
* Results: '''????'''
* Results: '''????'''
* Bugs reported:
==== Mac (maczilla) ====
* Date: 2008-12-05
* Build ID: version 3.0b1 (20081204113938)
* Test Type: [https://litmus.mozilla.org/test_run_report.cgi?test_run_id=38 Litmus BFT]
* Results: '''PASS''' [http://tinyurl.com/6b6nre partial test]
* Locale: de
* Bugs reported: {{bug|468056}} {{bug|468081}} {{bug|468083}} {{bug|468073}}


==== Linux (-who-) ====
==== Linux (-who-) ====
Line 158: Line 214:
* Issues Noted - Bugs filed:
* Issues Noted - Bugs filed:
** _
** _
(


=== Spot Check ===
=== Spot Check ===
Visually check accounts manager and addons, login to an account, send a message to ensure there is nothing vividly broken, and tinker around other parts of Thunderbird. To record your test results below: login to the wiki, click edit above at '''Thunderbird Spot Check''', copy one of the blocks below and paste a new one for yourself, and :
Visually check accounts manager and addons, login to an account, send a message to ensure nothing is obviously broken, tinker around other parts of Thunderbird. To record your test results below: login to the wiki, click edit above at '''Thunderbird Spot Check''', copy one of the blocks below and paste a new one for yourself, and :
* change PASS/FAILED to PASS or FAILED
* change PASS/FAILED to PASS or FAILED
* change '''-who-''' to your name
* change '''-who-''' to your name
Line 168: Line 226:
* Build ID: version 3.0b1 (200812????)
* Build ID: version 3.0b1 (200812????)
* Status: '''PASS/FAILED'''
* Status: '''PASS/FAILED'''
==== Spot Check Linux (-who-) ====
==== Spot Check Linux x86_64 (HenryNe) ====
* Build ID: version 3.0b1 (200812????)
* Build ID: version 3.0b1 (20081204114258, build2)
* Status: '''PASS/FAILED'''
* Status: '''PASS''' all languages, <br>unless locale 'af' '''FAIL''' (total broken, no pane, no menu)
==== Spot Check Linux (-who-) ====
 
* Build ID: version 3.0b1 (200812????)
* Status: '''PASS/FAILED'''
==== Spot Check OS X [PPC Mac] 10.4 (who) ====
==== Spot Check OS X [PPC Mac] 10.4 (who) ====
* Build ID: version 3.0b1 (200812????)
* Build ID: version 3.0b1 (200812????)

Latest revision as of 23:45, 11 December 2008

« Back to Thunderbird 3 Beta 1 Release Page | How you can help in Thunderbird QA

Thunderbird 3 Beta 1 Test Results

Testing will focus on smoketests and BFTs of en-US, and spotchecks all shipping locales.

Related Pages

Contact Info

Get announcements about the progress of this and future test events by subscribing to thunderbird-testers mailing list

Release Notes

Please read draft release notes before testing.

Additional information for testers:

Schedule

Builds handed to QA
2008-12-02
QA signoff
  • Smoketests - 2008-12-08
  • Spotchecks - 2008-12-08
  • BFTs - 2008-12-08

Build Location and IDs

Build 2

Build 2 incorporates bug fixes documented in Beta 1 respin bug 467813.

Please do not test using old builds (prior alphas, betas, nightlies). Download candidates builds from http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/3.0b1-candidates/build2

en-US build ids:

  • Windows - version 3.0b1 (20081204115318)
  • Linux - version 3.0b1 (20081204114258)
  • OS X [PPC Mac]: version 3.0b1 (20081204113938)
  • OS X [Intel Mac] - version 3.0b1 (20081204113938)

Build 1

(build1 is obsolete)

Please do not test using prior alphas, prior betas or a nightly. Download candidate builds at http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/3.0b1-candidates/build1

en-US build ids:

  • Windows - version 3.0b1 (20081201164237)
  • Linux - version 3.0b1 (20081201162550)
  • OS X [PPC Mac]: version 3.0b1 (20081201162550)
  • OS X [Intel Mac] - version 3.0b1 (20081201162550)

Test Coverage

Objectives:

  • smoketest - all 3 major platforms, and as many locales as possible
  • BFT - at least one platform (in "pieces", i.e. all not done by one person).
  • Areas Covered (en-US):
    • Spotchecks
    • Smoketests
    • BFTs - excluding themes, updates, addons
    • L10n, spotcheck all shipping locales
  • Areas Not Covered:
    • FFTs

Previous release (for reference purposes): Alpha 3 smoketested all 3 major platforms, full coverage of windows BFT, all shipping locales spot checked

QA Summary

  • Smoketests: 2008-12-02 - 2008-12-07
  • Windows XP/Vista BFTs: 2008-12-02 - 2008-12-07
  • Linux BFTs: 2008-12-02 - 2008-12-07
  • Mac BFTs: -
  • Release to World: 2008-12-??

Testing Results Summary

Smoketests

  • Windows XP SP1 (-who-) - 2008-12-?? [1]
  • Windows Vista SP1 (-who-) - 2008-12-?? [2]
  • Mac OS X [PPC Mac] 10.4 (-who-) - 2008-12-?? [3]
  • Mac OS X [Intel Mac] 10.5 (-who-) - 2008-12-?? [4]
  • Linux x86_64 (henryne) - 2008-12-02 .de [5]

BFTs

  • Windows XP SP2 (-who-) - 2008-12-?? [6]
  • Windows XP Vista (-who-) - 2008-12-?? [7]
  • Mac OS X [PPC Mac] 10.4 (-who-) - 2008-12-?? [8]
  • Mac OS X [Intel Mac] 10.5 (-who-) - 2008-12-?? [9]
  • Linux i686,x86_64 (hhenkel,henryne) - 2008-12-02 [10]

Spot checks

  • Windows Vista SP1 - 2008-12-??
  • Mac OS X [PPC Mac] 10.4 - 2008-12-??
  • Mac OS X [Intel Mac] 10.5 - 2008-12-??
  • Linux (multiple) - 2008-12-??

locales:

  • all PASS - except af which failed, and ja-JP which no one tested

crashes:

  • newly found - bug 467840
  • no topcrash/major worth fixing based on monitoring crash-stats the past week
  • nothing major that wasn't fixed in build2

bugs found:

bugs fixed:

postings:

  • 7 Hendrix in mozilla.feedback.thunderbird.prerelease - no show stoppers

litmus tests completed

Testday

A post release Thunderbird 3 Beta 1 testday has not yet been scheduled. A pre release Thunderbird 3 Beta 1 testday was held.

Detailed Results

Smoketests

Windows XP SP2 (-who-)

  • Date: 2008-12-??
  • Build ID: version 3.0b1 (200812????)
  • Test Type: Litmus Smoketests
  • Results: ????
  • Locale:
  • Bugs reported: ()

Mac OS X [PPC Mac] 10.4 (-who-)

  • Date: 2008-12-??
  • Build ID: version 3.0b1 (200812????)
  • Test Type: Litmus Smoketests
  • Results: ????
  • Bugs reported: bug nn

Mac OS X [Intel Mac] 10.5 (-who-)

  • Date: 2008-12-??
  • Build ID: version 3.0b1 (200812????)
  • Test Type: Litmus Smoketests
  • Results: ????
  • Bugs reported:

Windows XP Vista (-who-)

  • Date: 2008-12-??
  • Build ID: version 3.0b1 (200812????)
  • Test Type: Litmus Smoketests
  • Results: ????
  • Bugs reported: ()

Windows Vista SP2 (-who-)

  • Date: 2008-12-??
  • Build ID: version 3.0b1 (200812????)
  • Test Type: Litmus Smoketests
  • Results: ????
  • Bugs reported: ()

Linux x86_64 (HenryNe)

  • Date: 2008-12-03
  • Build ID: version 3.0b1 (20081201162550)
  • Test Type: Litmus Smoketests
  • Results: PASS
  • Locale: de
  • Bugs reported: (467841 467847 467860)

BFT

Windows XP SP2/Vista (wsmwk and others)

  • Date: 2008-08-08 2008-12-??
  • Build ID: version 3.0b1 (200812????)
  • Test Type: Litmus BFT
  • Results: ????
  • Bugs reported:

Mac (maczilla)

Linux (-who-)

  • Date: 2008-12-?? - 2008-12-??
  • Build ID: version 3.0b1 (200812????)
  • Test Type: Litmus BFT
  • Results: ????
  • Bugs reported:
  • Issues Noted - Bugs filed:
    • _

(

Spot Check

Visually check accounts manager and addons, login to an account, send a message to ensure nothing is obviously broken, tinker around other parts of Thunderbird. To record your test results below: login to the wiki, click edit above at Thunderbird Spot Check, copy one of the blocks below and paste a new one for yourself, and :

  • change PASS/FAILED to PASS or FAILED
  • change -who- to your name
  • change 200812???? to your buildid from help|about

Spot Check Vista SP1 (-who-)

  • Build ID: version 3.0b1 (200812????)
  • Status: PASS/FAILED

Spot Check Linux x86_64 (HenryNe)

  • Build ID: version 3.0b1 (20081204114258, build2)
  • Status: PASS all languages,
    unless locale 'af' FAIL (total broken, no pane, no menu)

Spot Check OS X [PPC Mac] 10.4 (who)

  • Build ID: version 3.0b1 (200812????)
  • Status: PASS/FAILED

Spot Check OS X [Intel Mac] 10.5 (who)

  • Build ID: version 3.0b1 (200812????)
  • Status: PASS/FAILED
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Thunderbird3/TestResults/Beta1&oldid=120455"