QA/Thunderbird3/TestResults/Beta1: Difference between revisions

From MozillaWiki
< QA
Jump to navigation Jump to search
m (→‎Testing Results Summary: add litmus test summary link)
(set one of changes wrapping up results)
Line 31: Line 31:
<!-- * [http://www.mozillamessaging.com/en-US/thunderbird/3.0b1/releasenotes/ Final Beta 1 Release Notes] -->
<!-- * [http://www.mozillamessaging.com/en-US/thunderbird/3.0b1/releasenotes/ Final Beta 1 Release Notes] -->


Additional information:  
Additional information for testers:  
* [[Thunderbird:QA_TestDay:2008-11-14#Focus|Some new features]], as documented in a previous testday for 3.0b1
* [[Thunderbird:QA_TestDay:2008-11-14#Focus|Some new features]], as documented in a previous testday for 3.0b1
* {{bug|463057}} bugs found during RC testing
* [http://tinyurl.com/5sbqpo all trunk bugs reported since 3.0b1 pull]
* [http://tinyurl.com/5ko8vf open trunk regressions, confirmed]
* [http://tinyurl.com/5ko8vf open trunk regressions, confirmed]
* [http://tinyurl.com/6mz5b9 open trunk regressions, unconfirmed]
* [http://tinyurl.com/6mz5b9 open trunk regressions, unconfirmed]
Line 43: Line 41:
;Builds handed to QA: 2008-12-02
;Builds handed to QA: 2008-12-02
;QA signoff:
;QA signoff:
* Smoketests - 2008-12-??
* Smoketests - 2008-12-08
* Spotchecks - 2008-12-??
* Spotchecks - 2008-12-08
* BFTs - 2008-12-??
* BFTs - 2008-12-08


== Build Location and IDs ==
== Build Location and IDs ==
Line 51: Line 49:
=== Build 2 ===
=== Build 2 ===


Build 2 incorporates fixes for the bugs documented in Beta 1 respin {{Bug|467813}}.
Build 2 incorporates bug fixes documented in Beta 1 respin {{Bug|467813}}.


Please do not test using old builds (prior alphas, betas, nightlies). Download release candidates from http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/3.0b1-candidates/build2
Please do not test using old builds (prior alphas, betas, nightlies). Download candidates builds from http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/3.0b1-candidates/build2


en-US build ids:
en-US build ids:
Line 65: Line 63:
<strike>(build1 is obsolete)
<strike>(build1 is obsolete)


Please do not test using prior alphas, prior betas or a nightly. Download release candidates at [http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/3.0b1-candidates/build1/ http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/3.0b1-candidates/build1]
Please do not test using prior alphas, prior betas or a nightly. Download candidate builds at [http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/3.0b1-candidates/build1/ http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/3.0b1-candidates/build1]


en-US build ids:
en-US build ids:
Line 78: Line 76:
Objectives:  
Objectives:  
* smoketest - all 3 major platforms, and as many locales as possible
* smoketest - all 3 major platforms, and as many locales as possible
* BFT - at least one platform (BFT is divided into pieces based on tester's last name to reduce time required of each person -- so all tests get done, but not necessarily all of a test by one person).
* BFT - at least one platform (in "pieces", i.e. all not done by one person).


* Areas Covered (en-US):
* Areas Covered (en-US):
** Spotchecks
** Spotchecks
** Smoketests  
** Smoketests  
** BFTs
** BFTs - excluding themes, updates, addons
** L10n, spotcheck all shipping locales
** L10n, spotcheck all shipping locales
* Areas Not Covered:
* Areas Not Covered:
** FFTs
** FFTs
** Updates


Previous release (for reference purposes): Alpha 3 smoketested all 3 major platforms, full coverage of windows BFT, all shipping locales were spot checked
Previous release (for reference purposes): Alpha 3 smoketested all 3 major platforms, full coverage of windows BFT, all shipping locales spot checked


=== QA Summary ===
=== QA Summary ===
* Smoketests: 2008-12-?? - 2008-12-??
* Smoketests: 2008-12-02 - 2008-12-07
* Windows XP/Vista BFTs: 2008-12-?? - 2008-12-??
* Windows XP/Vista BFTs: 2008-12-02 - 2008-12-07
* Linux BFTs: 2008-12-?? - 2008-12-??
* Linux BFTs: 2008-12-02 - 2008-12-07
* Mac BFTs: -
* Mac BFTs: -
<!-- * Spot check of Mac and Linux Platforms:: 2008-12-?? to 2008-12-?? -->
<!-- * Spot check of Mac and Linux Platforms:: 2008-12-?? to 2008-12-?? -->
Line 100: Line 97:


=== Testing Results Summary ===
=== Testing Results Summary ===
[https://litmus.mozilla.org/testday_report.cgi?start_timestamp=20081202000000&finish_timestamp=20081222235959&product=3&branch=21&testgroup=&build_id=&locale= litmus test summary]


'''Smoketests'''
'''Smoketests'''
Line 111: Line 106:


'''BFTs'''
'''BFTs'''
* Windows XP SP2/Vista (-who-) - 2008-12-??
* Windows XP SP2/Vista (henryne) - 2008-12-?? .de


'''Spot checks'''
'''Spot checks'''
Line 118: Line 113:
* Mac OS X [Intel Mac] 10.5 - 2008-12-??
* Mac OS X [Intel Mac] 10.5 - 2008-12-??
* Linux (multiple) - 2008-12-??
* Linux (multiple) - 2008-12-??
'''locales:'''
* all PASS - except af which failed, and ja-JP which no one tested
'''crashes:'''
* newly found - {{bug|467840}}
* no topcrash/major worth fixing based on monitoring crash-stats the past week
* nothing major that wasn't fixed in build2
'''bugs found:'''
* 10 QAing candidate builds {{bug|463057}}, major issues fixed in build2
* [http://tinyurl.com/5sbqpo on trunk since 3.0b1 pull] - no show stoppers
'''bugs fixed:'''
* 8 fixed in build2 via {{bug|467813}}
'''postings:'''
* 7 Hendrix in mozilla.feedback.thunderbird.prerelease - no show stoppers
[https://litmus.mozilla.org/testday_report.cgi?start_timestamp=20081202000000&finish_timestamp=20081222235959&product=3&branch=21&testgroup=&build_id=&locale= litmus tests completed]


=== Testday ===
=== Testday ===
Line 147: Line 162:
* Bugs reported:  
* Bugs reported:  


Used wrong builds...
==== Windows XP Vista (-who-) ====
==== Windows XP Vista (-who-) ====
* Date: 2008-12-??
* Date: 2008-12-??
Line 181: Line 195:


=== Spot Check ===
=== Spot Check ===
Visually check accounts manager and addons, login to an account, send a message to ensure there is nothing vividly broken, and tinker around other parts of Thunderbird. To record your test results below: login to the wiki, click edit above at '''Thunderbird Spot Check''', copy one of the blocks below and paste a new one for yourself, and :
Visually check accounts manager and addons, login to an account, send a message to ensure nothing is obviously broken, tinker around other parts of Thunderbird. To record your test results below: login to the wiki, click edit above at '''Thunderbird Spot Check''', copy one of the blocks below and paste a new one for yourself, and :
* change PASS/FAILED to PASS or FAILED
* change PASS/FAILED to PASS or FAILED
* change '''-who-''' to your name
* change '''-who-''' to your name
Line 187: Line 201:


==== Spot Check Vista SP1 (-who-) ====
==== Spot Check Vista SP1 (-who-) ====
* Build ID: version 3.0b1 (200812????)
* Status: '''PASS/FAILED'''
==== Spot Check Linux (-who-) ====
* Build ID: version 3.0b1 (200812????)
* Build ID: version 3.0b1 (200812????)
* Status: '''PASS/FAILED'''
* Status: '''PASS/FAILED'''

Revision as of 18:51, 8 December 2008

« Back to Thunderbird 3 Beta 1 Release Page | How you can help in Thunderbird QA

Thunderbird 3 Beta 1 Test Results

Testing will focus on smoketests and BFTs of en-US, and spotchecks all shipping locales.

Related Pages

Contact Info

Get announcements about the progress of this and future test events by subscribing to thunderbird-testers mailing list

Release Notes

Please read draft release notes before testing.

Additional information for testers:

Schedule

Builds handed to QA
2008-12-02
QA signoff
  • Smoketests - 2008-12-08
  • Spotchecks - 2008-12-08
  • BFTs - 2008-12-08

Build Location and IDs

Build 2

Build 2 incorporates bug fixes documented in Beta 1 respin bug 467813.

Please do not test using old builds (prior alphas, betas, nightlies). Download candidates builds from http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/3.0b1-candidates/build2

en-US build ids:

  • Windows - version 3.0b1 (20081204115318)
  • Linux - version 3.0b1 (20081204114258)
  • OS X [PPC Mac]: version 3.0b1 (20081204113938)
  • OS X [Intel Mac] - version 3.0b1 (20081204113938)

Build 1

(build1 is obsolete)

Please do not test using prior alphas, prior betas or a nightly. Download candidate builds at http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/3.0b1-candidates/build1

en-US build ids:

  • Windows - version 3.0b1 (20081201164237)
  • Linux - version 3.0b1 (20081201162550)
  • OS X [PPC Mac]: version 3.0b1 (20081201162550)
  • OS X [Intel Mac] - version 3.0b1 (20081201162550)

Test Coverage

Objectives:

  • smoketest - all 3 major platforms, and as many locales as possible
  • BFT - at least one platform (in "pieces", i.e. all not done by one person).
  • Areas Covered (en-US):
    • Spotchecks
    • Smoketests
    • BFTs - excluding themes, updates, addons
    • L10n, spotcheck all shipping locales
  • Areas Not Covered:
    • FFTs

Previous release (for reference purposes): Alpha 3 smoketested all 3 major platforms, full coverage of windows BFT, all shipping locales spot checked

QA Summary

  • Smoketests: 2008-12-02 - 2008-12-07
  • Windows XP/Vista BFTs: 2008-12-02 - 2008-12-07
  • Linux BFTs: 2008-12-02 - 2008-12-07
  • Mac BFTs: -
  • Release to World: 2008-12-??

Testing Results Summary

Smoketests

  • Windows XP SP1 (-who-) - 2008-12-??
  • Windows Vista SP1 (-who-) - 2008-12-??
  • Mac OS X [PPC Mac] 10.4 (-who-) - 2008-12-??
  • Mac OS X [Intel Mac] 10.5 (-who-) - 2008-12-??
  • Linux (-who-) - 2008-12-??

BFTs

  • Windows XP SP2/Vista (henryne) - 2008-12-?? .de

Spot checks

  • Windows Vista SP1 - 2008-12-??
  • Mac OS X [PPC Mac] 10.4 - 2008-12-??
  • Mac OS X [Intel Mac] 10.5 - 2008-12-??
  • Linux (multiple) - 2008-12-??

locales:

  • all PASS - except af which failed, and ja-JP which no one tested

crashes:

  • newly found - bug 467840
  • no topcrash/major worth fixing based on monitoring crash-stats the past week
  • nothing major that wasn't fixed in build2

bugs found:

bugs fixed:

postings:

  • 7 Hendrix in mozilla.feedback.thunderbird.prerelease - no show stoppers

litmus tests completed

Testday

A post release Thunderbird 3 Beta 1 testday has not yet been scheduled. A pre release Thunderbird 3 Beta 1 testday was held.

Detailed Results

Smoketests

Windows XP SP2 (-who-)

  • Date: 2008-12-??
  • Build ID: version 3.0b1 (200812????)
  • Test Type: Litmus Smoketests
  • Results: ????
  • Locale:
  • Bugs reported: ()

Mac OS X [PPC Mac] 10.4 (-who-)

  • Date: 2008-12-??
  • Build ID: version 3.0b1 (200812????)
  • Test Type: Litmus Smoketests
  • Results: ????
  • Bugs reported: bug nn

Mac OS X [Intel Mac] 10.5 (-who-)

  • Date: 2008-12-??
  • Build ID: version 3.0b1 (200812????)
  • Test Type: Litmus Smoketests
  • Results: ????
  • Bugs reported:

Windows XP Vista (-who-)

  • Date: 2008-12-??
  • Build ID: version 3.0b1 (200812????)
  • Test Type: Litmus Smoketests
  • Results: ????
  • Bugs reported: ()

Windows Vista SP2 (-who-)

  • Date: 2008-12-??
  • Build ID: version 3.0b1 (200812????)
  • Test Type: Litmus Smoketests
  • Results: ????
  • Bugs reported: ()

BFT

Windows XP SP2/Vista (wsmwk and others)

  • Date: 2008-08-08 2008-12-??
  • Build ID: version 3.0b1 (200812????)
  • Test Type: Litmus BFT
  • Results: ????

Linux (-who-)

  • Date: 2008-12-?? - 2008-12-??
  • Build ID: version 3.0b1 (200812????)
  • Test Type: Litmus BFT
  • Results: ????
  • Bugs reported:
  • Issues Noted - Bugs filed:
    • _

Spot Check

Visually check accounts manager and addons, login to an account, send a message to ensure nothing is obviously broken, tinker around other parts of Thunderbird. To record your test results below: login to the wiki, click edit above at Thunderbird Spot Check, copy one of the blocks below and paste a new one for yourself, and :

  • change PASS/FAILED to PASS or FAILED
  • change -who- to your name
  • change 200812???? to your buildid from help|about

Spot Check Vista SP1 (-who-)

  • Build ID: version 3.0b1 (200812????)
  • Status: PASS/FAILED

Spot Check Linux (-who-)

  • Build ID: version 3.0b1 (200812????)
  • Status: PASS/FAILED

Spot Check OS X [PPC Mac] 10.4 (who)

  • Build ID: version 3.0b1 (200812????)
  • Status: PASS/FAILED

Spot Check OS X [Intel Mac] 10.5 (who)

  • Build ID: version 3.0b1 (200812????)
  • Status: PASS/FAILED
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Thunderbird3/TestResults/Beta1&oldid=119835"