QA/Thunderbird3/TestResults/Alpha2: Difference between revisions

From MozillaWiki
< QA
Jump to navigation Jump to search
(New page: <small>« Back to Thunderbird 3 Alpha 2 Release Page | Thunderbird Testing</small> = Thunderbird 3 Alpha 2 Test Results = == S...)
 
 
(19 intermediate revisions by 3 users not shown)
Line 1: Line 1:
<small>[[Releases/Thunderbird_3.0a2|&laquo; Back to Thunderbird 3 Alpha 2 Release Page]] | [[Thunderbird:Testing|Thunderbird Testing]]</small>
<small>[[Releases/Thunderbird_3.0a2|&laquo; Back to Thunderbird 3 Alpha 2 Release Page]] | [[Thunderbird:Testing|How you can help in Thunderbird QA]]</small>


= Thunderbird 3 Alpha 2 Test Results =
= Thunderbird 3 Alpha 2 Test Results =
Testing for this release will focus on smoketests, BFTs, and spotchecks of en-US and selected locales.
== Related Pages ==
[[Thunderbird:Testing|How you can help in Thunderbird QA]]
[[Releases/Thunderbird_3.0a2|Alpha 2 Release Tracking Page]]
Links to key test results.  More are referenced further below.
<!-- * [http://wiki.mozilla.org/QA/Firefox3/TestResults/RC3/Updates Update snippets Test Results] -->
* [[Thunderbird3/TestResults/Alpha2/L10N|L10n Spotchecks Test Results]]
* [[Thunderbird3/TestResults/Alpha2/Testday|Alpha2 Testday Results]]
== Contact Info ==
QA: Wayne Mery, wsmwk in #maildev channel on irc.mozilla.org or post in newsgroup mozilla.support.thunderbird with subject Shredder Alpha 2
Testers: Can meet each other in #testday channel on irc.mozilla.org
[[Thunderbird:Bugdays#Where|irc instructions]]


== Schedule ==  
== Schedule ==  
;Code Freeze: 08 July 2008 2359h PDT
;Builds hand to QA: 08/04
;Builds: __ July 2008 - __ July 2008
;QA signoff:
;QA/Testing: __ July 2008 - __ July 2008
* Smoketests - 2008-08-11
;Final Release: __ July 2008
* Spotchecks - 2008-08-11
* BFTs - 2008-08-11
&nbsp; initial ETA 8/08<br>
&nbsp; Gave verbal PASS to DavidA ~4pm EST Friday 2008-08-08 (except Mac smoketest yet to complete)


== Release Notes ==
== Release Notes ==
See [[QA/Thunderbird3/TestResults/Alpha2/Release_Notes|list of release notes here]].
Release notes will be based on:


== Test Plan ==
* [http://www.mozillamessaging.com/en-US/thunderbird/3.0a1/releasenotes/ Alpha 1 Release Notes]
* {{Bug|443369}} - Prepare Thunderbird 3 Alpha 2 release notes
* [http://tinyurl.com/6eh2or Bug query approximating fixes in Alpha 2]  (bugs resolved as fixed between 2008-04-24 and 2008-07-08)


=== QA Details ===
== Build Location and IDs ==
* Smoketests: QA+0
=== Build 1 ===
* Windows XP BFTs: 2008-07-??
* Spot check of Mac and Linux Platforms:: 2008-07-?? to 2008-07-??
* Release to World: 2008-07-??


=== Details ===
* [http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/3.0a2-candidates/build1/ http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/3.0a2-candidates/build1/]
** '''Windows''': version 3.0a2 (2008072418)
** '''Linux''': version 3.0a2 (2008072418)
** '''OS X [PPC Mac]''': version 3.0a2 (200807????)
** '''OS X [Intel Mac]''': version 3.0a2 (2008072822)


* [http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/3.0a2-candidates/TBD Build 1]
== Test Coverage ==
Overview: smoketest all 3 major platforms. Alpha 1 did a partial BFT. So goal is to BFT all 3 platforms to get BFTs in better shape for beta testing. Divide BFT into pieces to reduce time required per person (IOW soo the full range of tests is accomplished, but not necessarily be a single OS). Check mozilla.feedback.thunderbird.prerelease and mozilla.feedback for additional result comments. [no comments have been found in there nor in m.support.thunderbird]


=== Testing Results ===
* Areas Covered (en-US):
** Spotchecks
** Smoketests
** BFTs
** L10n, spotcheck selected locales
* Areas Not Covered:
** FFTs
** Updates
** L10n
 
=== QA Summary ===
* Smoketests: 2008-08-05 - 2008-08-09
* Windows XP/Vista BFTs: 2008-08-05 - 2008-08-11
* Linux BFTs: 2008-08-05 - 2008-08-09
* Mac BFTs: -
<!-- * Spot check of Mac and Linux Platforms:: 2008-08-?? to 2008-08-?? -->
* Release to World: 2008-08-??
 
=== Testing Results Summary ===


'''Smoketests'''
'''Smoketests'''
* Windows XP SP2 (who) - 2008-07-??
* Windows XP SP1 (nshopik) - 2008-08-06
* Windows Vista SP1 (who) - 2008-07-??
* Windows Vista SP1 (simon.johansson) - 2008-08-06
* Mac OS X [PPC Mac] 10.4 (who) - 2008-07-??
* Mac OS X [PPC Mac] 10.4 (who) - 2008-08-??
* Mac OS X [Intel Mac] 10.5 (who) - 2008-07-??
* Mac OS X [Intel Mac] 10.5 (nth10sd) - 2008-08-08
* Linux (who) - 2008-07-??
* Linux (multiple) - 2008-08-??
 
'''BFTs'''
'''BFTs'''
* Windows XP SP2 (who) - 2008-07-??
* Windows XP SP2/Vista (wsmwk) - 2008-08-11
 
'''Spot checks'''
'''Spot checks'''
* Windows Vista SP1 (who) - 2008-07-??
* Windows Vista SP1 - 2008-08-??
* Mac OS X [PPC Mac] 10.4 (who) - 2008-07-??
* Mac OS X [PPC Mac] 10.4 - 2008-08-??
* Mac OS X [Intel Mac] 10.5 (who) - 2008-07-??
* Mac OS X [Intel Mac] 10.5 - 2008-08-??
* Linux (who) - 2008-07-??
* Linux (multiple) - 2008-08-10


=== Testday ===
=== Testday ===
A Thunderbird 3 Alpha 2 testday will be held and call for the community support to assist with testing.
A Thunderbird 3 Alpha 2 testday may be held and a call for community support to assist with testing.
 
[TBD Testday results] are posted after the testday is held.
 


== <FONT COLOR = "RED">Detailed Results </FONT> ==
== <FONT COLOR = "RED">Detailed Results </FONT> ==


=== Smoketests ===


=== Thunderbird Smoketests ===
==== Windows XP SP2 (nshopik) ====
'''Build IDs'''
* Date: 2008-08-06
* '''Windows''': version 3.0a2 (200807????)
* Build ID: version 3.0a2 (? 2008070208)
* '''Linux''': version 3.0a2 (200807????)
* '''OS X [PPC Mac]''': version 3.0a2 (200807????)
* '''OS X [Intel Mac]''': version 3.0a2 (200807????)
 
==== Windows XP SP2 (who) ====
* Date: 2008-07-??
* Build ID: version 3.0a2 (200807????)
* Test Type: Litmus Smoketests
* Results: '''????'''
* Notes: See PPC Mac notes for tests not run.
** Bugs reported: ()
 
==== Windows Vista SP1 (who) ====
* Date: 2008-07-??
* Build ID: version 3.0a2 (200807????)
* Test Type: Litmus Smoketests
* Test Type: Litmus Smoketests
* Results: '''????'''
* Results: '''PASS'''
* Notes: See PPC Mac notes for tests not run.
* Locale: ru
* Bugs reported: ()
* Bugs reported: ()
** {{bug|nn}} - xx
==== Mac OS X [PPC Mac] 10.4 (who) ====
==== Mac OS X [PPC Mac] 10.4 (who) ====
* Date: 2008-07-??
* Date: 2008-08-??
* Build ID: version 3.0a2 (200807????)
* Build ID: version 3.0a2 (200807????)
* Test Type: Litmus Smoketests
* Test Type: Litmus Smoketests
* Results: '''????'''
* Results: '''????'''
* Notes:
* Bugs reported: ()
* Bugs reported: ()
==== Mac OS X [Intel Mac] 10.5 (nth10sd) ====
* Date: 2008-08-08
* Build ID: version 3.0a2 (2008072822)
* Test Type: Litmus Smoketests
* Results: '''PASS'''
* Bugs reported: {{bug|450038}} Help doesn't redirect to locale, {{bug|450109}} File | Print fails from a standalone message window -- option is blanked out, not selectable, {{bug|450113}} sends via the non-default account until i delete all other smtp accounts except the intended one


==== Mac OS X [Intel Mac] 10.5 (who) ====
Used wrong builds...
* Date: 2008-07-??
==== Windows XP Vista (simon.johansson) ====
* Build ID: version 3.0a2 (200807????)
* Date: 2008-08-06
* Build ID: version 3.0a2 (? 2008080600)
* Test Type: Litmus Smoketests
* Test Type: Litmus Smoketests
* Results: '''????'''
* Results: '''????'''
* Notes: See PPC Mac notes for tests not run.
* Bugs reported: ()
* Bugs reported: ()
*
 
==== Linux (who) ====
==== Windows Vista SP2 (neander) ====
* Date: 2008-05-??
* Date: 2008-08-06
* Build ID: version 3.0a2 (200807????)
* Build ID: version 3.0a2 (? 2008070208)
* Test Type: Litmus Smoketests
* Test Type: Litmus Smoketests
* Results: '''????'''
* Results: '''????'''
* Notes: See PPC Mac notes for tests not run.
* Bugs reported: ()
* Bugs reported: ()


=== Thunderbird BFT (only one platform) ===
=== BFT ===
==== Windows XP SP2 (who) ====
 
* Date: 2008-07-??
==== Windows XP SP2/Vista (wsmwk and others) ====
* Build ID: version 3.0a2 (200807????)
* Date: 2008-08-08 2008-08-11
* Test Type: Litmus BFT
* Build ID: version 3.0a2 (2008072418)
* Results: '''????'''
* Test Type: [https://litmus.mozilla.org/test_run_report.cgi?test_run_id=29 Litmus BFT]
* Results: '''PASS'''
* Untested
** [https://litmus.mozilla.org/show_test.cgi?id=5694 Test 5694] Auto Proxy Connection
** unimportant or less important to alpha): Extensions, themes, software update
* Bugs reported: bug [pending] Outlook not offered as import option [testresult 146563], {{bug|404609}} [already on file] installer fails to run without administrator - lacks manifest [testresult 146402], {{bug|449759}} crash @ nsTreeSelection::GetSingle(int*) [testresult 146278], bug [pending+unconfirmed] inline splellchecker fails in html compose [testcase 146005]


* The BFT (adapted from Thunderbird 2.0 BFTs since none exist yet for Thunderbird trunk) will be selectively done by who focusing on the selected areas:
==== Linux (nestler, nigg, ms07, miquelmartin, badgerfiles) ====
** Install, shutdown, uninstall - '''who'''
* Date: 2008-08-05 - 2008-08-09
*** Status: '''????'''
* Build ID: version 3.0a2 (2008072418)
** IMAP accounts - '''who'''
* Test Type: [https://litmus.mozilla.org/advanced_search.cgi?test_run=&product=3&branch=21&testgroup=93&subgroup=&testcase=&platform=2&opsys=&locale=&result_status=&start_date=20080804000000&timespan=&end_date=20080811235959&search_field1=&match_criteria1=contains_all&search_value1=&search_field2=&match_criteria2=contains_all&search_value2=&search_field3=&match_criteria3=contains_all&search_value3=&search_field4=&match_criteria4=contains_all&search_value4=&sort_field1=&sort_order1=ASC&sort_field2=&sort_order2=ASC&sort_field3=&sort_order3=ASC&sort_field4=&sort_order4=ASC&limit=999&automated=all&withbugs=all&email=&trusted_only=all&vetted_only=all&valid_only=1 Litmus BFT]
*** Status: '''''' (see issues below)
* Results: '''PASS'''
* Bugs reported:
* Bugs pending: bug??? Breakpad "Enabled=1" is missing [https://litmus.mozilla.org/single_result.cgi?id=146798 testresult 146798], [unconfirmed] Contacts button is missing [testresult 146200], bug ?? "Show the article summary instead..." has no effect. The RSS is all times showed as HTML webpage [testresult 146507], bug??? RSS subscription drops from drag are rejected [testresult 146509], bug???? Unable to add downoad actions via Options -> Attachments -> Download Action [https://litmus.mozilla.org/single_result.cgi?id=146526 testresult 146526], bug??? HTML email does not send as HTML [https://litmus.mozilla.org/single_result.cgi?id=146528 testcase 146528], bug??? phishing detection https://litmus.mozilla.org/single_result.cgi?id=146545 testresult 146545]
* Bugs existing: {{Bug|449482}} Changing draft folder from an Imap folder to a local folder has no effect [testresult 146184]


* Issues Noted:
* Issues Noted - Bugs filed:
** _
** _


=== Spot Check Vista SP1 (who) ===
=== Spot Check ===
* Build ID: version 3.0a1 (200807????)
Visually check accounts manager and addons, login to an account, send a message to ensure there is nothing vividly broken, and tinker around other parts of Thunderbird. To record your test results below: login to the wiki, click edit above at '''Thunderbird Spot Check''', copy one of the blocks below and paste a new one for yourself, and :
* Status: '''????'''
* change PASS/FAILED to PASS or FAILED
* change '''who''' to your name
* change '''200807????''' to your buildid from help|about


=== Spot Check Linux (who) ===
==== Spot Check Vista SP1 (who) ====
* Build ID: version 3.0a1 (200807????)
* Build ID: version 3.0a2 (200807????)
* Status: '''????'''
* Status: '''PASS/FAILED'''
 
==== Spot Check Linux (who) ====
=== Spot Check OS X [PPC Mac] 10.4 (who) ===
* Build ID: version 3.0a2 (200807????)
* Build ID: version 3.0a1 (200807????)
* Status: '''PASS/FAILED'''
* Status: ''''''
==== Spot Check Linux (kquiggle) ====
 
* Build ID: version 3.0a2 (200807????)
=== Spot Check OS X [Intel Mac] 10.5 (who) ===
* Status: '''PASS'''
* Build ID: version 3.0a1 (200807????)
==== Spot Check OS X [PPC Mac] 10.4 (who) ====
* Status: '''????'''
* Build ID: version 3.0a2 (200807????)
* Status: '''PASS/FAILED'''
==== Spot Check OS X [Intel Mac] 10.5 (who) ====
* Build ID: version 3.0a2 (200807????)
* Status: '''PASS/FAILED'''

Latest revision as of 18:30, 11 August 2008

« Back to Thunderbird 3 Alpha 2 Release Page | How you can help in Thunderbird QA

Thunderbird 3 Alpha 2 Test Results

Testing for this release will focus on smoketests, BFTs, and spotchecks of en-US and selected locales.

Related Pages

How you can help in Thunderbird QA

Alpha 2 Release Tracking Page

Links to key test results. More are referenced further below.

Contact Info

QA: Wayne Mery, wsmwk in #maildev channel on irc.mozilla.org or post in newsgroup mozilla.support.thunderbird with subject Shredder Alpha 2

Testers: Can meet each other in #testday channel on irc.mozilla.org

irc instructions

Schedule

Builds hand to QA
08/04
QA signoff
  • Smoketests - 2008-08-11
  • Spotchecks - 2008-08-11
  • BFTs - 2008-08-11

  initial ETA 8/08
  Gave verbal PASS to DavidA ~4pm EST Friday 2008-08-08 (except Mac smoketest yet to complete)

Release Notes

Release notes will be based on:

Build Location and IDs

Build 1

Test Coverage

Overview: smoketest all 3 major platforms. Alpha 1 did a partial BFT. So goal is to BFT all 3 platforms to get BFTs in better shape for beta testing. Divide BFT into pieces to reduce time required per person (IOW soo the full range of tests is accomplished, but not necessarily be a single OS). Check mozilla.feedback.thunderbird.prerelease and mozilla.feedback for additional result comments. [no comments have been found in there nor in m.support.thunderbird]

  • Areas Covered (en-US):
    • Spotchecks
    • Smoketests
    • BFTs
    • L10n, spotcheck selected locales
  • Areas Not Covered:
    • FFTs
    • Updates
    • L10n

QA Summary

  • Smoketests: 2008-08-05 - 2008-08-09
  • Windows XP/Vista BFTs: 2008-08-05 - 2008-08-11
  • Linux BFTs: 2008-08-05 - 2008-08-09
  • Mac BFTs: -
  • Release to World: 2008-08-??

Testing Results Summary

Smoketests

  • Windows XP SP1 (nshopik) - 2008-08-06
  • Windows Vista SP1 (simon.johansson) - 2008-08-06
  • Mac OS X [PPC Mac] 10.4 (who) - 2008-08-??
  • Mac OS X [Intel Mac] 10.5 (nth10sd) - 2008-08-08
  • Linux (multiple) - 2008-08-??

BFTs

  • Windows XP SP2/Vista (wsmwk) - 2008-08-11

Spot checks

  • Windows Vista SP1 - 2008-08-??
  • Mac OS X [PPC Mac] 10.4 - 2008-08-??
  • Mac OS X [Intel Mac] 10.5 - 2008-08-??
  • Linux (multiple) - 2008-08-10

Testday

A Thunderbird 3 Alpha 2 testday may be held and a call for community support to assist with testing.

Detailed Results

Smoketests

Windows XP SP2 (nshopik)

  • Date: 2008-08-06
  • Build ID: version 3.0a2 (? 2008070208)
  • Test Type: Litmus Smoketests
  • Results: PASS
  • Locale: ru
  • Bugs reported: ()

Mac OS X [PPC Mac] 10.4 (who)

  • Date: 2008-08-??
  • Build ID: version 3.0a2 (200807????)
  • Test Type: Litmus Smoketests
  • Results: ????
  • Bugs reported: ()

Mac OS X [Intel Mac] 10.5 (nth10sd)

  • Date: 2008-08-08
  • Build ID: version 3.0a2 (2008072822)
  • Test Type: Litmus Smoketests
  • Results: PASS
  • Bugs reported: bug 450038 Help doesn't redirect to locale, bug 450109 File | Print fails from a standalone message window -- option is blanked out, not selectable, bug 450113 sends via the non-default account until i delete all other smtp accounts except the intended one

Used wrong builds...

Windows XP Vista (simon.johansson)

  • Date: 2008-08-06
  • Build ID: version 3.0a2 (? 2008080600)
  • Test Type: Litmus Smoketests
  • Results: ????
  • Bugs reported: ()

Windows Vista SP2 (neander)

  • Date: 2008-08-06
  • Build ID: version 3.0a2 (? 2008070208)
  • Test Type: Litmus Smoketests
  • Results: ????
  • Bugs reported: ()

BFT

Windows XP SP2/Vista (wsmwk and others)

  • Date: 2008-08-08 2008-08-11
  • Build ID: version 3.0a2 (2008072418)
  • Test Type: Litmus BFT
  • Results: PASS
  • Untested
    • Test 5694 Auto Proxy Connection
    • unimportant or less important to alpha): Extensions, themes, software update
  • Bugs reported: bug [pending] Outlook not offered as import option [testresult 146563], bug 404609 [already on file] installer fails to run without administrator - lacks manifest [testresult 146402], bug 449759 crash @ nsTreeSelection::GetSingle(int*) [testresult 146278], bug [pending+unconfirmed] inline splellchecker fails in html compose [testcase 146005]

Linux (nestler, nigg, ms07, miquelmartin, badgerfiles)

  • Date: 2008-08-05 - 2008-08-09
  • Build ID: version 3.0a2 (2008072418)
  • Test Type: Litmus BFT
  • Results: PASS
  • Bugs reported:
  • Bugs pending: bug??? Breakpad "Enabled=1" is missing testresult 146798, [unconfirmed] Contacts button is missing [testresult 146200], bug ?? "Show the article summary instead..." has no effect. The RSS is all times showed as HTML webpage [testresult 146507], bug??? RSS subscription drops from drag are rejected [testresult 146509], bug???? Unable to add downoad actions via Options -> Attachments -> Download Action testresult 146526, bug??? HTML email does not send as HTML testcase 146528, bug??? phishing detection https://litmus.mozilla.org/single_result.cgi?id=146545 testresult 146545]
  • Bugs existing: bug 449482 Changing draft folder from an Imap folder to a local folder has no effect [testresult 146184]
  • Issues Noted - Bugs filed:
    • _

Spot Check

Visually check accounts manager and addons, login to an account, send a message to ensure there is nothing vividly broken, and tinker around other parts of Thunderbird. To record your test results below: login to the wiki, click edit above at Thunderbird Spot Check, copy one of the blocks below and paste a new one for yourself, and :

  • change PASS/FAILED to PASS or FAILED
  • change who to your name
  • change 200807???? to your buildid from help|about

Spot Check Vista SP1 (who)

  • Build ID: version 3.0a2 (200807????)
  • Status: PASS/FAILED

Spot Check Linux (who)

  • Build ID: version 3.0a2 (200807????)
  • Status: PASS/FAILED

Spot Check Linux (kquiggle)

  • Build ID: version 3.0a2 (200807????)
  • Status: PASS

Spot Check OS X [PPC Mac] 10.4 (who)

  • Build ID: version 3.0a2 (200807????)
  • Status: PASS/FAILED

Spot Check OS X [Intel Mac] 10.5 (who)

  • Build ID: version 3.0a2 (200807????)
  • Status: PASS/FAILED
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Thunderbird3/TestResults/Alpha2&oldid=103567"