Firefox:1.5.0.8:Test Plan: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(33 intermediate revisions by 2 users not shown)
Line 12: Line 12:
* Bug triage
* Bug triage
* Test plan completion
* Test plan completion
* Minor/Major Software Updates discussions and testing
* Bug verification
* Bug verification
* Tuesday, October 24 - Smoketests - tracy - '''PASS'''


== Schedule ==
== Schedule ==


=== Monday, August 28th ===
=== Tuesday, October 17 ===
* RC1 delayed till Monday night/Tuesday morning. Shifting Monday's planned activities to Tuesday
* Start bug verifications for bugs marked fixed1.8.0.8
* Post to QA blog for Community Testing ?


=== Tuesday, August 29th ===
=== Friday, October 20 ===
* Firefox builds should be available for Win/Linux/PPC
* Continue bug verifications
* Smoketests (rc2) on Windows, Linux, Mac en-us
* Start testing minor/major softare updates with mock-ups provided by Seth
** [[Firefox:1.5.0.7:Smoketests|1.5.0.7 Firefox and Thunderbird Smoketest results]]


=== Wednesday, August 30th ===
=== Tuesday, October 24 ===
* Firefox BFTs - PASS
* FF 1.5.0.8rc1 builds ready
** [[Firefox:1.5.0.7:BFTs|1.5.0.7 Firefox and Thunderbird BFT results]]
* Firefox Smoketets on all platforms - tracy - '''PASS'''
* Build should begin l10n packaging after BFTs signoff (Rhelmer/TR)
* L10n builds ready to be tested
* Get post up on the QA blog with link to candidate builds
* BFT's started [[Firefox:1.5.0.8:Test_Plan:BFT | BFT Results]]
* Testing of QA identified hot spots
* Targeted exploratory testing over Developer identified hot spots


=== Thursday, August 31st ===
=== Wednesday, October 25 ===
* Continued testing of developer identified hot spots and bug verifications
*Smoketest and BFTs for Thunderbird
*Start Firefox L10 Testing
*Continue Verifications
*Minor/Major Software Updates Testing


=== Friday, September 1st ===
=== Thursday, October 26 ===
* Community Testday for 1.5.0.7 RC builds
'''If not ahead of schedule:'''
* Smoketests (rc2) on Windows, Linux, Mac en-us
*Smoketest and BFTs for Thunderbird
** [[Firefox:1.5.0.7:Smoketests|1.5.0.7 Firefox and Thunderbird Smoketest results]]
*Start Firefox L10 Testing
* Thunderbird BFTs - PASS
*Continue Verifications
** [[Firefox:1.5.0.7:BFTs|1.5.0.7 Firefox and Thunderbird BFT results]]
*Minor/Major Software Updates Testing
* Bug verifications and focus testing


=== Tuesday, September 5th ===
=== Friday, October 27 - Friday, November 3 ===
* Further bug verifications and testing of hot spots
* Start [http://wiki.mozilla.org/Firefox:1.5.0.8:Test_Plan:l10n l10n testing] for both Firefox and Thunderbird
* Thunderbird Smoketests
* Testing of QA identified hot spots: [http://wiki.mozilla.org/Firefox:1.5.0.8:Test_Plan:Software_Update Software Updates]
* Start Thunderbird BFTs
* [[Firefox:1.5.0.8:Test_Plan:Minor Major Software Update | Minor-Major Software Update Testing]]
* L10n + Updates testing
* Community Test Day


=== Wednesday, September 6th ===
=== Monday, November 6th ===
* Firefox RC4 will be released this afternoon
* End-to-end Major update testing for 1.5.0.8 -> 2.0
* Smoketest Firefox RC4
* Wrap up bug verifications
** [[Firefox:1.5.0.7:Smoketests|1.5.0.7 Firefox Smoketest results]]
* Final l10n + update testing on staging server.
* Thunderbird bug verifications


=== Thursday, September 7th ===
=== Tuesday, November 7th ===
* Respins for rc5 out late Sept. 6
* Firefox 1.5.0.8 is released, probably 4pm
* Smoketest rc5 builds
* Push website changes and security notes
** [[Firefox:1.5.0.7:Smoketests|1.5.0.7 Firefox Smoketest results]]
* Spot check live updates and download links
* Firefox security testing for RSA/Certificate changes
** https://bugzilla.mozilla.org/show_bug.cgi?id=340198
** https://bugzilla.mozilla.org/show_bug.cgi?id=351079
** https://bugzilla.mozilla.org/show_bug.cgi?id=350640
* Thunderbird bug verifications


=== Friday, September 8th ===
=== Wednesday, November 8 - Monday, November 13 ===
* Spot check P1 locales
* Start special distro spotchecks, and continue until Monday, Nov 13
** [[Firefox:1.5.0.7:LocalizationTesting|1.5.0.7 Firefox and Thunderbird Localization testing results]]
** Check other tier 2 locales as time permits
**[[Firefox:1.5.0.7:Extension Testing|1.5.0.7 Firefox and Thunderbird Extension Testing results]]


=== Monday, September 11th ===
=== Tuesday, November 14 ===  
* Smoketest rc6 builds
* Test updates for special distributions
** [[Firefox:1.5.0.7:Smoketests#RC6_builds_-_tested_09.2F10|1.5.0.7 Firefox Smoketest results]]
* Spot check Thunderbird P1 locales
** [[Firefox:1.5.0.7:LocalizationTesting|1.5.0.7 Firefox and Thunderbird Localization testing results]]
** Check other tier 2 locales as time permits
* Spot check a few CJK builds (WinXP)
** zhCN, ko Yahoo!: PASS (new build ID: 2006090918)
** ja, zhTW Google: PASS (new build ID: 2006090918)
** Other locales verified by diff scripts (rc5 vs rc6 PASS, no changes)
** Original test results:
*** https://intranet.mozilla.org/QA:FF1.5.0.7:GoogleCJK:Test_Results
*** https://intranet.mozilla.org/QA:FF1.5.0.7:YahooCJK:Test_Results


=== Tuesday, September 12th ===
=== Thursday, November 16 ===
* Focus/Exploratory testing for SSL/SMIME bug fixes
* Special distributions are released.
* Update testing for P1 locales
* Live updates for special distros.
** [[Firefox:1.5.0.7:UpdateTesting|1.5.0.7 Firefox and Thunderbird Update testing results]]


=== Wednesday, September 13th ===
== Test Results ==
* Update testing for P1 locales
Any 1.5.0.8 testing results that will help us track progress, identify regressions, and verify fixes should go here.
** [[Firefox:1.5.0.7:UpdateTesting|1.5.0.7 Firefox and Thunderbird Update testing results]]
* Update testing for Google/Yahoo! CJK
** [[Firefox:1.5.0.7:CJKUpdateTesting|1.5.0.7 Firefox Google/Yahoo! CJK Update testing results]]


=== Thursday, September 14th ===
* [[Firefox:1.5.0.8:Test_Plan:BFT | Basic Functional Tests (BFTs)]]
* Firefox 1.5.0.7 is released, probably 4pm
* [[Firefox:1.5.0.8:Test_Plan:l10n | l10n related testing]]
* Spot check live updates
* [[Firefox:1.5.0.8:Test_Plan:Software Update | Updates Testing]]
* [[Firefox:1.5.0.8:Test_Plan:Minor Major Software Update | Minor-Major Software Update Testing]]
*[https://intranet.mozilla.org/Firefox:Distribution:1.5:Google:1.5.0.8_Test_Results  Google Results]
 
'''The above list is just a sample of what we will be testing and links to blank pages we should us to track our test results.  Please add any other test results here.  Thanks!'''


== Bug Verifications ==
== Bug Verifications ==


When you verify the bug is fixed, resolve it as VERIFIED FIXED and replace the fixed1.8.0.7 keyword with verified1.8.0.7
When you verify the bug is fixed, resolve it as VERIFIED FIXED and replace the fixed1.8.0.8 keyword with verified1.8.0.8


Tips:
Tips:
* Bugs verified on 1.8.1 still need to be verified in 1.8.0.7
* Bugs verified on 1.8.1 still need to be verified in 1.8.0.8
* If you don't see an easy way to verify or need further clarification add a comment to the bug or add "[needs testcase]" (without quotes) to the bug's Status Whiteboard.  
* If you don't see an easy way to verify or need further clarification add a comment to the bug or add "[needs testcase]" (without quotes) to the bug's Status Whiteboard.  


'''Fixed'''
'''Fixed'''
*[https://bugzilla.mozilla.org/buglist.cgi?keywords_type=allwords&keywords=fixed1.8.0.7&order=Bug+Number fixed 1.5.0.7 bugs]
*[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=fixed1.8.0.8&emailtype1=substring&email1=&emailtype2=subst fixed 1.5.0.8 bugs]


'''Verified'''
'''Verified'''
*[https://bugzilla.mozilla.org/buglist.cgi?keywords_type=allwords&keywords=verified1.8.0.7&order=Bug+Number verified 1.5.0.7 bugs]
*[https://bugzilla.mozilla.org/buglist.cgi?keywords_type=allwords&keywords=verified1.8.0.8&order=Bug+Number verified 1.5.0.8 bugs]


=== Priority Verifications ===
=== Priority Verifications ===
* 7 mail bugs ("fixed1.8.0.7" bugs, component is "mailnews" or product is "thunderbird")
* 6 mail bugs ("fixed1.8.0.8" bugs, component is "mailnews" or product is "thunderbird")
**[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&keywords_type=allwords&keywords=fixed1.8.0.7&chfieldto=Now&field0-0-0=component&type0-0-0=substring&value0-0-0=mailnews&field0-0-1=product&type0-0-1=substring&value0-0-1=thunderbird mail bugs]
**[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&keywords_type=allwords&keywords=fixed1.8.0.8&chfieldto=Now&field0-0-0=component&type0-0-0=substring&value0-0-0=mailnews&field0-0-1=product&type0-0-1=substring&value0-0-1=thunderbird mail bugs]


* 20 critical bugs ("fixed1.8.0.7" bugs, no "needs testcase" in status whiteboard, and severity is "critical")
* 16 critical bugs ("fixed1.8.0.8" bugs, no "needs testcase" in status whiteboard, and severity is "critical")
**[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&keywords_type=allwords&keywords=fixed1.8.0.7&chfieldto=Now&field0-0-0=status_whiteboard&type0-0-0=notsubstring&value0-0-0=need+testcase&field0-1-0=bug_severity&type0-1-0=equals&value0-1-0=critical critical bugs]
**[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&keywords_type=allwords&keywords=fixed1.8.0.8&chfieldto=Now&field0-0-0=status_whiteboard&type0-0-0=notsubstring&value0-0-0=need+testcase&field0-1-0=bug_severity&type0-1-0=equals&value0-1-0=critical critical bugs]


* 11 security bugs ("fixed1.8.0.7" bugs, no "needs testcase" in status whiteboard, and is marked "security-sensitive"):
* 5 security bugs ("fixed1.8.0.8" bugs, no "needs testcase" in status whiteboard, and is marked "security-sensitive"):
**[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&keywords_type=allwords&keywords=fixed1.8.0.7&chfieldto=Now&field0-0-0=bug_group&type0-0-0=substring&value0-0-0=security&field0-1-0=status_whiteboard&type0-1-0=notsubstring&value0-1-0=need+testcase security bugs]
**[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&keywords_type=allwords&keywords=fixed1.8.0.8&chfieldto=Now&field0-0-0=bug_group&type0-0-0=substring&value0-0-0=security&field0-1-0=status_whiteboard&type0-1-0=notsubstring&value0-1-0=need+testcase security bugs]


Note: If you run into a difficult bug, just add the "[need testcase]" in the status whiteboard and move on.
Note: If you run into a difficult bug, just add the "[need testcase]" in the status whiteboard and move on.


== Need Verification ==
== Need Verification ==
bc's JS bugs:
* https://bugzilla.mozilla.org/show_bug.cgi?id=348532


== Cannot Verify ==
== Cannot Verify ==
* https://bugzilla.mozilla.org/show_bug.cgi?id=347725 (no crash with 1506, 1507, or 2)
* https://bugzilla.mozilla.org/show_bug.cgi?id=350081 (not sure what to look for with testcase)

Latest revision as of 19:51, 10 November 2006

General Firefox 1.5.0.x Test Plan

This is the basic set of tests we run for every security & maintenance release.


Assumptions

  • This release will be piggy-backing on 1.8.1
  • Builds should have gone through the l10n meta-diff checker and the update checker before they are released to QA

Current Tasks

  • Bug triage
  • Test plan completion
  • Minor/Major Software Updates discussions and testing
  • Bug verification
  • Tuesday, October 24 - Smoketests - tracy - PASS

Schedule

Tuesday, October 17

  • Start bug verifications for bugs marked fixed1.8.0.8
  • Post to QA blog for Community Testing ?

Friday, October 20

  • Continue bug verifications
  • Start testing minor/major softare updates with mock-ups provided by Seth

Tuesday, October 24

  • FF 1.5.0.8rc1 builds ready
  • Firefox Smoketets on all platforms - tracy - PASS
  • L10n builds ready to be tested
  • BFT's started BFT Results

Wednesday, October 25

  • Smoketest and BFTs for Thunderbird
  • Start Firefox L10 Testing
  • Continue Verifications
  • Minor/Major Software Updates Testing

Thursday, October 26

If not ahead of schedule:

  • Smoketest and BFTs for Thunderbird
  • Start Firefox L10 Testing
  • Continue Verifications
  • Minor/Major Software Updates Testing

Friday, October 27 - Friday, November 3

Monday, November 6th

  • End-to-end Major update testing for 1.5.0.8 -> 2.0
  • Wrap up bug verifications
  • Final l10n + update testing on staging server.

Tuesday, November 7th

  • Firefox 1.5.0.8 is released, probably 4pm
  • Push website changes and security notes
  • Spot check live updates and download links

Wednesday, November 8 - Monday, November 13

  • Start special distro spotchecks, and continue until Monday, Nov 13

Tuesday, November 14

  • Test updates for special distributions

Thursday, November 16

  • Special distributions are released.
  • Live updates for special distros.

Test Results

Any 1.5.0.8 testing results that will help us track progress, identify regressions, and verify fixes should go here.

The above list is just a sample of what we will be testing and links to blank pages we should us to track our test results. Please add any other test results here. Thanks!

Bug Verifications

When you verify the bug is fixed, resolve it as VERIFIED FIXED and replace the fixed1.8.0.8 keyword with verified1.8.0.8

Tips:

  • Bugs verified on 1.8.1 still need to be verified in 1.8.0.8
  • If you don't see an easy way to verify or need further clarification add a comment to the bug or add "[needs testcase]" (without quotes) to the bug's Status Whiteboard.

Fixed

Verified

Priority Verifications

  • 6 mail bugs ("fixed1.8.0.8" bugs, component is "mailnews" or product is "thunderbird")
  • 16 critical bugs ("fixed1.8.0.8" bugs, no "needs testcase" in status whiteboard, and severity is "critical")
  • 5 security bugs ("fixed1.8.0.8" bugs, no "needs testcase" in status whiteboard, and is marked "security-sensitive"):

Note: If you run into a difficult bug, just add the "[need testcase]" in the status whiteboard and move on.

Need Verification

Cannot Verify

Retrieved from "https://wiki.mozilla.org/index.php?title=Firefox:1.5.0.8:Test_Plan&oldid=41009"