Firefox:1.5.0.7:Test Plan

General Firefox 1.5.0.x Test Plan

This is the basic set of tests we run for every security & maintenance release.

Assumptions

• Goal of this release is security fixes and important regressions, other fixes will not be included
• Builds should have gone through the l10n meta-diff checker and the update checker before they are released to QA
• Thunderbird will be released a few days after Firefox
• Monday, September 4th is a holiday

Current Tasks

• Bug triage
• Test plan completion
• Bug verification

Schedule

Monday, August 28th

• RC1 delayed till Monday night/Tuesday morning. Shifting Monday's planned activities to Tuesday

Tuesday, August 29th

• Firefox builds should be available for Win/Linux/PPC
• Smoketests (rc2) on Windows, Linux, Mac en-us
  • 1.5.0.7 Firefox and Thunderbird Smoketest results
• Work out BFT schedule (divide and conquer)

Wednesday, August 30th

• Firefox BFTs - PASS
  • tester: tracy
  • Windows XP
  • Build ID: 2006082915
  • Results: results query
    • note: long standing failure in copy/paste in bookmarks - not a blocker for release
• Build should begin l10n packaging after BFTs signoff (Rhelmer/TR)
• Get post up on the QA blog with link to candidate builds
• Testing of QA identified hot spots
• Targeted exploratory testing over Developer identified hot spots

Thursday, August 31st

• Continued testing of developer identified hot spots and bug verifications

Friday, September 1st

• Community Testday for 1.5.0.7 RC builds
• Smoketests (rc2) on Windows, Linux, Mac en-us
  • 1.5.0.7 Firefox and Thunderbird Smoketest results
• Windows BFT run on Thunderbird RC3 by Tomcat
  • Build ID= 2006090109
  • Testing results in Litmus
• Bug verifications and focus testing

Tuesday, September 5th

• Further bug verifications and testing of hot spots
• Thunderbird Smoketests
• Start Thunderbird BFTs

Wednesday, September 6th

• Firefox RC4 will be released this afternoon
• Smoketest Firefox RC4
• Thunderbird bug verifications

Thursday, September 7th

• Firefox security testing for RSA/Certificate changes
  • https://bugzilla.mozilla.org/show_bug.cgi?id=340198
  • https://bugzilla.mozilla.org/show_bug.cgi?id=351079
  • https://bugzilla.mozilla.org/show_bug.cgi?id=350640
• Thunderbird bug verifications

Friday, September 8th

• Spot check P1 locales
  • 1.5.0.7 Firefox and Thunderbird Localization testing results
  • Check other tier 2 locales as time permits

Monday, September 11th

• Update testing for P1 locales
  • 1.5.0.7 Firefox and Thunderbird Update testing results
• Random update spot checks of other locales

Tuesday, September 12th

• Firefox 1.5.0.7 is released, probably 4pm
• Spot check live updates

Wednesday, September 13th

• Spot check Thunderbird P1 locales
  • 1.5.0.7 Firefox and Thunderbird Localization testing results
  • Check other tier 2 locales as time permits

Thursday, September 14th

• Update testing for Thunderbird P1 locales
  • 1.5.0.7 Firefox and Thunderbird Update testing results
• Random update spot checks of other locales

Bug Verifications

When you verify the bug is fixed, resolve it as VERIFIED FIXED and replace the fixed1.8.0.7 keyword with verified1.8.0.7

Tips:

• Bugs verified on 1.8.1 still need to be verified in 1.8.0.7
• If you don't see an easy way to verify or need further clarification add a comment to the bug or add "[needs testcase]" (without quotes) to the bug's Status Whiteboard.

Fixed

• fixed 1.5.0.7 bugs

Verified

• verified 1.5.0.7 bugs

Need Verification

bc's JS bugs:

• https://bugzilla.mozilla.org/show_bug.cgi?id=348532

Cannot Verify

• https://bugzilla.mozilla.org/show_bug.cgi?id=347725 (no crash with 1506, 1507, or 2)
• https://bugzilla.mozilla.org/show_bug.cgi?id=350081 (not sure what to look for with testcase)