Firefox:1.5.0.7:Test Plan

General Firefox 1.5.0.x Test Plan

This is the basic set of tests we run for every security & maintenance release.

Assumptions

• Goal of this release is security fixes and important regressions, other fixes will not be included
• Builds should have gone through the l10n meta-diff checker and the update checker before they are released to QA
• Thunderbird will be released a few days after Firefox
• Monday, September 4th is a holiday

Current Tasks

• Bug triage
• Test plan completion
• Bug verification

Schedule

Monday, August 28th

• RC1 delayed till Monday night/Tuesday morning. Shifting Monday's planned activities to Tuesday

Tuesday, August 29th

• Firefox builds should be available for Win/Linux/PPC
• Smoketests (rc2) on Windows, Linux, Mac en-us
  • 1.5.0.7 Firefox and Thunderbird Smoketest results

Wednesday, August 30th

• Firefox BFTs - PASS
  • 1.5.0.7 Firefox and Thunderbird BFT results
• Build should begin l10n packaging after BFTs signoff (Rhelmer/TR)
• Get post up on the QA blog with link to candidate builds
• Testing of QA identified hot spots
• Targeted exploratory testing over Developer identified hot spots

Thursday, August 31st

• Continued testing of developer identified hot spots and bug verifications

Friday, September 1st

• Community Testday for 1.5.0.7 RC builds
• Smoketests (rc2) on Windows, Linux, Mac en-us
  • 1.5.0.7 Firefox and Thunderbird Smoketest results
• Thunderbird BFTs - PASS
  • 1.5.0.7 Firefox and Thunderbird BFT results
• Bug verifications and focus testing

Tuesday, September 5th

• Further bug verifications and testing of hot spots
• Thunderbird Smoketests
• Start Thunderbird BFTs

Wednesday, September 6th

• Firefox RC4 will be released this afternoon
• Smoketest Firefox RC4
  • 1.5.0.7 Firefox Smoketest results
• Thunderbird bug verifications

Thursday, September 7th

• Respins for rc5 out late Sept. 6
• Smoketest rc5 builds
  • 1.5.0.7 Firefox Smoketest results
• Firefox security testing for RSA/Certificate changes
  • https://bugzilla.mozilla.org/show_bug.cgi?id=340198
  • https://bugzilla.mozilla.org/show_bug.cgi?id=351079
  • https://bugzilla.mozilla.org/show_bug.cgi?id=350640
• Thunderbird bug verifications

Friday, September 8th

• Spot check P1 locales
  • 1.5.0.7 Firefox and Thunderbird Localization testing results
  • Check other tier 2 locales as time permits
  • 1.5.0.7 Firefox and Thunderbird Extension Testing results

Monday, September 11th

• Smoketest rc6 builds
  • 1.5.0.7 Firefox Smoketest results
• Spot check Thunderbird P1 locales
  • 1.5.0.7 Firefox and Thunderbird Localization testing results
  • Check other tier 2 locales as time permits
• Spot check a few CJK builds (WinXP)
  • zhCN, ko Yahoo!: PASS (new build ID: 2006090918)
  • ja, zhTW Google: PASS (new build ID: 2006090918)
  • Other locales verified by diff scripts (rc5 vs rc6 PASS, no changes)
  • Original test results:
    • https://intranet.mozilla.org/QA:FF1.5.0.7:GoogleCJK:Test_Results
    • https://intranet.mozilla.org/QA:FF1.5.0.7:YahooCJK:Test_Results

Tuesday, September 12th

• Focus/Exploratory testing for SSL/SMIME bug fixes
• Update testing for P1 locales
  • 1.5.0.7 Firefox and Thunderbird Update testing results

Wednesday, September 13th

• Update testing for P1 locales
  • 1.5.0.7 Firefox and Thunderbird Update testing results
• Update testing for Google/Yahoo! CJK
  • 1.5.0.7 Firefox Google/Yahoo! CJK Update testing results

Thursday, September 14th

• Firefox 1.5.0.7 is released, probably 4pm
• Spot check live updates

Bug Verifications

When you verify the bug is fixed, resolve it as VERIFIED FIXED and replace the fixed1.8.0.7 keyword with verified1.8.0.7

Tips:

• Bugs verified on 1.8.1 still need to be verified in 1.8.0.7
• If you don't see an easy way to verify or need further clarification add a comment to the bug or add "[needs testcase]" (without quotes) to the bug's Status Whiteboard.

Fixed

• fixed 1.5.0.7 bugs

Verified

• verified 1.5.0.7 bugs

Priority Verifications

• 7 mail bugs ("fixed1.8.0.7" bugs, component is "mailnews" or product is "thunderbird")
  • mail bugs

• 20 critical bugs ("fixed1.8.0.7" bugs, no "needs testcase" in status whiteboard, and severity is "critical")
  • critical bugs

• 11 security bugs ("fixed1.8.0.7" bugs, no "needs testcase" in status whiteboard, and is marked "security-sensitive"):
  • security bugs

Note: If you run into a difficult bug, just add the "[need testcase]" in the status whiteboard and move on.

Need Verification

bc's JS bugs:

• https://bugzilla.mozilla.org/show_bug.cgi?id=348532

Cannot Verify

• https://bugzilla.mozilla.org/show_bug.cgi?id=347725 (no crash with 1506, 1507, or 2)
• https://bugzilla.mozilla.org/show_bug.cgi?id=350081 (not sure what to look for with testcase)