Firefox:1.5.0.5:Test Plan

General Firefox 1.5.0.x Test Plan

This is the basic set of tests we run for every security & maintenance release.

Current Tasks

I am including all end game tasks including build, security notifications, etc. If anyone doesn't like this, I can create a separate cross-functional tasks page at a higher level.

Assumption: Builds should have gone through the l10n meta-diff checker and the update checker before they are released to QA.

Friday, June 16 - Wednesday, June 22

• Ongoing bug verifications. Please see list here: http://tinyurl.com/e6haf
  • If everyone on the QA team could pitch in for a few verifications, that would be great.

Wednesday - June 22nd

• Firefox and Thunderbird Builds should be available Win/Linux/PPC - rhelmer/preed
  • Build should tag the releases

Thursday/Friday - June 23/24

• Smoketests begin on all platforms for Firefox - tracy/juanb
• Smoketests begin on all platforms for Thunderbird - marcia/adam
• Post builds to the community blog for QA testing - marcia
• Build should begin l10n Packaging

Tuesday - June 27

• Smoketest candidate builds on Windows and Mac - tracy/marcia
  • Get post up on the QA blog with link to candidate builds
  • Work out BFT schedule (divide and conquer)

Wednesday - June 27 and Thursday June 28

• Review feedback from QA blog

Week of June 26 - June 30

• Informal Community Testing and Test Day

Wednesday July 5 - Friday July 7th

*Firefox 1.5.0.5 Smoketest:*

• Target completion date is 7-5-2006
• Results go here: http://wiki.mozilla.org/Firefox:1.5.0.5:Test_Plan:Smoketest
• Windows: Marcia
• Intel Mac: Marcia
• Linux: Juanb (unless you can't get to a Linux machine in which case we can swap platforms)

If the smoketests pass, we should immediately move to running the Firefox BFT on Windows:

• FF BFT Windows - Marcia/Tracy/Juanb (we should divide up the sections like we did previously-please coordinate on IRC to get this done)
  • Target Completion date is 7-6-2006

*Thunderbird 1.5.0.5 Smoketest*

• Target completion date: ASAP after Firefox
• Results go here: http://wiki.mozilla.org/Firefox:1.5.0.5:Test_Plan:Smoketest
• Windows: Marcia
• Intel Mac: Juanb
• Linux: Adam

• Tbird BFT on Windows - marcia/tracy/juanb - 7.13.2006 Thunderbird BFT results
• Target completion date: ASAP after Firefox

*Miscellaneous Errata

• Bug Verifications should be ongoing
• We are on track for a July 19th release

Monday, July 10 - Wed July 12 =

• l10n + Update Testing: Firefox
• Prep Security Notes and Notify Security Group - (Dveditz)
• Prep Release Notes for Firefox - ?
• Prepare Release Notes for Thunderbird - mscott

Wednesday July 12 - Friday 14

• l10n + Update Testing: Thunderbird
  • Test Assignments for New Locales being Released:
    • Thunderbird ga-IE [marcia]
    • Firefox fy-NL [juanb] - Bookmarks menu item text not displaying correctly bug 344633

• Build: Release Signing Begins for FF and Tbird

All l10n test results should go here: http://wiki.mozilla.org/QA:FF1.5.0.5:L10n_Spot_Check_Results

Specific Testing for 1.5.0.5

There are a few areas not covered by the general test plan above, and we will track that testing here. A few things to think about and test/verify with 1.5.0.5 include:

• MacOS X Universal Binaries (marcia/Juanb/tracy)
  • both BFTs and software update need to be thoroughly tested to ensure all existing users of 1.5.0.4 are properly updated to 1.5.0.5 (both PPC and UB users).
• Thunderbird PPB-UB Update Testing
• Additional Extension testing
  • http://www.mozilla.com/add-ons/jogacompanion/ - joga.com extension

Please add any other bugs/areas that need special attention for this release. Thanks!

Test Results (Firefox and Thunderbird)

Any 1.5.0.5 testing results that will help us track progress, identify regressions, and verify fixes should go here.

• Smoketests
• Basic Functional Tests (BFTs)
• MacOS X UB testing
• l10n related testing
• Software Update Testing
• Results for all extension testing

The above list is just a sample of what we will be testing and links to blank pages we should use to track our test results. Please add any other test results here. Thanks!

Bug Verifications

Fixed

• fixed1.8.0.5 bugs

Verified

• verified1.8.0.5 Bugs

Need Verification

• Bugs with JS testcases (bc?)
  • verified.

Cannot Verify

• https://bugzilla.mozilla.org/show_bug.cgi?id=340129 - topcrash bug with no testcase, can only be verified after release (to see if crash goes away).