Firefox:3.0.3:Test Plan: Difference between revisions

Latest revision as of 00:20, 26 September 2008

General Firefox 3.0.3 Test Plan

This is the basic set of tests we run for every security & maintenance release.

Description of Release

3.0.3 is a emergency security release for Firefox. Reason for it the regression that caused bug 454708 - storage-Legacy can throw when calling ConvertToUnicode

Specific Test Plan for this Release

Run smoketest, updates, and l10n tests
Run focused testing on password manager and associated areas around bug 454708

Assumptions

Current Tasks

Ongoing: Bug Reproduction

Schedule

This is an estimated schedule as a normal security release.

Code freeze: Sept 24
Builds start: Sept 25
QA starts: Sept 25 (4 testers)
- Smoketests: 1 day
- Feature Spotchecks (focus on Password Manager): 1/2 day
- l10n Testing: 4 hours
- Betatest & Beta Channel Testing: 1 day
Release to beta channel: Sept 25
- Releasetest & Release Channel Testing: Sept 26
Final Release: Sept 26

Builds

Downloaded from http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/3.0.3-candidates/build1/

Mac 10.5
- Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.3) Gecko/2008092414 Firefox/3.0.3

Mac 10.4
- Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.0.3) Gecko/2008092414 Firefox/3.0.3

Windows XP And Vista (unsigned)
- Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3
- Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3

Linux
- Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092416 Firefox/3.0.3

Test Results

**Build 1**
	Windows	Mac	Linux
Smoketests	tracy Pass	marcia PASS	jmaher Pass
Password Manager BFTs	tracy Pass	marcia/tchung PASS	jmaher Pass
L10n	Tchung PASS	Tchung PASS	Tchung PASS
Updates	tracy Betatest - PASS Release - result Releasetest - result Release - result	tomcat Betatest - PASS Release - result Releasetest - result Release - result	jmaher Betatest - PASS Release - result Releasetest - result Release - result

Detailed Test Results

P1 Spotchecks

Check out the builds for sanity and regressions. Pay attention especially to the password manager code.

Bug Verifications

When you verify the bug is fixed:

Mark the whiteboard from fixed1.9.0.2 to verified1.9.0.2
Affected Bugs:
- bug 451155 - Password manager does not work correctly on IDN site whose name contains any character over U+0100 verified fixed
- bug 454708 - storage-Legacy can throw when calling ConvertToUnicode verified fixed
- bug 454993 - saved passwords lost/gone on migration/update from 3.0.1 to 3.0.2 Duped to bug 454708 above

Tips:

If you don't see an easy way to verify or need further clarification add a comment to the bug or add "[needs testcase]" (without quotes) to the bug's Status Whiteboard.

@@ Line 21: / Line 21: @@
 * Builds start: Sept 25
 * QA starts: Sept 25 (4 testers)
-** Smoketests: 1 1/2 day
+** Smoketests: 1 day
 ** Feature Spotchecks (focus on Password Manager): 1/2 day
 ** l10n Testing: 4 hours
 ** Betatest & Beta Channel Testing: 1 day
-* Release to beta channel: Sept 29
+* Release to beta channel: Sept 25
-** Releasetest & Release Channel Testing: TBD
+** Releasetest & Release Channel Testing: Sept 26
-* Final Release: TBD
+* Final Release: Sept 26
 == Builds ==
@@ Line 34: / Line 34: @@
 *Mac 10.5
 ** Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.3) Gecko/2008092414 Firefox/3.0.3
 *Mac 10.4
 **Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.0.3) Gecko/2008092414 Firefox/3.0.3
+*Windows XP And Vista (unsigned)
+** Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3
+** Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3
+*Linux
+** Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092416 Firefox/3.0.3
 == Test Results ==
@@ Line 48: / Line 56: @@
 |-
 | style="background:#ffdead;" | '''Smoketests'''
-| tracy '''result'''
+| tracy '''Pass'''
-| marcia '''result'''
+| marcia '''PASS'''
-| jmaher '''result'''
+| jmaher '''Pass'''
+|-
+| style="background:#ffdead;" | '''Password Manager BFTs'''
+| tracy '''Pass'''
+| marcia/tchung '''PASS'''
+| jmaher '''Pass'''
 |-
 | style="background:#ffdead;" | '''L10n'''
-| Tchung '''result'''
+| Tchung '''PASS'''
-| Tchung '''result'''
+| Tchung '''PASS'''
-| Tchung '''result'''
+| Tchung '''PASS'''
 |-
 | style="background:#ffdead;" | '''Updates'''
-| tracy <BR> '''Betatest - result'''<BR> '''Release  - result''' <BR> '''Releasetest - result'''  <BR> '''Release - result''' <BR>
+| tracy <BR> '''Betatest - PASS'''<BR> '''Release  - result''' <BR> '''Releasetest - result'''  <BR> '''Release - result''' <BR>
-| tomcat <BR> '''Betatest - result'''<BR> '''Release  - result''' <BR> '''Releasetest - result'''  <BR> '''Release - result''' <BR>
+| tomcat <BR> '''Betatest - PASS'''<BR> '''Release  - result''' <BR> '''Releasetest - result'''  <BR> '''Release - result''' <BR>
-| jmaher <BR> '''Betatest - result''' <BR> '''Release - result''' <BR>  '''Releasetest - result''' <BR> '''Release - result''' <BR>
+| jmaher <BR> '''Betatest - PASS''' <BR> '''Release - result''' <BR>  '''Releasetest - result''' <BR> '''Release - result''' <BR>
 |-
 |}
@@ Line 78: / Line 91: @@
 * Mark the whiteboard from ''fixed1.9.0.2'' to ''verified1.9.0.2''
 * Affected Bugs:
-** {{Bug|451155}} - Password manager does not work correctly on IDN site whose name contains any character over U+0100
+** {{Bug|451155}} - Password manager does not work correctly on IDN site whose name contains any character over U+0100 '''verified fixed'''
-** {{Bug|454708}} -  storage-Legacy can throw when calling ConvertToUnicode
+** {{Bug|454708}} -  storage-Legacy can throw when calling ConvertToUnicode '''verified fixed'''
-** {{Bug|454993}} -  saved passwords lost/gone on migration/update from 3.0.1 to 3.0.2
+** {{Bug|454993}} -  saved passwords lost/gone on migration/update from 3.0.1 to 3.0.2 ''Duped to bug 454708 above''
 Tips:
 * If you don't see an easy way to verify or need further clarification add a comment to the bug or add "[needs testcase]" (without quotes) to the bug's Status Whiteboard.