Firefox:1.5.0.10-2.0.0.2:Test Plan: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
|||
| (45 intermediate revisions by 7 users not shown) | |||
| Line 12: | Line 12: | ||
== Current Tasks == | == Current Tasks == | ||
*Bug Triage | * Bug Triage (1/23 - completed, working with devs on case by case basis) | ||
* '''[[Firefox:1.5.0.10-2.0.0.2:Test_Plan#Bug_Verifications | Bug Verifications]]''' | |||
* ''' | |||
== Schedule == | == Schedule == | ||
* Tentative Release Date: ''' | * Tentative Release Date: '''Mid February''' | ||
* Overall Schedule from [[Firefox:1.5.0.10-2.0.0.2:Community#Overall_Schedule|Community Page]] | * Overall Schedule from [[Firefox:1.5.0.10-2.0.0.2:Community#Overall_Schedule|Community Page]] | ||
| Line 27: | Line 23: | ||
{| border="1" cellpadding="5" cellspacing="0" align="center" | {| border="1" cellpadding="5" cellspacing="0" align="center" | ||
|+''' | |+'''COMPLETED''' | ||
|- | |- | ||
! | ! | ||
| Line 34: | Line 30: | ||
|- | |- | ||
| style="background:#ffdead;" | '''Firefox 2.0.0.2''' | | style="background:#ffdead;" | '''Firefox 2.0.0.2''' | ||
| | | '''PASS''' | ||
| | | '''PASS''' | ||
|- | |- | ||
| style="background:#ffdead;" | '''Firefox 1.5.0.10''' | | style="background:#ffdead;" | '''Firefox 1.5.0.10''' | ||
| | | '''PASS''' | ||
| | | '''PASS''' | ||
|- | |- | ||
|} | |} | ||
* [ | *[http://wiki.mozilla.org/index.php?title=Firefox:1.5.0.10-2.0.0.2:Test_Plan:BFT Smoketests and BFTs] | ||
* [[Firefox:1.5.0.10-2.0.0.2:Test_Plan:l10n | l10n related testing]] | * [[Firefox:1.5.0.10-2.0.0.2:Test_Plan:l10n | l10n related testing]] | ||
* [[Firefox:1.5.0.10-2.0.0.2:Test_Plan:Software Update | Updates Testing]] | * [[Firefox:1.5.0.10-2.0.0.2:Test_Plan:Software Update | Updates Testing]] | ||
* [https://intranet.mozilla.org/Major_Update_1.5.0.x_to_2.0.0.x | * [https://intranet.mozilla.org/Major_Update_1.5.0.x_to_2.0.0.x Major Software Updates Testing] | ||
* [[Firefox:1.5.0.10-2.0.0.2:Test_Plan:Vista Focused Testing | Vista Focused Testing]] | |||
* [[Firefox:1.5.0.10-2.0.0.2:Test_Plan:Vista Software Update | Vista Updates Testing]] | * [[Firefox:1.5.0.10-2.0.0.2:Test_Plan:Vista Software Update | Vista Updates Testing]] | ||
* [[Firefox:1.5.0.10-2.0.0.2:Test_Plan:Distro Test Results | Distro Test Results]] | * [[Firefox:1.5.0.10-2.0.0.2:Test_Plan:Distro Test Results | Distro Test Results]] | ||
* Security Regression tests Windows - Passed | |||
* JavaScript tests - Passed | |||
* Search Engine Checker Results: (need link) | * Search Engine Checker Results: (need link) | ||
* Continued new locale testing results: | * Continued new locale testing results: | ||
* [[Firefox:1.5.0.10-2.0.0.2:Test_Plan:Thunderbird 1.5.0.10 | Thunderbird 1.5.0.10]] | |||
'''The above list is just a sample of what we will be testing and links to blank pages we should us to track our test results. Please add any other test results here. Thanks!''' | '''The above list is just a sample of what we will be testing and links to blank pages we should us to track our test results. Please add any other test results here. Thanks!''' | ||
| Line 56: | Line 56: | ||
== Bug Verifications == | == Bug Verifications == | ||
'''When you verify the bug is fixed, resolve it as VERIFIED FIXED | '''When you verify the bug is fixed:''' | ||
* If you test on Trunk, resolve it as VERIFIED FIXED | |||
* If you test on 1.8.x.x branch, replace the fixed1.8.0.10 / fixed1.8.1.2 keyword with verified1.8.0.10 / verified1.8.1.2 | |||
Tips: | Tips: | ||
* Verify on both 1.8.0.10 and 1.8.1.2 | * Verify on both 1.8.0.10 and 1.8.1.2 for bugs fixed on both branches | ||
* If you don't see an easy way to verify or need further clarification add a comment to the bug or add "[needs testcase]" (without quotes) to the bug's Status Whiteboard. | * If you don't see an easy way to verify or need further clarification add a comment to the bug or add "[needs testcase]" (without quotes) to the bug's Status Whiteboard. | ||
'''Fixed''' | '''Fixed''' | ||
*[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type= | *[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=anywords&keywords=fixed1.8.0.10+fixed1.8.1.2&emailtype1=substring&email1=&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&field0-0-0=keywords&type0-0-0=notsubstring&value0-0-0=&field0-1-0=product&type0-1-0=notsubstring&value0-1-0=Camino&field0-2-0=product&type0-2-0=notsubstring&value0-2-0=Localizations&field0-3-0=product&type0-3-0=nowordssubstr&value0-3-0=Thunderbird&field0-4-0=component&type0-4-0=notsubstring&value0-4-0=XForms '''All Fixed Bugs''' (including 1.8.1.2 and 1.8.0.10, with no bugs marked "verified1.8.x.x")] | ||
*[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=fixed1.8.1.2&emailtype1=substring&email1=&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&field0-0-0=keywords&type0-0-0=notsubstring&value0-0-0= | ** Refined bug lists for quick reference fo what's fixed where: | ||
** | ***[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=anywords&keywords=fixed1.8.0.10+fixed1.8.1.2&emailtype1=substring&email1=&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&field0-0-0=status_whiteboard&type0-0-0=notregexp&value0-0-0=%5Bneed.%3Fs+testcase%5D&field0-1-0=product&type0-1-0=notsubstring&value0-1-0=Camino&field0-2-0=product&type0-2-0=notsubstring&value0-2-0=Localizations&field0-3-0=product&type0-3-0=nowordssubstr&value0-3-0=Thunderbird&field0-4-0=component&type0-4-0=notsubstring&value0-4-0=XForms ALL fixed1.8.1.2 OR Fixed1.8.0.10 (without "need.?s testcase")] | ||
***[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=fixed1.8.0.10+fixed1.8.1.2&emailtype1=substring&email1=&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&field0-0-0=keywords&type0-0-0=notsubstring&value0-0-0=&field0-1-0=product&type0-1-0=notsubstring&value0-1-0=Camino&field0-2-0=product&type0-2-0=notsubstring&value0-2-0=Localizations&field0-3-0=product&type0-3-0=nowordssubstr&value0-3-0=Thunderbird&field0-4-0=component&type0-4-0=notsubstring&value0-4-0=XForms BOTH fixed1.8.1.2 AND fixed1.8.0.10] | |||
***[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=fixed1.8.0.10&emailtype1=substring&email1=&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&field0-0-0=keywords&type0-0-0=notsubstring&value0-0-0=fixed1.8.1.2&field0-1-0=product&type0-1-0=notsubstring&value0-1-0=Camino&field0-2-0=product&type0-2-0=notsubstring&value0-2-0=Localizations&field0-3-0=product&type0-3-0=nowordssubstr&value0-3-0=Thunderbird&field0-4-0=component&type0-4-0=notsubstring&value0-4-0=XForms fixed1.8.0.10 (1.5.0.10) only] | |||
***[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=fixed1.8.1.2&emailtype1=substring&email1=&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&field0-0-0=keywords&type0-0-0=notsubstring&value0-0-0=fixed1.8.0.10&field0-1-0=product&type0-1-0=notsubstring&value0-1-0=Camino&field0-2-0=product&type0-2-0=notsubstring&value0-2-0=Localizations&field0-3-0=product&type0-3-0=nowordssubstr&value0-3-0=Thunderbird&field0-4-0=component&type0-4-0=notsubstring&value0-4-0=XForms fixed1.8.1.2 (2.0.0.2) only] | |||
** All bug lists above exclude Camino, Thunderbird, Localizations and XForms | |||
'''Verified''' | '''Verified''' | ||
*[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=anywords&keywords=verified1.8.1.2+verified1.8.0.10&emailtype1=substring&email1=&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&field0-0-0=keywords&type0-0-0=nowordssubstr&value0-0-0=fixed1.8.1.2+fixed1.8.0.10 '''All Verified Bugs''' (including 1.8.1.2 and 1.8.0.10, with no bugs marked "fixed1.8.x.x")] | |||
*[https://bugzilla.mozilla.org/buglist.cgi?keywords_type=allwords&keywords=verified1.8.0.10&order=Bug+Number verified 1.5.0.10 bugs] | *[https://bugzilla.mozilla.org/buglist.cgi?keywords_type=allwords&keywords=verified1.8.0.10&order=Bug+Number verified 1.5.0.10 bugs] | ||
*[https://bugzilla.mozilla.org/buglist.cgi?keywords_type=allwords&keywords=verified1.8.1.2&order=Bug+Number verified 2.0.0.2 bugs] | *[https://bugzilla.mozilla.org/buglist.cgi?keywords_type=allwords&keywords=verified1.8.1.2&order=Bug+Number verified 2.0.0.2 bugs] | ||
| Line 77: | Line 84: | ||
* Security bugs ("fixed1.8.0.10" or "fixed1.8.1.2" bugs, no "needs testcase" in status whitebooard, and is marked "security-sensitive") | * Security bugs ("fixed1.8.0.10" or "fixed1.8.1.2" bugs, no "needs testcase" in status whitebooard, and is marked "security-sensitive") | ||
**[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=fixed1.8.0.10&emailtype1=substring&email1=&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&field0-0-0=bug_group&type0-0-0=substring&value0-0-0=security&field0-1-0=status_whiteboard&type0-1-0=notsubstring&value0-1-0=need+testcase 1.5.0.10 security bugs] | **[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=fixed1.8.0.10&emailtype1=substring&email1=&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&field0-0-0=bug_group&type0-0-0=substring&value0-0-0=security&field0-1-0=status_whiteboard&type0-1-0=notsubstring&value0-1-0=need+testcase 1.5.0.10 security bugs] | ||
**[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=fixed1.8.1.2&emailtype1=substring&email1=&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&field0-0-0=bug_group&type0-0-0=substring&value0-0-0=security&field0-1-0=status_whiteboard&type0-1-0=notsubstring&value0-1-0=need+testcase 2.0.0.2 security bugs] | **[https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=fixed1.8.1.2&emailtype1=substring&email1=&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&field0-0-0=bug_group&type0-0-0=substring&value0-0-0=security&field0-1-0=status_whiteboard&type0-1-0=notsubstring&value0-1-0=need+testcase 2.0.0.2 security bugs] | ||
| Line 83: | Line 90: | ||
== Need Verification == | == Need Verification == | ||
* bc's js tests | |||
https://bugzilla.mozilla.org/show_bug.cgi?id=357398 | |||
https://bugzilla.mozilla.org/show_bug.cgi?id=357392 | |||
== Cannot/Won't Verify == | |||
* Needs testcase or difficult to reproduce: | |||
https://bugzilla.mozilla.org/show_bug.cgi?id=364323 | |||
https://bugzilla.mozilla.org/show_bug.cgi?id=360526 | |||
https://bugzilla.mozilla.org/show_bug.cgi?id=361781 | |||
https://bugzilla.mozilla.org/show_bug.cgi?id=362110 (need to run test in JS Shell) | |||
https://bugzilla.mozilla.org/show_bug.cgi?id=362547 (need xpcshell to run test) | |||
https://bugzilla.mozilla.org/show_bug.cgi?id=363849 (need JAWS for accessibility testcase) | |||
* Verified by code inspection (should revisit later to confirm): | |||
https://bugzilla.mozilla.org/show_bug.cgi?id=361332 | |||
== | * Non-Firefox bugs (Camino, Sunbird, Thunderbird, etc.): | ||
https://bugzilla.mozilla.org/show_bug.cgi?id=365248 | |||
https://bugzilla.mozilla.org/show_bug.cgi?id=362186 (tried to test with TB, but didn't have any luck, asked reporter to retest).<br> | |||
https://bugzilla.mozilla.org/show_bug.cgi?id=363043 (minor change for Sunbird only) | |||
Latest revision as of 16:43, 17 August 2007
General Firefox 1.5.0.x Test Plan
This is the basic set of tests we run for every security & maintenance release.
Windows Vista Test Plan
Firefox 2.0.0.2 will be the first release where we support the Windows Vista platform. To that end, Marcia created a Windows Vista Test Plan. You can view the latest iteration of that document here: Windows Vista Test Plan
Assumptions
- 1.5.0.10 and 2.0.0.2 are simultaneous releases of Firefox. It hasn't been decided if we will ship Thunderbird 1.5.0.10 at the same time.
Current Tasks
- Bug Triage (1/23 - completed, working with devs on case by case basis)
- Bug Verifications
Schedule
- Tentative Release Date: Mid February
- Overall Schedule from Community Page
Test Results
Any 1.5.0.10-2.0.0.2 testing results that will help us track progress, identify regressions, and verify fixes should go here.
| Smoketests | BFTs | |
|---|---|---|
| Firefox 2.0.0.2 | PASS | PASS |
| Firefox 1.5.0.10 | PASS | PASS |
- Smoketests and BFTs
- l10n related testing
- Updates Testing
- Major Software Updates Testing
- Vista Focused Testing
- Vista Updates Testing
- Distro Test Results
- Security Regression tests Windows - Passed
- JavaScript tests - Passed
- Search Engine Checker Results: (need link)
- Continued new locale testing results:
- Thunderbird 1.5.0.10
The above list is just a sample of what we will be testing and links to blank pages we should us to track our test results. Please add any other test results here. Thanks!
Bug Verifications
When you verify the bug is fixed:
- If you test on Trunk, resolve it as VERIFIED FIXED
- If you test on 1.8.x.x branch, replace the fixed1.8.0.10 / fixed1.8.1.2 keyword with verified1.8.0.10 / verified1.8.1.2
Tips:
- Verify on both 1.8.0.10 and 1.8.1.2 for bugs fixed on both branches
- If you don't see an easy way to verify or need further clarification add a comment to the bug or add "[needs testcase]" (without quotes) to the bug's Status Whiteboard.
Fixed
- All Fixed Bugs (including 1.8.1.2 and 1.8.0.10, with no bugs marked "verified1.8.x.x")
- Refined bug lists for quick reference fo what's fixed where:
- All bug lists above exclude Camino, Thunderbird, Localizations and XForms
Verified
- All Verified Bugs (including 1.8.1.2 and 1.8.0.10, with no bugs marked "fixed1.8.x.x")
- verified 1.5.0.10 bugs
- verified 2.0.0.2 bugs
Priority Verifications
- Critical bugs ("fixed1.8.0.10" or "fixed1.8.1.2" bugs, no "needs testcase" in status whiteboard, and severity is "critical")
- Security bugs ("fixed1.8.0.10" or "fixed1.8.1.2" bugs, no "needs testcase" in status whitebooard, and is marked "security-sensitive")
Note: If you run into a difficult bug, just add the "[needs testcase]" in the status whiteboard and move on.
Need Verification
- bc's js tests
https://bugzilla.mozilla.org/show_bug.cgi?id=357398 https://bugzilla.mozilla.org/show_bug.cgi?id=357392
Cannot/Won't Verify
- Needs testcase or difficult to reproduce:
https://bugzilla.mozilla.org/show_bug.cgi?id=364323 https://bugzilla.mozilla.org/show_bug.cgi?id=360526 https://bugzilla.mozilla.org/show_bug.cgi?id=361781 https://bugzilla.mozilla.org/show_bug.cgi?id=362110 (need to run test in JS Shell) https://bugzilla.mozilla.org/show_bug.cgi?id=362547 (need xpcshell to run test) https://bugzilla.mozilla.org/show_bug.cgi?id=363849 (need JAWS for accessibility testcase)
- Verified by code inspection (should revisit later to confirm):
https://bugzilla.mozilla.org/show_bug.cgi?id=361332
- Non-Firefox bugs (Camino, Sunbird, Thunderbird, etc.):
https://bugzilla.mozilla.org/show_bug.cgi?id=365248
https://bugzilla.mozilla.org/show_bug.cgi?id=362186 (tried to test with TB, but didn't have any luck, asked reporter to retest).
https://bugzilla.mozilla.org/show_bug.cgi?id=363043 (minor change for Sunbird only)