Releases/Firefox 6/Test Plan: Difference between revisions

Latest revision as of 16:03, 16 August 2011

« mozilla-beta:fx6 | Firefox 6 | Firefox 7 »

Firefox 6 Test Plan

Release: Tuesday, August 16, 2011
QA Contact: ~~Juan Becerra~~ Anthony Hughes
IRC: irc://irc.mozilla.org#qa (juanb, ashughes, whimboo)
Builds: Firefox 6.0

Tasks

The following tasks need to be completed to sign-off the release.

[DONE] [whimboo] Automated BFTs: details
[DONE] [softvision] Manual Testing: details
[DONE] [softvision] Feature sign-off: details
[DONE] [softvision] Bug Verifications: details
[DONE] [ashughes] Automated Updates: details
Docs: timeline, softvision testplan

Spotchecks should be performed across all supported operating systems, including Mac OSX 10.7:

Windows 2000, XP, Vista, 7, 7 64bit
Mac OS X 10.5, 10.6, 10.7
Ubuntu 11.04, 11.04 64-bit

If you find any issues use [fx6] whiteboard entry of the bug to keep track.

Automation

The following are the detailed results of the automated BFT test runs.

[DONE] Windows 2000: normal | restart
[DONE] Windows XP: normal | restart
[DONE] Windows Vista: normal | restart
[DONE] Windows 7: normal | restart
[DONE] Windows 7 64-bit: normal | restart
[DONE] Mac OS X 10.6: normal | restart
[DONE] Mac OS X 10.7: normal | restart
[DONE] Ubuntu 11.04: normal | restart
[DONE] Ubuntu 11.04 64-bit (x86): normal | restart
[DONE] Ubuntu 11.04 64-bit (x64): normal | restart
Litmus: Sync results to here
Mozmill: results

Issues Found

Manual

The following are the detailed results of the manual smoketests.

[DONE] [Alin] Windows 2000: PASS
[DONE] [Mihaela] Windows XP: PASS
[DONE] [Simona] Windows Vista: PASS
[DONE] [Simona] Windows 7: PASS
[DONE] [AndreiD] Windows 7 64-bit: PASS
[DONE] [VladG] Mac OS X 10.6: PASS
[DONE] [VladG] Mac OS X 10.7: PASS
[DONE] [Virgil] Ubuntu 11.04: PASS
[DONE] [AndreiD] Ubuntu 11.04 32-bit: PASS
Litmus testrun, report
Firefox 6 builds

Issues Found

No issues were found

Exploratory Testing

Conduct exploratory tests on WebGL, Javascript, Flash, and QuickTime plugin content
Tested plugin content using the following testing matrix: testing matrix
- [DONE] [Mihaela] Windows XP: PASS with known issues
- [DONE] [Alin] Windows 7: PASS with NEW issues
- [DONE] [VladG] Mac OS X 10.6: PASS with known issues
- [DONE] [SimonaB] Windows Vista: PASS with known issues
- [DONE] [Virgil] Ubuntu 11.04: PASS with known issues

New Issues Found

bug 678482 - Scrolling in web pages with java and shockwave scrambles the content

Feature sign-off

The following are the details about spotchecks performed on landed features

[DONE] [andreid] 50ms Assert: OK for release
[DONE] [vladmaniac] PlugincheckLinkInAOM: SIGN-OFF
[DONE] [tracy] Sync Discoverability: SIGN-OFF
[DONE] [george] Persist Panorama Groups: SIGN-OFF
[DONE] [george] Panorama Groups On Demand: SIGN-OFF
[DONE] [george] Site-based data management UI: SIGN-OFF
[DONE] [george] windows-scrolling: SIGN-OFF
[DONE] [simona] Locationbar Domain Highlight: SIGN-OFF
[DONE] [simona] WebDeveloper Menu: SIGN-OFF
[DONE] [vladg] RevampAboutMemory: SIGN-OFF
[DONE] [vladg] Progress Element: SIGN-OFF
[DONE] [vladmaniac] Autocomplete Popup: SIGN-OFF
[DONE] [vladmaniac] WebConsoleUpgrade: SIGN-OFF
[DONE] [alex] Scratchpad: SIGN-OFF

Issues Found

Bug Verifications

The following are the detailed results of bug fix verifications for Firefox 6.

Bugzilla: tracking-firefox6+
Results: VERIFIED, REOPENED

Issues Found

[DONE] bug 677510 - Video Controls buttons "Mute/unmute" and "Expand/Shrink" are lacking icons on HTML5 videos
- Issue has been fixed on YouTube's end
[DONE] bug 677095 - 1px background-image miss-handling when position is fixed
- Issue will be rel-noted and fixed in Firefox 7
bug 675260 - Crash in arena_dalloc_small coming from ShutdownNSS
- Not a priority for Firefox 6 -- If you have free cycles, we need help identifying builds affected and steps to reproduce

Updates

The following are details of update verifications for Firefox 6

Coverage

Platforms: Windows (2000, XP, Vista, 7, 7 64-bit), Mac OS X (10.6, 10.7), Ubuntu (11.04, 11.04 64-bit)
Updates: partial, complete, fallback
Locales: en-US, random
Versions: Firefox 4*, 5*, 6b*
Target Build: Firefox 6.0 (20110811165603)

Results

Betatest: report (l10n)

[DONE] Windows XP: SIGNED-OFF
[DONE] Windows Vista: SIGNED-OFF
[DONE] Windows 7: SIGNED-OFF
[DONE] Windows 7 64-bit: SIGNED-OFF
[DONE] Mac OS X 10.6: SIGNED-OFF
[DONE] Mac OS X 10.7: SIGNED-OFF
[DONE] Ubuntu 11.04: SIGNED-OFF
[DONE] Ubuntu 11.04 64-bit: SIGNED-OFF

Beta: report (l10n)

[DONE] Windows XP: SIGNED-OFF
[DONE] Windows Vista: SIGNED-OFF
[DONE] Windows 7: SIGNED-OFF
[DONE] Windows 7 64-bit: SIGNED-OFF
[DONE] Mac OS X 10.6: SIGNED-OFF
[DONE] Mac OS X 10.7: SIGNED-OFF
[DONE] Ubuntu 11.04: SIGNED-OFF
[DONE] Ubuntu 11.04 64-bit: SIGNED-OFF

Releasetest: report

[DONE] Windows XP: PASS
[DONE] Windows Vista: PASS
[DONE] Windows 7: PASS
[DONE] Windows 7 64-bit: PASS
[DONE] Mac OS X 10.6: PASS
[DONE] Mac OS X 10.7: PASS
[DONE] Ubuntu 11.04: PASS
[DONE] Ubuntu 11.04 64-bit: PASS

Release: report

[DONE] Windows XP: PASS
[DONE] Windows Vista: PASS
[DONE] Windows 7: PASS
[DONE] Windows 7 64-bit: PASS
[DONE] Mac OS X 10.6: PASS
[DONE] Mac OS X 10.7: PASS
[DONE] Ubuntu 11.04: PASS
[DONE] Ubuntu 11.04 64-bit: PASS

Issues Found

@@ Line 9: / Line 9: @@
 The following tasks need to be completed to sign-off the release.
-* {{|}} [ashughes] Automated BFTs: [[QA/Firefox_6#Automation|details]]
+* {{done|}} [whimboo] Automated BFTs: [[Releases/Firefox_6/Test_Plan#Automation|details]]
-* {{|}} [softvision] Manual Testing: [[QA/Firefox_6#Manual|details]]
+* {{done|}} [softvision] Manual Testing: [[Releases/Firefox_6/Test_Plan#Manual|details]]
-* {{|}} [softvision] Feature sign-off: [[QA/Firefox_6#Feature_sign-off|details]]
+* {{done|}} [softvision] Feature sign-off: [[Releases/Firefox_6/Test_Plan#Feature_sign-off|details]]
-* {{|}} [softvision] Bug Verifications: [[QA/Firefox_6#Bug_Verifications|details]]
+* {{done|}} [softvision] Bug Verifications: [[Releases/Firefox_6/Test_Plan#Bug_Verifications|details]]
-* {{|}} [ashughes] Automated Updates: [[QA/Firefox_6#Updates|details]]
+* {{done|}} [ashughes] Automated Updates: [[Releases/Firefox_6/Test_Plan#Updates|details]]
 * '''Docs''': [[QA/Firefox/Merge_Plan_2011-08-16|timeline]], [[QA_SoftVision_Team/Firefox-6-release-testplan|softvision testplan]]
@@ Line 55: / Line 55: @@
 ; Issues Found
+*No issues were found
 ; Exploratory Testing
@@ Line 62: / Line 62: @@
 ** {{done|}} [Mihaela] Windows XP: PASS with known issues
 ** {{done|}} [Alin] Windows 7: PASS with NEW issues
-** {{ok|}} [VladG] Mac OS X 10.6: status
+** {{done|}} [VladG] Mac OS X 10.6: PASS with known issues
 ** {{done|}} [SimonaB] Windows Vista: PASS with known issues
 ** {{done|}} [Virgil] Ubuntu 11.04: PASS with known issues
 ; New Issues Found
-*[https://bugzilla.mozilla.org/show_bug.cgi?id=638770 Bug 678482 - Scrolling in web pages with java and shockwave scrambles the content]
+* {{bug|678482}} - Scrolling in web pages with java and shockwave scrambles the content
 ===Feature sign-off===
@@ Line 109: / Line 109: @@
 * Locales: en-US, random
 * Versions: Firefox 4*, 5*, 6b*
-* Target Build:
+* Target Build: Firefox 6.0 (20110811165603)
 ; Results
-Betatest: report
+Betatest: [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=betatest&from=2011-08-12&to=2011-08-12&target=6.0 report] ([http://mozmill-archive.brasstacks.mozilla.com/#/update/reports?branch=6.0&platform=Mac&from=2011-08-12&to=2011-08-12 l10n])
-* {{|}} Windows XP: status
+* {{done|}} Windows XP: SIGNED-OFF
-* {{|}} Windows Vista: status
+* {{done|}} Windows Vista: SIGNED-OFF
-* {{|}} Windows 7: status
+* {{done|}} Windows 7: SIGNED-OFF
-* {{|}} Windows 7 64-bit: status
+* {{done|}} Windows 7 64-bit: SIGNED-OFF
-* {{|}} Mac OS X 10.6: status
+* {{done|}} Mac OS X 10.6: SIGNED-OFF
-* {{|}} Mac OS X 10.7: status
+* {{done|}} Mac OS X 10.7: SIGNED-OFF
-* {{|}} Ubuntu 11.04: status
+* {{done|}} Ubuntu 11.04: SIGNED-OFF
-* {{|}} Ubuntu 11.04 64-bit: status
+* {{done|}} Ubuntu 11.04 64-bit: SIGNED-OFF
-Beta: report
+Beta: [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=beta&from=2011-08-12&to=2011-08-12&target=6.0 report] ([http://mozmill-archive.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=beta&from=2011-08-12&to=2011-08-12&target=6.0 l10n])
-* {{|}} Windows XP: status
+* {{done|}} Windows XP: SIGNED-OFF
-* {{|}} Windows Vista: status
+* {{done|}} Windows Vista: SIGNED-OFF
-* {{|}} Windows 7: status
+* {{done|}} Windows 7: SIGNED-OFF
-* {{|}} Windows 7 64-bit: status
+* {{done|}} Windows 7 64-bit: SIGNED-OFF
-* {{|}} Mac OS X 10.6: status
+* {{done|}} Mac OS X 10.6: SIGNED-OFF
-* {{|}} Mac OS X 10.7: status
+* {{done|}} Mac OS X 10.7: SIGNED-OFF
-* {{|}} Ubuntu 11.04: status
+* {{done|}} Ubuntu 11.04: SIGNED-OFF
-* {{|}} Ubuntu 11.04 64-bit: status
+* {{done|}} Ubuntu 11.04 64-bit: SIGNED-OFF
-Releasetest: report
+Releasetest: [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=releasetest&from=2011-08-15&to=2011-08-16&target=6.0 report]
-* {{|}} Windows XP: status
+* {{done|}} Windows XP: '''PASS'''
-* {{|}} Windows Vista: status
+* {{done|}} Windows Vista: '''PASS'''
-* {{|}} Windows 7: status
+* {{done|}} Windows 7: '''PASS'''
-* {{|}} Windows 7 64-bit: status
+* {{done|}} Windows 7 64-bit: '''PASS'''
-* {{|}} Mac OS X 10.6: status
+* {{done|}} Mac OS X 10.6: '''PASS'''
-* {{|}} Mac OS X 10.7: status
+* {{done|}} Mac OS X 10.7: '''PASS'''
-* {{|}} Ubuntu 11.04: status
+* {{done|}} Ubuntu 11.04: '''PASS'''
-* {{|}} Ubuntu 11.04 64-bit: status
+* {{done|}} Ubuntu 11.04 64-bit: '''PASS'''
-Release: report
+Release: [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=release&from=2011-08-16&to=2011-08-16&target=6.0 report]
-* {{|}} Windows XP: status
+* {{done|}} Windows XP: '''PASS'''
-* {{|}} Windows Vista: status
+* {{done|}} Windows Vista: '''PASS'''
-* {{|}} Windows 7: status
+* {{done|}} Windows 7: '''PASS'''
-* {{|}} Windows 7 64-bit: status
+* {{done|}} Windows 7 64-bit: '''PASS'''
-* {{|}} Mac OS X 10.6: status
+* {{done|}} Mac OS X 10.6: '''PASS'''
-* {{|}} Mac OS X 10.7: status
+* {{done|}} Mac OS X 10.7: '''PASS'''
-* {{|}} Ubuntu 11.04: status
+* {{done|}} Ubuntu 11.04: '''PASS'''
-* {{|}} Ubuntu 11.04 64-bit: status
+* {{done|}} Ubuntu 11.04 64-bit: '''PASS'''
 ; Issues Found