Releases/Firefox 6/Test Plan: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(23 intermediate revisions by 4 users not shown)
Line 9: Line 9:
The following tasks need to be completed to sign-off the release.
The following tasks need to be completed to sign-off the release.


* {{|}} [ashughes] Automated BFTs: [[QA/Firefox_6#Automation|details]]
* {{done|}} [whimboo] Automated BFTs: [[Releases/Firefox_6/Test_Plan#Automation|details]]
* {{|}} [softvision] Manual Testing: [[QA/Firefox_6#Manual|details]]
* {{done|}} [softvision] Manual Testing: [[Releases/Firefox_6/Test_Plan#Manual|details]]
* {{|}} [softvision] Feature sign-off: [[QA/Firefox_6#Feature_sign-off|details]]
* {{done|}} [softvision] Feature sign-off: [[Releases/Firefox_6/Test_Plan#Feature_sign-off|details]]
* {{|}} [softvision] Bug Verifications: [[QA/Firefox_6#Bug_Verifications|details]]
* {{done|}} [softvision] Bug Verifications: [[Releases/Firefox_6/Test_Plan#Bug_Verifications|details]]
* {{|}} [ashughes] Automated Updates: [[QA/Firefox_6#Updates|details]]
* {{done|}} [ashughes] Automated Updates: [[Releases/Firefox_6/Test_Plan#Updates|details]]
* '''Docs''': [[QA/Firefox/Merge_Plan_2011-08-16|timeline]], [[QA_SoftVision_Team/Firefox-6-release-testplan|softvision testplan]]
* '''Docs''': [[QA/Firefox/Merge_Plan_2011-08-16|timeline]], [[QA_SoftVision_Team/Firefox-6-release-testplan|softvision testplan]]


Line 25: Line 25:
=== Automation ===
=== Automation ===
The following are the detailed results of the automated BFT test runs.
The following are the detailed results of the automated BFT test runs.
* {{|}} Windows 2000: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9797bd normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97aecf restart]
* {{done|}} Windows 2000: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9797bd normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97aecf restart]
* {{|}} Windows XP: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9de820 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9de8c3 restart]
* {{done|}} Windows XP: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9de820 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9de8c3 restart]
* {{|}} Windows Vista: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97a259 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97bac9 restart]
* {{done|}} Windows Vista: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97a259 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97bac9 restart]
* {{|}} Windows 7: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f976f64 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9778ce restart]
* {{done|}} Windows 7: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f976f64 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9778ce restart]
* {{|}} Windows 7 64-bit: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97786f normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f978743 restart]
* {{done|}} Windows 7 64-bit: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97786f normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f978743 restart]
* {{|}} Mac OS X 10.6: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97a565 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97beb0 restart]
* {{done|}} Mac OS X 10.6: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97a565 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97beb0 restart]
* {{|}} Mac OS X 10.7: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97cbf3 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97d82a restart]
* {{done|}} Mac OS X 10.7: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97cbf3 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97d82a restart]
* {{|}} Ubuntu 11.04: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9e7bd8 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9e8b96 restart]
* {{done|}} Ubuntu 11.04: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9e7bd8 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9e8b96 restart]
* {{|}} Ubuntu 11.04 64-bit (x86): [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9e7392 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9e8030 restart]
* {{done|}} Ubuntu 11.04 64-bit (x86): [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9e7392 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9e8030 restart]
* {{|}} Ubuntu 11.04 64-bit (x64): [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9ea428 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9eae1d restart]
* {{done|}} Ubuntu 11.04 64-bit (x64): [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9ea428 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9eae1d restart]
* '''Litmus''': Sync results to [https://litmus.mozilla.org/run_tests.cgi?test_run_id=331 here]
* '''Litmus''': Sync results to [https://litmus.mozilla.org/run_tests.cgi?test_run_id=331 here]
* '''Mozmill''': [http://mozmill-release.brasstacks.mozilla.com/#/functional/reports?branch=6.0&platform=All&from=2011-08-12&to=2011-08-12 results]
* '''Mozmill''': [http://mozmill-release.brasstacks.mozilla.com/#/functional/reports?branch=6.0&platform=All&from=2011-08-12&to=2011-08-12 results]
Line 55: Line 55:


; Issues Found
; Issues Found
 
*No issues were found


; Exploratory Testing
; Exploratory Testing
* Conduct exploratory tests on WebGL, Javascript, Flash, and QuickTime plugin content
* Conduct exploratory tests on WebGL, Javascript, Flash, and QuickTime plugin content
* Tested plugin content using the following testing matrix: [http://bit.ly/q0dKtA testing matrix]
* Tested plugin content using the following testing matrix: [http://bit.ly/q0dKtA testing matrix]
** {{|}} [tester] Windows XP: status
** {{done|}} [Mihaela] Windows XP: PASS with known issues
** {{|}} [tester] Windows 7: status
** {{done|}} [Alin] Windows 7: PASS with NEW issues
** {{|}} [VladG] Mac OS X 10.6: status
** {{done|}} [VladG] Mac OS X 10.6: PASS with known issues
** {{|}} [tester] Ubuntu 11.04: status
** {{done|}} [SimonaB] Windows Vista: PASS with known issues
** {{done|}} [Virgil] Ubuntu 11.04: PASS with known issues


; Issues Found
; New Issues Found
*[https://bugzilla.mozilla.org/show_bug.cgi?id=638741 Bug 638741 Right click on flash content displays two context menus sometimes]
* {{bug|678482}} - Scrolling in web pages with java and shockwave scrambles the content


===Feature sign-off===
===Feature sign-off===
The following are the details about spotchecks performed on landed features
The following are the details about spotchecks performed on landed features
* {{done|}} [andreid] [https://wiki.mozilla.org/Firefox/Features/50ms_ASSERT 50ms Assert]: OK for release
* {{done|}} [andreid] [https://wiki.mozilla.org/Firefox/Features/50ms_ASSERT 50ms Assert]: OK for release
* {{|}} [whimboo] [https://wiki.mozilla.org/Features/Firefox/PlugincheckLinkInAOM PlugincheckLinkInAOM]: status
* {{done|}} [vladmaniac] [https://wiki.mozilla.org/Features/Firefox/PlugincheckLinkInAOM PlugincheckLinkInAOM]: SIGN-OFF
* {{done|}} [tracy] [https://wiki.mozilla.org/Firefox/Features/Sync_Discoverability Sync Discoverability]: SIGN-OFF
* {{done|}} [tracy] [https://wiki.mozilla.org/Firefox/Features/Sync_Discoverability Sync Discoverability]: SIGN-OFF
* {{done|}} [george] [https://wiki.mozilla.org/Firefox/Features/Persist_Panorama_Groups Persist Panorama Groups]: SIGN-OFF
* {{done|}} [george] [https://wiki.mozilla.org/Firefox/Features/Persist_Panorama_Groups Persist Panorama Groups]: SIGN-OFF
Line 81: Line 82:
* {{done|}} [vladg] [https://wiki.mozilla.org/Platform/Features/RevampAboutMemory RevampAboutMemory]: SIGN-OFF
* {{done|}} [vladg] [https://wiki.mozilla.org/Platform/Features/RevampAboutMemory RevampAboutMemory]: SIGN-OFF
* {{done|}} [vladg] [https://wiki.mozilla.org/Platform/Features/Progress_Element Progress Element]: SIGN-OFF
* {{done|}} [vladg] [https://wiki.mozilla.org/Platform/Features/Progress_Element Progress Element]: SIGN-OFF
* {{|}} [vladmaniac] [https://wiki.mozilla.org/DevTools/Features/AutocompletionPopup Autocomplete Popup]: status
* {{done|}} [vladmaniac] [https://wiki.mozilla.org/DevTools/Features/AutocompletionPopup Autocomplete Popup]: SIGN-OFF
* {{|}} [vladmaniac] [https://wiki.mozilla.org/DevTools/Features/WebConsoleUpgrade WebConsoleUpgrade]: status
* {{done|}} [vladmaniac] [https://wiki.mozilla.org/DevTools/Features/WebConsoleUpgrade WebConsoleUpgrade]: SIGN-OFF
* {{done|}} [alex] [https://wiki.mozilla.org/DevTools/Features/Workspaces Scratchpad]: SIGN-OFF
* {{done|}} [alex] [https://wiki.mozilla.org/DevTools/Features/Workspaces Scratchpad]: SIGN-OFF


Line 108: Line 109:
* Locales: en-US, random
* Locales: en-US, random
* Versions: Firefox 4*, 5*, 6b*
* Versions: Firefox 4*, 5*, 6b*
* Target Build:  
* Target Build: Firefox 6.0 (20110811165603)


; Results
; Results
Betatest: report
Betatest: [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=betatest&from=2011-08-12&to=2011-08-12&target=6.0 report] ([http://mozmill-archive.brasstacks.mozilla.com/#/update/reports?branch=6.0&platform=Mac&from=2011-08-12&to=2011-08-12 l10n])
* {{|}} Windows XP: status
* {{done|}} Windows XP: SIGNED-OFF
* {{|}} Windows Vista: status
* {{done|}} Windows Vista: SIGNED-OFF
* {{|}} Windows 7: status
* {{done|}} Windows 7: SIGNED-OFF
* {{|}} Windows 7 64-bit: status
* {{done|}} Windows 7 64-bit: SIGNED-OFF
* {{|}} Mac OS X 10.6: status
* {{done|}} Mac OS X 10.6: SIGNED-OFF
* {{|}} Mac OS X 10.7: status
* {{done|}} Mac OS X 10.7: SIGNED-OFF
* {{|}} Ubuntu 11.04: status
* {{done|}} Ubuntu 11.04: SIGNED-OFF
* {{|}} Ubuntu 11.04 64-bit: status
* {{done|}} Ubuntu 11.04 64-bit: SIGNED-OFF


Beta: report
Beta: [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=beta&from=2011-08-12&to=2011-08-12&target=6.0 report] ([http://mozmill-archive.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=beta&from=2011-08-12&to=2011-08-12&target=6.0 l10n])
* {{|}} Windows XP: status
* {{done|}} Windows XP: SIGNED-OFF
* {{|}} Windows Vista: status
* {{done|}} Windows Vista: SIGNED-OFF
* {{|}} Windows 7: status
* {{done|}} Windows 7: SIGNED-OFF
* {{|}} Windows 7 64-bit: status
* {{done|}} Windows 7 64-bit: SIGNED-OFF
* {{|}} Mac OS X 10.6: status
* {{done|}} Mac OS X 10.6: SIGNED-OFF
* {{|}} Mac OS X 10.7: status
* {{done|}} Mac OS X 10.7: SIGNED-OFF
* {{|}} Ubuntu 11.04: status
* {{done|}} Ubuntu 11.04: SIGNED-OFF
* {{|}} Ubuntu 11.04 64-bit: status
* {{done|}} Ubuntu 11.04 64-bit: SIGNED-OFF


Releasetest: report
Releasetest: [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=releasetest&from=2011-08-15&to=2011-08-16&target=6.0 report]
* {{|}} Windows XP: status
* {{done|}} Windows XP: '''PASS'''
* {{|}} Windows Vista: status
* {{done|}} Windows Vista: '''PASS'''
* {{|}} Windows 7: status
* {{done|}} Windows 7: '''PASS'''
* {{|}} Windows 7 64-bit: status
* {{done|}} Windows 7 64-bit: '''PASS'''
* {{|}} Mac OS X 10.6: status
* {{done|}} Mac OS X 10.6: '''PASS'''
* {{|}} Mac OS X 10.7: status
* {{done|}} Mac OS X 10.7: '''PASS'''
* {{|}} Ubuntu 11.04: status
* {{done|}} Ubuntu 11.04: '''PASS'''
* {{|}} Ubuntu 11.04 64-bit: status
* {{done|}} Ubuntu 11.04 64-bit: '''PASS'''


Release: report
Release: [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=release&from=2011-08-16&to=2011-08-16&target=6.0 report]
* {{|}} Windows XP: status
* {{done|}} Windows XP: '''PASS'''
* {{|}} Windows Vista: status
* {{done|}} Windows Vista: '''PASS'''
* {{|}} Windows 7: status
* {{done|}} Windows 7: '''PASS'''
* {{|}} Windows 7 64-bit: status
* {{done|}} Windows 7 64-bit: '''PASS'''
* {{|}} Mac OS X 10.6: status
* {{done|}} Mac OS X 10.6: '''PASS'''
* {{|}} Mac OS X 10.7: status
* {{done|}} Mac OS X 10.7: '''PASS'''
* {{|}} Ubuntu 11.04: status
* {{done|}} Ubuntu 11.04: '''PASS'''
* {{|}} Ubuntu 11.04 64-bit: status
* {{done|}} Ubuntu 11.04 64-bit: '''PASS'''


; Issues Found
; Issues Found

Latest revision as of 16:03, 16 August 2011

« mozilla-beta:fx6 | Firefox 6 | Firefox 7 »

Firefox 6 Test Plan

Tasks

The following tasks need to be completed to sign-off the release.

Spotchecks should be performed across all supported operating systems, including Mac OSX 10.7:

  • Windows 2000, XP, Vista, 7, 7 64bit
  • Mac OS X 10.5, 10.6, 10.7
  • Ubuntu 11.04, 11.04 64-bit

If you find any issues use [fx6] whiteboard entry of the bug to keep track.

Automation

The following are the detailed results of the automated BFT test runs.

Issues Found

Manual

The following are the detailed results of the manual smoketests.

  • [DONE] [Alin] Windows 2000: PASS
  • [DONE] [Mihaela] Windows XP: PASS
  • [DONE] [Simona] Windows Vista: PASS
  • [DONE] [Simona] Windows 7: PASS
  • [DONE] [AndreiD] Windows 7 64-bit: PASS
  • [DONE] [VladG] Mac OS X 10.6: PASS
  • [DONE] [VladG] Mac OS X 10.7: PASS
  • [DONE] [Virgil] Ubuntu 11.04: PASS
  • [DONE] [AndreiD] Ubuntu 11.04 32-bit: PASS
  • Litmus testrun, report
  • Firefox 6 builds
Issues Found
  • No issues were found
Exploratory Testing
  • Conduct exploratory tests on WebGL, Javascript, Flash, and QuickTime plugin content
  • Tested plugin content using the following testing matrix: testing matrix
    • [DONE] [Mihaela] Windows XP: PASS with known issues
    • [DONE] [Alin] Windows 7: PASS with NEW issues
    • [DONE] [VladG] Mac OS X 10.6: PASS with known issues
    • [DONE] [SimonaB] Windows Vista: PASS with known issues
    • [DONE] [Virgil] Ubuntu 11.04: PASS with known issues
New Issues Found
  • bug 678482 - Scrolling in web pages with java and shockwave scrambles the content

Feature sign-off

The following are the details about spotchecks performed on landed features

Issues Found

Bug Verifications

The following are the detailed results of bug fix verifications for Firefox 6.

Issues Found
  • [DONE] bug 677510 - Video Controls buttons "Mute/unmute" and "Expand/Shrink" are lacking icons on HTML5 videos
    • Issue has been fixed on YouTube's end
  • [DONE] bug 677095 - 1px background-image miss-handling when position is fixed
    • Issue will be rel-noted and fixed in Firefox 7
  • bug 675260 - Crash in arena_dalloc_small coming from ShutdownNSS
    • Not a priority for Firefox 6 -- If you have free cycles, we need help identifying builds affected and steps to reproduce

Updates

The following are details of update verifications for Firefox 6

Coverage
  • Platforms: Windows (2000, XP, Vista, 7, 7 64-bit), Mac OS X (10.6, 10.7), Ubuntu (11.04, 11.04 64-bit)
  • Updates: partial, complete, fallback
  • Locales: en-US, random
  • Versions: Firefox 4*, 5*, 6b*
  • Target Build: Firefox 6.0 (20110811165603)
Results

Betatest: report (l10n)

  • [DONE] Windows XP: SIGNED-OFF
  • [DONE] Windows Vista: SIGNED-OFF
  • [DONE] Windows 7: SIGNED-OFF
  • [DONE] Windows 7 64-bit: SIGNED-OFF
  • [DONE] Mac OS X 10.6: SIGNED-OFF
  • [DONE] Mac OS X 10.7: SIGNED-OFF
  • [DONE] Ubuntu 11.04: SIGNED-OFF
  • [DONE] Ubuntu 11.04 64-bit: SIGNED-OFF

Beta: report (l10n)

  • [DONE] Windows XP: SIGNED-OFF
  • [DONE] Windows Vista: SIGNED-OFF
  • [DONE] Windows 7: SIGNED-OFF
  • [DONE] Windows 7 64-bit: SIGNED-OFF
  • [DONE] Mac OS X 10.6: SIGNED-OFF
  • [DONE] Mac OS X 10.7: SIGNED-OFF
  • [DONE] Ubuntu 11.04: SIGNED-OFF
  • [DONE] Ubuntu 11.04 64-bit: SIGNED-OFF

Releasetest: report

  • [DONE] Windows XP: PASS
  • [DONE] Windows Vista: PASS
  • [DONE] Windows 7: PASS
  • [DONE] Windows 7 64-bit: PASS
  • [DONE] Mac OS X 10.6: PASS
  • [DONE] Mac OS X 10.7: PASS
  • [DONE] Ubuntu 11.04: PASS
  • [DONE] Ubuntu 11.04 64-bit: PASS

Release: report

  • [DONE] Windows XP: PASS
  • [DONE] Windows Vista: PASS
  • [DONE] Windows 7: PASS
  • [DONE] Windows 7 64-bit: PASS
  • [DONE] Mac OS X 10.6: PASS
  • [DONE] Mac OS X 10.7: PASS
  • [DONE] Ubuntu 11.04: PASS
  • [DONE] Ubuntu 11.04 64-bit: PASS
Issues Found
Retrieved from "https://wiki.mozilla.org/index.php?title=Releases/Firefox_6/Test_Plan&oldid=340175"