Confirmed users
14,525
edits
Releases/Firefox 6/Test Plan: Difference between revisions
→Updates
| (45 intermediate revisions by 10 users not shown) | |||
| Line 1: | Line 1: | ||
<small>[[QA/mozilla-beta/fx6|« mozilla-beta:fx6]] | [[Releases/Firefox_6/Test_Plan|Firefox 6]] | [[ | <small>[[QA/mozilla-beta/fx6|« mozilla-beta:fx6]] | [[Releases/Firefox_6/Test_Plan|Firefox 6]] | [[Releases/Firefox_7/Test_Plan|Firefox 7 »]] </small> | ||
== Firefox 6 Test Plan == | == Firefox 6 Test Plan == | ||
* Release: Tuesday, August 16, 2011 | * Release: Tuesday, August 16, 2011 | ||
| Line 9: | Line 9: | ||
The following tasks need to be completed to sign-off the release. | The following tasks need to be completed to sign-off the release. | ||
* {{|}} [ | * {{done|}} [whimboo] Automated BFTs: [[Releases/Firefox_6/Test_Plan#Automation|details]] | ||
* {{|}} [softvision] Manual Testing: [[ | * {{done|}} [softvision] Manual Testing: [[Releases/Firefox_6/Test_Plan#Manual|details]] | ||
* {{|}} [softvision] Feature sign-off: [[ | * {{done|}} [softvision] Feature sign-off: [[Releases/Firefox_6/Test_Plan#Feature_sign-off|details]] | ||
* {{|}} [softvision] Bug Verifications: [[ | * {{done|}} [softvision] Bug Verifications: [[Releases/Firefox_6/Test_Plan#Bug_Verifications|details]] | ||
* {{|}} [ashughes] Automated Updates: [[ | * {{done|}} [ashughes] Automated Updates: [[Releases/Firefox_6/Test_Plan#Updates|details]] | ||
* '''Docs''': [[QA/Firefox/Merge_Plan_2011-08-16|timeline]], [[QA_SoftVision_Team/Firefox-6-release-testplan|softvision testplan]] | * '''Docs''': [[QA/Firefox/Merge_Plan_2011-08-16|timeline]], [[QA_SoftVision_Team/Firefox-6-release-testplan|softvision testplan]] | ||
| Line 25: | Line 25: | ||
=== Automation === | === Automation === | ||
The following are the detailed results of the automated BFT test runs. | The following are the detailed results of the automated BFT test runs. | ||
* {{|}} Windows 2000: [ http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9797bd normal] | [ http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97aecf restart] | * {{done|}} Windows 2000: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9797bd normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97aecf restart] | ||
* {{|}} Windows XP: [ http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9de820 normal] | [ http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9de8c3 restart] | * {{done|}} Windows XP: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9de820 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9de8c3 restart] | ||
* {{|}} Windows Vista: [ http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97a259 normal] | [ http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97bac9 restart] | * {{done|}} Windows Vista: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97a259 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97bac9 restart] | ||
* {{|}} Windows 7: [ http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f976f64 normal] | [ http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9778ce restart] | * {{done|}} Windows 7: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f976f64 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9778ce restart] | ||
* {{|}} Windows 7 64-bit: [ http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97786f normal] | [ http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f978743 restart] | * {{done|}} Windows 7 64-bit: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97786f normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f978743 restart] | ||
* {{|}} Mac OS X 10.6: [ http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97a565 normal] | [ http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97beb0 restart] | * {{done|}} Mac OS X 10.6: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97a565 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97beb0 restart] | ||
* {{|}} Mac OS X 10.7: [ http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97cbf3 normal] | [ http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97d82a restart] | * {{done|}} Mac OS X 10.7: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97cbf3 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97d82a restart] | ||
* {{|}} Ubuntu 11.04: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9e7bd8 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9e8b96 restart] | * {{done|}} Ubuntu 11.04: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9e7bd8 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9e8b96 restart] | ||
* {{|}} Ubuntu 11.04 64-bit (x86): [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9e7392 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9e8030 restart] | * {{done|}} Ubuntu 11.04 64-bit (x86): [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9e7392 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9e8030 restart] | ||
* {{|}} Ubuntu 11.04 64-bit (x64): [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9ea428 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9eae1d restart] | * {{done|}} Ubuntu 11.04 64-bit (x64): [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9ea428 normal] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9eae1d restart] | ||
* '''Litmus''': Sync results to [https://litmus.mozilla.org/run_tests.cgi?test_run_id=331 here] | * '''Litmus''': Sync results to [https://litmus.mozilla.org/run_tests.cgi?test_run_id=331 here] | ||
* '''Mozmill''': [ ] | * '''Mozmill''': [http://mozmill-release.brasstacks.mozilla.com/#/functional/reports?branch=6.0&platform=All&from=2011-08-12&to=2011-08-12 results] | ||
; Issues Found | ; Issues Found | ||
| Line 42: | Line 42: | ||
=== Manual === | === Manual === | ||
The following are the detailed results of the manual smoketests. | The following are the detailed results of the manual smoketests. | ||
* {{|}} [ | * {{done|}} [Alin] Windows 2000: PASS | ||
* {{|}} [ | * {{done|}} [Mihaela] Windows XP: PASS | ||
* {{|}} [ | * {{done|}} [Simona] Windows Vista: PASS | ||
* {{|}} [ | * {{done|}} [Simona] Windows 7: PASS | ||
* {{|}} [ | * {{done|}} [AndreiD] Windows 7 64-bit: PASS | ||
* {{|}} [VladG] Mac OS X 10.6: | * {{done|}} [VladG] Mac OS X 10.6: PASS | ||
* {{|}} [VladG] Mac OS X 10.7: | * {{done|}} [VladG] Mac OS X 10.7: PASS | ||
* {{|}} [ | * {{done|}} [Virgil] Ubuntu 11.04: PASS | ||
* {{|}} [ | * {{done|}} [AndreiD] Ubuntu 11.04 32-bit: PASS | ||
* '''Litmus''' [https://litmus.mozilla.org/run_tests.cgi?test_run_id=331 testrun], [https://litmus.mozilla.org/test_run_report.cgi?test_run_id=331 report] | * '''Litmus''' [https://litmus.mozilla.org/run_tests.cgi?test_run_id=331 testrun], [https://litmus.mozilla.org/test_run_report.cgi?test_run_id=331 report] | ||
* '''Firefox 6''' [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/6.0-candidates/build1/ builds] | * '''Firefox 6''' [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/6.0-candidates/build1/ builds] | ||
; Issues Found | ; Issues Found | ||
*No issues were found | |||
; Exploratory Testing | ; Exploratory Testing | ||
Conduct exploratory tests on WebGL, Javascript, Flash, and QuickTime plugin content | * Conduct exploratory tests on WebGL, Javascript, Flash, and QuickTime plugin content | ||
* {{|}} [ | * Tested plugin content using the following testing matrix: [http://bit.ly/q0dKtA testing matrix] | ||
* {{|}} [ | ** {{done|}} [Mihaela] Windows XP: PASS with known issues | ||
* {{|}} [ | ** {{done|}} [Alin] Windows 7: PASS with NEW issues | ||
* {{|}} [ | ** {{done|}} [VladG] Mac OS X 10.6: PASS with known issues | ||
** {{done|}} [SimonaB] Windows Vista: PASS with known issues | |||
** {{done|}} [Virgil] Ubuntu 11.04: PASS with known issues | |||
; Issues Found | ; New Issues Found | ||
* {{bug|678482}} - Scrolling in web pages with java and shockwave scrambles the content | |||
===Feature sign-off=== | ===Feature sign-off=== | ||
The following are the details about spotchecks performed on landed features | The following are the details about spotchecks performed on landed features | ||
* {{|}} [andreid] [https://wiki.mozilla.org/Firefox/Features/50ms_ASSERT 50ms Assert]: | * {{done|}} [andreid] [https://wiki.mozilla.org/Firefox/Features/50ms_ASSERT 50ms Assert]: OK for release | ||
* {{|}} [ | * {{done|}} [vladmaniac] [https://wiki.mozilla.org/Features/Firefox/PlugincheckLinkInAOM PlugincheckLinkInAOM]: SIGN-OFF | ||
* {{|}} [tracy] [https://wiki.mozilla.org/Firefox/Features/Sync_Discoverability Sync Discoverability]: | * {{done|}} [tracy] [https://wiki.mozilla.org/Firefox/Features/Sync_Discoverability Sync Discoverability]: SIGN-OFF | ||
* {{|}} [george] [https://wiki.mozilla.org/Firefox/Features/Persist_Panorama_Groups Persist Panorama Groups]: | * {{done|}} [george] [https://wiki.mozilla.org/Firefox/Features/Persist_Panorama_Groups Persist Panorama Groups]: SIGN-OFF | ||
* {{|}} [george] [https://wiki.mozilla.org/Firefox/Features/Panorama_Groups_On_Demand Panorama Groups On Demand]: | * {{done|}} [george] [https://wiki.mozilla.org/Firefox/Features/Panorama_Groups_On_Demand Panorama Groups On Demand]: SIGN-OFF | ||
* {{|}} [george] [https://wiki.mozilla.org/Privacy/Features/Site-based_data_management_UI Site-based data management UI]: | * {{done|}} [george] [https://wiki.mozilla.org/Privacy/Features/Site-based_data_management_UI Site-based data management UI]: SIGN-OFF | ||
* {{|}} [george] [https://wiki.mozilla.org/Firefox/Features/windows-scrolling windows-scrolling]: | * {{done|}} [george] [https://wiki.mozilla.org/Firefox/Features/windows-scrolling windows-scrolling]: SIGN-OFF | ||
* {{|}} [simona] [https://wiki.mozilla.org/Firefox/Features/Locationbar_Domain_Highlight Locationbar Domain Highlight]: | * {{done|}} [simona] [https://wiki.mozilla.org/Firefox/Features/Locationbar_Domain_Highlight Locationbar Domain Highlight]: SIGN-OFF | ||
* {{|}} [simona] [https://wiki.mozilla.org/DevTools/Features/WebDeveloperMenu WebDeveloper Menu]: | * {{done|}} [simona] [https://wiki.mozilla.org/DevTools/Features/WebDeveloperMenu WebDeveloper Menu]: SIGN-OFF | ||
* {{|}} [vladg] [https://wiki.mozilla.org/Platform/Features/RevampAboutMemory RevampAboutMemory]: | * {{done|}} [vladg] [https://wiki.mozilla.org/Platform/Features/RevampAboutMemory RevampAboutMemory]: SIGN-OFF | ||
* {{|}} [vladg] [https://wiki.mozilla.org/Platform/Features/Progress_Element Progress Element]: | * {{done|}} [vladg] [https://wiki.mozilla.org/Platform/Features/Progress_Element Progress Element]: SIGN-OFF | ||
* {{|}} [vladmaniac] [https://wiki.mozilla.org/DevTools/Features/AutocompletionPopup Autocomplete Popup]: | * {{done|}} [vladmaniac] [https://wiki.mozilla.org/DevTools/Features/AutocompletionPopup Autocomplete Popup]: SIGN-OFF | ||
* {{|}} [vladmaniac] [https://wiki.mozilla.org/DevTools/Features/WebConsoleUpgrade WebConsoleUpgrade]: | * {{done|}} [vladmaniac] [https://wiki.mozilla.org/DevTools/Features/WebConsoleUpgrade WebConsoleUpgrade]: SIGN-OFF | ||
* {{|}} [alex] [https://wiki.mozilla.org/DevTools/Features/Workspaces Scratchpad]: | * {{done|}} [alex] [https://wiki.mozilla.org/DevTools/Features/Workspaces Scratchpad]: SIGN-OFF | ||
; Issues Found | ; Issues Found | ||
| Line 106: | Line 109: | ||
* Locales: en-US, random | * Locales: en-US, random | ||
* Versions: Firefox 4*, 5*, 6b* | * Versions: Firefox 4*, 5*, 6b* | ||
* Target Build: | * Target Build: Firefox 6.0 (20110811165603) | ||
; Results | ; Results | ||
Betatest: report | Betatest: [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=betatest&from=2011-08-12&to=2011-08-12&target=6.0 report] ([http://mozmill-archive.brasstacks.mozilla.com/#/update/reports?branch=6.0&platform=Mac&from=2011-08-12&to=2011-08-12 l10n]) | ||
* {{|}} Windows XP: | * {{done|}} Windows XP: SIGNED-OFF | ||
* {{|}} Windows Vista: | * {{done|}} Windows Vista: SIGNED-OFF | ||
* {{|}} Windows 7: | * {{done|}} Windows 7: SIGNED-OFF | ||
* {{|}} Windows 7 64-bit: | * {{done|}} Windows 7 64-bit: SIGNED-OFF | ||
* {{|}} Mac OS X 10.6: | * {{done|}} Mac OS X 10.6: SIGNED-OFF | ||
* {{|}} Mac OS X 10.7: | * {{done|}} Mac OS X 10.7: SIGNED-OFF | ||
* {{|}} Ubuntu 11.04: | * {{done|}} Ubuntu 11.04: SIGNED-OFF | ||
* {{|}} Ubuntu 11.04 64-bit: | * {{done|}} Ubuntu 11.04 64-bit: SIGNED-OFF | ||
Beta: report | Beta: [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=beta&from=2011-08-12&to=2011-08-12&target=6.0 report] ([http://mozmill-archive.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=beta&from=2011-08-12&to=2011-08-12&target=6.0 l10n]) | ||
* {{|}} Windows XP: | * {{done|}} Windows XP: SIGNED-OFF | ||
* {{|}} Windows Vista: | * {{done|}} Windows Vista: SIGNED-OFF | ||
* {{|}} Windows 7: | * {{done|}} Windows 7: SIGNED-OFF | ||
* {{|}} Windows 7 64-bit: | * {{done|}} Windows 7 64-bit: SIGNED-OFF | ||
* {{|}} Mac OS X 10.6: | * {{done|}} Mac OS X 10.6: SIGNED-OFF | ||
* {{|}} Mac OS X 10.7: | * {{done|}} Mac OS X 10.7: SIGNED-OFF | ||
* {{|}} Ubuntu 11.04: | * {{done|}} Ubuntu 11.04: SIGNED-OFF | ||
* {{|}} Ubuntu 11.04 64-bit: | * {{done|}} Ubuntu 11.04 64-bit: SIGNED-OFF | ||
Releasetest: report | Releasetest: [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=releasetest&from=2011-08-15&to=2011-08-16&target=6.0 report] | ||
* {{|}} Windows XP: | * {{done|}} Windows XP: '''PASS''' | ||
* {{|}} Windows Vista: | * {{done|}} Windows Vista: '''PASS''' | ||
* {{|}} Windows 7: | * {{done|}} Windows 7: '''PASS''' | ||
* {{|}} Windows 7 64-bit: | * {{done|}} Windows 7 64-bit: '''PASS''' | ||
* {{|}} Mac OS X 10.6: | * {{done|}} Mac OS X 10.6: '''PASS''' | ||
* {{|}} Mac OS X 10.7: | * {{done|}} Mac OS X 10.7: '''PASS''' | ||
* {{|}} Ubuntu 11.04: | * {{done|}} Ubuntu 11.04: '''PASS''' | ||
* {{|}} Ubuntu 11.04 64-bit: | * {{done|}} Ubuntu 11.04 64-bit: '''PASS''' | ||
Release: report | Release: [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=release&from=2011-08-16&to=2011-08-16&target=6.0 report] | ||
* {{|}} Windows XP: | * {{done|}} Windows XP: '''PASS''' | ||
* {{|}} Windows Vista: | * {{done|}} Windows Vista: '''PASS''' | ||
* {{|}} Windows 7: | * {{done|}} Windows 7: '''PASS''' | ||
* {{|}} Windows 7 64-bit: | * {{done|}} Windows 7 64-bit: '''PASS''' | ||
* {{|}} Mac OS X 10.6: | * {{done|}} Mac OS X 10.6: '''PASS''' | ||
* {{|}} Mac OS X 10.7: | * {{done|}} Mac OS X 10.7: '''PASS''' | ||
* {{|}} Ubuntu 11.04: | * {{done|}} Ubuntu 11.04: '''PASS''' | ||
* {{|}} Ubuntu 11.04 64-bit: | * {{done|}} Ubuntu 11.04 64-bit: '''PASS''' | ||
; Issues Found | ; Issues Found | ||