374
edits
(→Manual) |
|||
| Line 1: | Line 1: | ||
<small> | <p><small><a href="QA/mozilla-beta/fx6">« mozilla-beta:fx6</a> | <a href="Releases/Firefox_6/Test_Plan" class="selflink">Firefox 6</a> | <a href="Releases/Firefox 7/Test Plan">Firefox 7 »</a> </small> | ||
</p> | |||
<h2> Firefox 6 Test Plan </h2> | |||
<ul><li> Release: Tuesday, August 16, 2011 | |||
</li><li> QA Contact: <strike><a href="mailto:juanb@mozilla.com">Juan Becerra</a></strike> <a href="mailto:ahughes@mozilla.com">Anthony Hughes</a> | |||
</li><li> IRC: irc://irc.mozilla.org#qa (juanb, ashughes, whimboo) | |||
</li><li> Builds: <a href="ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/6.0-candidates/">Firefox 6.0</a> | |||
</li></ul> | |||
The following tasks need to be completed to sign-off the release. | <h3> Tasks </h3> | ||
<p>The following tasks need to be completed to sign-off the release. | |||
</p> | |||
<ul><li> <span class="fck_mw_template">{{|}}</span> [ashughes] Automated BFTs: <a href="QA/Firefox 6#Automation">details</a> | |||
</li><li> <span class="fck_mw_template">{{|}}</span> [softvision] Manual Testing: <a href="QA/Firefox 6#Manual">details</a> | |||
</li><li> <span class="fck_mw_template">{{|}}</span> [softvision] Feature sign-off: <a href="QA/Firefox 6#Feature_sign-off">details</a> | |||
</li><li> <span class="fck_mw_template">{{|}}</span> [softvision] Bug Verifications: <a href="QA/Firefox 6#Bug_Verifications">details</a> | |||
</li><li> <span class="fck_mw_template">{{|}}</span> [ashughes] Automated Updates: <a href="QA/Firefox 6#Updates">details</a> | |||
</li><li> <b>Docs</b>: <a href="QA/Firefox/Merge Plan 2011-08-16">timeline</a>, <a href="QA SoftVision Team/Firefox-6-release-testplan">softvision testplan</a> | |||
Spotchecks should be performed across all supported operating systems, including Mac OSX 10.7: | </li></ul> | ||
<p>Spotchecks should be performed across all supported operating systems, including Mac OSX 10.7: | |||
</p> | |||
<ul><li> Windows 2000, XP, Vista, 7, 7 64bit | |||
</li><li> Mac OS X 10.5, 10.6, 10.7 | |||
If you find any issues use | </li><li> Ubuntu 11.04, 11.04 64-bit | ||
</li></ul> | |||
<p>If you find any issues use <b>[fx6]</b> whiteboard entry of the bug to keep track. | |||
The following are the detailed results of the automated BFT test runs. | </p> | ||
<h3> Automation </h3> | |||
<p>The following are the detailed results of the automated BFT test runs. | |||
</p> | |||
<ul><li> <span class="fck_mw_template">{{done|}}</span> Windows 2000: <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9797bd">normal</a> | <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97aecf">restart</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> Windows XP: <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9de820">normal</a> | <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9de8c3">restart</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> Windows Vista: <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97a259">normal</a> | <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97bac9">restart</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> Windows 7: <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f976f64">normal</a> | <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9778ce">restart</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> Windows 7 64-bit: <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97786f">normal</a> | <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f978743">restart</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> Mac OS X 10.6: <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97a565">normal</a> | <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97beb0">restart</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> Mac OS X 10.7: <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97cbf3">normal</a> | <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f97d82a">restart</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> Ubuntu 11.04: <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9e7bd8">normal</a> | <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9e8b96">restart</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> Ubuntu 11.04 64-bit (x86): <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9e7392">normal</a> | <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9e8030">restart</a> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> Ubuntu 11.04 64-bit (x64): <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9ea428">normal</a> | <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/report/1aa1a55cbadc6f0e705647266f9eae1d">restart</a> | |||
</li><li> <b>Litmus</b>: Sync results to <a href="https://litmus.mozilla.org/run_tests.cgi?test_run_id=331">here</a> | |||
</li><li> <b>Mozmill</b>: <a href="http://mozmill-release.brasstacks.mozilla.com/#/functional/reports?branch=6.0&platform=All&from=2011-08-12&to=2011-08-12">results</a> | |||
</li></ul> | |||
The following are the detailed results of the manual smoketests. | <dl><dt> Issues Found | ||
</dt></dl> | |||
<h3> Manual </h3> | |||
<p>The following are the detailed results of the manual smoketests. | |||
</p> | |||
<ul><li> <span class="fck_mw_template">{{done|}}</span> [Alin] Windows 2000: PASS | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> [Mihaela] Windows XP: PASS | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> [Simona] Windows Vista: PASS | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> [Simona] Windows 7: PASS | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> [AndreiD] Windows 7 64-bit: PASS | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> [VladG] Mac OS X 10.6: PASS | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> [VladG] Mac OS X 10.7: PASS | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> [Virgil] Ubuntu 11.04: PASS | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> [AndreiD] Ubuntu 11.04 32-bit: PASS | |||
</li><li> <b>Litmus</b> <a href="https://litmus.mozilla.org/run_tests.cgi?test_run_id=331">testrun</a>, <a href="https://litmus.mozilla.org/test_run_report.cgi?test_run_id=331">report</a> | |||
</li><li> <b>Firefox 6</b> <a href="ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/6.0-candidates/build1/">builds</a> | |||
</li></ul> | |||
<dl><dt> Issues Found | |||
</dt></dl> | |||
<p><br /> | |||
</p> | |||
<dl><dt> Exploratory Testing | |||
</dt></dl> | |||
<ul><li> Conduct exploratory tests on WebGL, Javascript, Flash, and QuickTime plugin content | |||
</li><li> Tested plugin content using the following testing matrix: <a href="http://bit.ly/q0dKtA">testing matrix</a> | |||
<ul><li> <span class="fck_mw_template">{{done|}}</span> [mihaela] Windows XP: PASS | |||
</li><li> <span class="fck_mw_template">{{ok|}}</span> [alin] Windows 7: status | |||
</li><li> <span class="fck_mw_template">{{ok|}}</span> [VladG] Mac OS X 10.6: status | |||
</li><li> <span class="fck_mw_template">{{ok|}}</span> [virgil] Ubuntu 11.04: status | |||
</li></ul> | |||
</li></ul> | |||
The following are the details about spotchecks performed on landed features | <dl><dt> Issues Found | ||
</dt></dl> | |||
<ul><li><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=638741">Bug 638741 - Right click on flash content displays two context menus sometimes</a> | |||
</li><li><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=638770">Bug 638770 - clicking on a quicktime video pauses it</a> | |||
</li><li><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=638770">Bug 678482 - Scrolling in web pages with java and shockwave scrambles the content</a> | |||
</li><li><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=638770">Bug 607586 - Realplayer Plugin does not work</a> | |||
</li></ul> | |||
<h3>Feature sign-off</h3> | |||
<p>The following are the details about spotchecks performed on landed features | |||
</p> | |||
<ul><li> <span class="fck_mw_template">{{done|}}</span> [andreid] <a href="https://wiki.mozilla.org/Firefox/Features/50ms_ASSERT">50ms Assert</a>: OK for release | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> [vladmaniac] <a href="https://wiki.mozilla.org/Features/Firefox/PlugincheckLinkInAOM">PlugincheckLinkInAOM</a>: SIGN-OFF | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> [tracy] <a href="https://wiki.mozilla.org/Firefox/Features/Sync_Discoverability">Sync Discoverability</a>: SIGN-OFF | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> [george] <a href="https://wiki.mozilla.org/Firefox/Features/Persist_Panorama_Groups">Persist Panorama Groups</a>: SIGN-OFF | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> [george] <a href="https://wiki.mozilla.org/Firefox/Features/Panorama_Groups_On_Demand">Panorama Groups On Demand</a>: SIGN-OFF | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> [george] <a href="https://wiki.mozilla.org/Privacy/Features/Site-based_data_management_UI">Site-based data management UI</a>: SIGN-OFF | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> [george] <a href="https://wiki.mozilla.org/Firefox/Features/windows-scrolling">windows-scrolling</a>: SIGN-OFF | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> [simona] <a href="https://wiki.mozilla.org/Firefox/Features/Locationbar_Domain_Highlight">Locationbar Domain Highlight</a>: SIGN-OFF | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> [simona] <a href="https://wiki.mozilla.org/DevTools/Features/WebDeveloperMenu">WebDeveloper Menu</a>: SIGN-OFF | |||
The following are the detailed results of bug fix verifications for Firefox 6. | </li><li> <span class="fck_mw_template">{{done|}}</span> [vladg] <a href="https://wiki.mozilla.org/Platform/Features/RevampAboutMemory">RevampAboutMemory</a>: SIGN-OFF | ||
</li><li> <span class="fck_mw_template">{{done|}}</span> [vladg] <a href="https://wiki.mozilla.org/Platform/Features/Progress_Element">Progress Element</a>: SIGN-OFF | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> [vladmaniac] <a href="https://wiki.mozilla.org/DevTools/Features/AutocompletionPopup">Autocomplete Popup</a>: SIGN-OFF | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> [vladmaniac] <a href="https://wiki.mozilla.org/DevTools/Features/WebConsoleUpgrade">WebConsoleUpgrade</a>: SIGN-OFF | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> [alex] <a href="https://wiki.mozilla.org/DevTools/Features/Workspaces">Scratchpad</a>: SIGN-OFF | |||
</li></ul> | |||
<dl><dt> Issues Found | |||
</dt></dl> | |||
<h3> Bug Verifications </h3> | |||
<p>The following are the detailed results of bug fix verifications for Firefox 6. | |||
</p> | |||
<ul><li> <b>Bugzilla</b>: <a href="http://bit.ly/qYzQ9L">tracking-firefox6+</a> | |||
</li><li> <b>Results</b>: <a href="http://bit.ly/qvPpwx">VERIFIED</a>, <a href="http://bit.ly/ouYORw">REOPENED</a> | |||
The following are details of update verifications for Firefox 6 | </li></ul> | ||
<dl><dt> Issues Found | |||
</dt></dl> | |||
<ul><li> <span class="fck_mw_template">{{done|}}</span> <span class="fck_mw_template">{{bug|677510}}</span> - Video Controls buttons "Mute/unmute" and "Expand/Shrink" are lacking icons on HTML5 videos | |||
<ul><li> <b>Issue has been fixed on YouTube's end</b> | |||
</li></ul> | |||
</li><li> <span class="fck_mw_template">{{done|}}</span> <span class="fck_mw_template">{{bug|677095}}</span> - 1px background-image miss-handling when position is fixed | |||
<ul><li> <b>Issue will be rel-noted and fixed in Firefox 7</b> | |||
</li></ul> | |||
</li><li> <span class="fck_mw_template">{{bug|675260}}</span> - Crash in arena_dalloc_small coming from ShutdownNSS | |||
Betatest: report | <ul><li> <b>Not a priority for Firefox 6</b> -- If you have free cycles, we need help identifying builds affected and steps to reproduce | ||
</li></ul> | |||
</li></ul> | |||
<h3> Updates </h3> | |||
<p>The following are details of update verifications for Firefox 6 | |||
</p> | |||
<dl><dt> Coverage | |||
</dt></dl> | |||
<ul><li> Platforms: Windows (2000, XP, Vista, 7, 7 64-bit), Mac OS X (10.6, 10.7), Ubuntu (11.04, 11.04 64-bit) | |||
</li><li> Updates: partial, complete, fallback | |||
Beta: report | </li><li> Locales: en-US, random | ||
</li><li> Versions: Firefox 4*, 5*, 6b* | |||
</li><li> Target Build: | |||
</li></ul> | |||
<dl><dt> Results | |||
</dt></dl> | |||
<p>Betatest: report | |||
</p> | |||
<ul><li> <span class="fck_mw_template">{{|}}</span> Windows XP: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Windows Vista: status | |||
Releasetest: report | </li><li> <span class="fck_mw_template">{{|}}</span> Windows 7: status | ||
</li><li> <span class="fck_mw_template">{{|}}</span> Windows 7 64-bit: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Mac OS X 10.6: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Mac OS X 10.7: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Ubuntu 11.04: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Ubuntu 11.04 64-bit: status | |||
</li></ul> | |||
<p>Beta: report | |||
</p> | |||
<ul><li> <span class="fck_mw_template">{{|}}</span> Windows XP: status | |||
Release: report | </li><li> <span class="fck_mw_template">{{|}}</span> Windows Vista: status | ||
</li><li> <span class="fck_mw_template">{{|}}</span> Windows 7: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Windows 7 64-bit: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Mac OS X 10.6: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Mac OS X 10.7: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Ubuntu 11.04: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Ubuntu 11.04 64-bit: status | |||
</li></ul> | |||
<p>Releasetest: report | |||
</p> | |||
<ul><li> <span class="fck_mw_template">{{|}}</span> Windows XP: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Windows Vista: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Windows 7: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Windows 7 64-bit: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Mac OS X 10.6: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Mac OS X 10.7: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Ubuntu 11.04: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Ubuntu 11.04 64-bit: status | |||
</li></ul> | |||
<p>Release: report | |||
</p> | |||
<ul><li> <span class="fck_mw_template">{{|}}</span> Windows XP: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Windows Vista: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Windows 7: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Windows 7 64-bit: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Mac OS X 10.6: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Mac OS X 10.7: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Ubuntu 11.04: status | |||
</li><li> <span class="fck_mw_template">{{|}}</span> Ubuntu 11.04 64-bit: status | |||
</li></ul> | |||
<dl><dt> Issues Found | |||
</dt></dl> | |||
edits