|
|
| Line 1: |
Line 1: |
| <small>[[QA/Firefox_6|« Firefox 6]] | [[QA/Firefox:7|Firefox 7]] | [[QA/Firefox_8|Firefox 8 »]] </small>
| | ; Moved to https://wiki.mozilla.org/Releases/Firefox_7/Test_Plan |
| | |
| = Firefox 7 Test Plan =
| |
| == Summary ==
| |
| {| class="fullwidth-table"
| |
| |-
| |
| | style="width:15%" | '''Milestone'''
| |
| | [https://wiki.mozilla.org/Firefox/Flight_Tracking#Firefox_7 Firefox 7]
| |
| |-
| |
| | '''Status'''
| |
| | Currently in '''Aurora''' -- merging to Beta on August 16th, 2011
| |
| |-
| |
| | '''Lead'''
| |
| | [mailto:ashughes@mozilla.com Anthony Hughes] (irc: ashughes)
| |
| |-
| |
| | '''2nd'''
| |
| | [mailto:jbecerra@mozilla.com Juan Becerra] (irc: juanb)
| |
| |}
| |
| | |
| '''Latest Build: [ftp://ftp.mozilla.org/pub/firefox/nightly/latest-mozilla-aurora/ ftp], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-aurora/ http], [http://www.mozilla.com/en-US/firefox/channel/ web]'''
| |
| | |
| The following is the test plan for Firefox 7 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 7. After the release this document will be used as an archive of what was done to validate this release.
| |
| | |
| ''If you have some free time, please pick a task below and get in touch with one of the leads.''
| |
| | |
| == Details ==
| |
| The following is a description of the tasks which need to be completed throughout this release.
| |
| ; Daily Tasks
| |
| * {{ok|}} [softvision] [[QA/Firefox_7#Automation | Automation]]: Check dashboards for new failures
| |
| * {{ok|}} [softvision] [[QA/Firefox_7#Bug_Triage | Bug Triage]]: Check Bugzilla for regressions
| |
| * {{ok|}} [softvision] [[QA/Firefox_7#Features | Feature Sign-off]]: Test features and triage feature bugs targeted for this releases
| |
| | |
| ; One-time-only Tasks
| |
| * {{done|}} [ashughes] [[QA/Firefox_7#Aurora_Merge_Sign-off | Aurora Merge Sign-off]]: Sign-off merge from Nightly to Aurora
| |
| * {{ok|}} [ashughes] [[QA/Firefox_7#Beta_Merge_Sign-off | Beta Merge Sign-off]]: Sign-off merge from Aurora to Beta
| |
| | |
| == Automation ==
| |
| Triage the [http://mozmill-release.brasstacks.mozilla.com/#/general/reports?branch=7.0&platform=All Mozmill dashboard] for test failures and debug for potential Firefox regressions.
| |
| * '''Lead''': [mailto:vlad.maniac@softvision.ro Vlad Maniac] (Softvision)
| |
| * '''Peer''': [mailto:alex.lakatos@softvision.ro Alex Lakatos] (Softvision)
| |
| * '''Failure Status''': [http://bit.ly/q7Vm95 NEW] (12) | [http://bit.ly/oTmHo2 ASSIGNED] (7) | [http://bit.ly/q1sH6a FIXED] (3) | REGRESSIONS (0)
| |
| * '''Tasks''':
| |
| ** Check the [http://mozmill-release.brasstacks.mozilla.com/#/general/reports?branch=7.0&platform=All Mozmill dashboard] for failing tests for the day
| |
| ** Cross reference any failure to the list of [http://bit.ly/q7Vm95 NEW] and [http://bit.ly/oTmHo2 ASSIGNED] bugs
| |
| *** If the bug is NEW, assign it to yourself and try to fix it
| |
| *** If the bug is ASSIGNED and there has been no activity on the bug in over a week, offer your assistance to fix the bug
| |
| *** If there is NO bug on file, file it using [https://bugzilla.mozilla.org/enter_bug.cgi?alias=&assigned_to=nobody%40mozilla.org&blocked=&bug_file_loc=&bug_severity=normal&bug_status=NEW&comment=-----%20INSTRUCTIONS%20FOR%20FILING%20A%20MOZMILL%20FAILURE%20BUG%20-----%0D%0ANOTE%3A%20REMOVE%20THIS%20SECTION%20BEFORE%20FILING%20THE%20BUG%0D%0A%0D%0A1.%20Include%20a%20URL%20above%20for%20the%20dashboard%20report%20where%20the%20failure%20occurs%0D%0A%0D%0AEXAMPLE%3A%0D%0Ahttp%3A%2F%2Fmozmill-release.brasstacks.mozilla.com%2F%23%2Ffunctional%2Ffailure%3Ftest%3D%252FtestSecurity%252FtestGreenLarry.js%26func%3DtestLarryGreen%0D%0A%0D%0A2.%20Include%20the%20test%20and%20failure%20type%20in%20the%20SUMMARY%20field%0D%0A%0D%0AEXAMPLE%3A%0D%0A%22Failure%20in%20%2FtestSecurity%2FtestGreenLarry.js%20%7C%20Timeout%20waiting%20for%20page%20loaded.%22%0D%0A%0D%0A3.%20Add%20%22%5Bmozmill-test-failure%5D%22%20to%20the%20whiteboard%20field%20below%0D%0A%0D%0A4.%20Include%20the%20test%20function%2C%20error%20message%2C%20date%20when%20the%20error%20occurred%2C%20date%20with%20the%20error%20first%20occurred%2C%20and%20number%20of%20times%20the%20error%20has%20been%20reported%20in%20the%20DESCRIPTION%20field%0D%0A%0D%0AEXAMPLE%3A%0D%0ATEST%3A%20testSecurity%2FtestGreenLarry.js%3AtestLarryGreen%28%29%0D%0AERROR%3A%20controller.waitForPageLoad%28%29%3A%20Timeout%20waiting%20for%20page%20loaded.%20%0D%0AWHEN%3A%202011-07-01%0D%0AFIRST%3A%202011-06-25%0D%0AFREQ%3A%205%0D%0A%0D%0A5.%20Remove%20these%20instructions%0D%0A%0D%0A6.%20Click%20SUBMIT%20BUG%0D%0A-----%20END%20INSTRUCTIONS%20-----%0D%0A%0D%0ATEST%3A%20%0D%0AERROR%3A%20%0D%0AWHEN%3A%20%0D%0AFIRST%3A%20%0D%0AFREQ%3A%20%0D%0A&component=Mozmill%20Tests&contenttypeentry=&contenttypemethod=autodetect&contenttypeselection=text%2Fplain&data=&dependson=&description=&flag_type-37=X&flag_type-4=X&flag_type-607=X&form_name=enter_bug&keywords=&maketemplate=Remember%20values%20as%20bookmarkable%20template&op_sys=All&priority=--&product=Mozilla%20QA&qa_contact=mozmill-tests%40mozilla-qa.bugs&rep_platform=All&requestee_type-325=&requestee_type-4=&requestee_type-607=&short_desc=&status_whiteboard=&target_milestone=---&version=unspecified this template]
| |
| ** If you need assistance, ask ashughes in the [irc://irc.mozilla.org/#mozmill #mozmill] channel
| |
| | |
| == Bug Triage ==
| |
| Check UNCONFIRMED, NEW, FIXED, REOPEN, and tracking/status bugs
| |
| * '''Lead''': [mailto:vlad.maniac@softvision.ro Vlad Maniac] (Softvision)
| |
| * '''Peer''': [mailto:alex.lakatos@softvision.ro Alex Lakatos] (Softvision)
| |
| * '''Bugs:''': [http://bit.ly/eeMILo UNCONFIRMED], [http://bit.ly/nNv8Yt FIXED], [http://bit.ly/iVEDcy TRACKING/STATUS]
| |
| * '''Tracking:''' [http://bit.ly/pjexpG UNCONFIRMED Spreadsheet], [http://bit.ly/qKMLGE FIXED Spreadsheet]
| |
| | |
| ; Important Bugs
| |
| * [FIXED] {{bug|598425}} - Asynchronous layer-based plugin painting on Mac
| |
| * [FIXED] {{bug|661471}} - Create preference(s) to allow specific font families to be forced to use GDI Classic rendering
| |
| | |
| ; Notes | |
| | |
| == Features ==
| |
| Track features which have [[Firefox/Flight_Tracking#Firefox_7|landed]] in Firefox 7 and are ready to be tested.
| |
| | |
| === [[Platform/Features/AzureD2DCanvas|Azure Direct2D backend for canvas]] ===
| |
| This feature is an implementation of an HTML5 canvas context using the new Azure 2D API with the Direct2D backend. It should give us significantly faster performance on canvas benchmarks and demos with no loss in functionality or quality.
| |
| * '''Leads''': Chris Blizzard (Product), Bas Schouten (Feature, Dev), Curtis Koenig (Security)
| |
| * '''QA Owner:''' [mailto:andrei.domuta@softvision.ro AndreiD]
| |
| * '''Status:''' ready for release
| |
| * '''Links:''' [https://wiki.mozilla.org/Platform/Features/AzureD2DCanvas Feature Page], [https://bugzilla.mozilla.org/show_bug.cgi?id=651858 Tracking Bug]
| |
| * '''Notes:'''
| |
| | |
| === [[Extension_Manager:Projects:Embedded_Add-on_Preferences|Embedded Add-on Preferences]] ===
| |
| Add-on developers should be able to create a series of simple preference options that can be embedded in the detail view of their add-on similar to mobile.
| |
| * '''Leads''': Justin Scott (Product), Dave Townsend (Feature), Geoff Lankow (Dev), Jennifer Boriss (UX)
| |
| * '''QA Owner''': Henrik Skupin
| |
| * '''Status''': landed -- needs testing
| |
| * '''Links''': [https://bugzilla.mozilla.org/show_bug.cgi?id=653637 Tracking Bug]
| |
| * '''Notes:'''
| |
| | |
| === [[Platform/Features/Telemetry|Telemetry]] ===
| |
| The goal for this feature is to give our developers the ability to know if changes they are making have wide-ranging positive and negative effects at scale. Another goal of this is to give us easy-to-use infrastructure to learn about the structure of the Internet as a whole.
| |
| * '''Leads''': Chris Blizzard (Product), Taras Glek (Feature, Dev), Curtis Koenig (Security), Sid Stamm (Privacy), Alex Limi (UX)
| |
| * '''QA Owner''': Andrei D
| |
| * '''Status''': landed -- needs testing
| |
| * '''Links''': [https://bugzilla.mozilla.org/show_bug.cgi?id=659396 Tracking Bug]
| |
| * '''Notes''':
| |
| | |
| == Beta Merge Sign-off ==
| |
| The following tasks need to be completed to sign-off the Beta merge.
| |
| * {{|}} [ashughes] Automated BFT execution: [[QA/Firefox_7#Automated_BFTs|details]]
| |
| * {{|}} [softvision] Manual Smoketest execution: [[QA/Firefox_7#Smoketests|details]]
| |
| * {{|}} [various] Feature spotchecks: [[QA/Firefox_7#Feature Spotchecks|details]]
| |
| * {{|}} [softvision] Bug fix verifications: [[QA/Firefox_7#Bug_Verifications|details]]
| |
| * {{|}} [ashughes] Updates from Firefox 5 Aurora to Firefox 6 Aurora: [[QA/Firefox_7#Updates|details]]
| |
| | |
| '''Builds''': ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-aurora/ (Firefox 7.0a2)<br>
| |
| '''Litmus Testrun''': https://litmus.mozilla.org/run_tests.cgi?test_run_id=315<br>
| |
| | |
| === Automated BFTs ===
| |
| The following are the detailed results of the automated BFT test runs.
| |
| * {{|}} Windows 2000: [ functional] | [ restart]
| |
| * {{|}} Windows XP: [ functional] | [ restart]
| |
| * {{|}} Windows Vista: [ functional] | [ restart]
| |
| * {{|}} Windows 7: [ functional] | [ restart]
| |
| * {{|}} Windows 7 64-bit: [ functional] | [ restart]
| |
| * {{|}} Mac OS X 10.6: [ functional] | [ restart]
| |
| * {{|}} Ubuntu 11.04: [ functional] | [ restart]
| |
| * {{|}} Ubuntu 11.04 64-bit: [ functional] | [ restart]
| |
| * '''Reports''': [ Litmus], [ Mozmill]
| |
| ; Issues Found
| |
| | |
| === Smoketests ===
| |
| The following are the detailed results of the manual smoketests.
| |
| * {{|}} [tester] Windows 2000: status
| |
| * {{|}} [tester] Windows XP: status
| |
| * {{|}} [tester] Windows Vista: status
| |
| * {{|}} [tester] Windows 7: status
| |
| * {{|}} [tester] Windows 7 64-bit: status
| |
| * {{|}} [tester] Mac OS X 10.6: status
| |
| * {{|}} [tester] Ubuntu 11.04: status
| |
| * {{|}} [tester] Ubuntu 11.04 32-bit: status
| |
| * '''Litmus''' [ testrun], [ report]
| |
| * '''Firefox 7''' [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-beta/ builds]
| |
| | |
| ; Issues Found
| |
| | |
| | |
| ; Exploratory Testing
| |
| Conduct exploratory tests on WebGL, Javascript, Flash, and QuickTime plugin content
| |
| * {{|}} [tester] Windows XP: status
| |
| * {{|}} [tester] Windows 7: status
| |
| * {{|}} [tester] Mac OS X 10.6: status
| |
| * {{|}} [tester] Ubuntu 11.04: status
| |
| | |
| ; Issues Found
| |
| | |
| === Feature Spotchecks ===
| |
| The following are the details about spotchecks performed on landed features
| |
| * {{|}} [andreid] Azure Direct2D Backend for Canvas: status
| |
| * {{|}} [whimboo] Embedded Add-on Preferences: status
| |
| * {{|}} [andreid] Telemetry: status
| |
| | |
| ; Issues Found
| |
| | |
| === Bug Verifications ===
| |
| The following are the details of bug verifications for Firefox 7
| |
| * {{done|}} [softvision] Triage as many fixed bugs as possible
| |
| * '''Bugzilla''': [http://bit.ly/orlThe tracking-firefox7+], [http://bit.ly/npeJyZ report]
| |
| * '''Key Bugs''':
| |
| ** {{bug|}} n/a
| |
| | |
| ; Issues Found
| |
| *
| |
| | |
| === Updates ===
| |
| The following are details of update verifications for Firefox 7 merge to Beta
| |
| * {{|}} [tester] Windows XP: status
| |
| ** Source:
| |
| ** Target:
| |
| * {{|}} [tester] Windows 7: status
| |
| ** Source:
| |
| ** Target:
| |
| * {{|}} [tester] Mac OS X 10.6: status
| |
| ** Source:
| |
| ** Target:
| |
| * {{|}} [tester] Ubuntu 11.04: status
| |
| ** Source:
| |
| ** Target:
| |
| | |
| ; Issues Found
| |
| | |
| == Aurora Merge Sign-off ==
| |
| The following tasks need to be completed to sign-off the Aurora merge.
| |
| * {{done|}} [gmealer] Automated BFT execution: [[QA/Firefox_7#Automated_BFTs_2|details]]
| |
| * {{done|}} [softvision] Manual Smoketest execution: [[QA/Firefox_7#Smoketests_2|details]]
| |
| * {{done|}} [softvision] Feature spotchecks: [[QA/Firefox_7#Feature Spotchecks_2|details]]
| |
| * {{done|}} [softvision] Bug fix verifications: [[QA/Firefox_7#Bug_Verifications_2|details]]
| |
| * {{done|}} [ashughes] Updates from Firefox 5 Aurora to Firefox 6 Aurora: [[QA/Firefox_7#Updates_2|details]]
| |
| | |
| '''Builds''': ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-aurora/ (Firefox 7.0a2)<br>
| |
| '''Litmus Testrun''': https://litmus.mozilla.org/run_tests.cgi?test_run_id=315<br>
| |
| | |
| === Automated BFTs ===
| |
| The following are the detailed results of the automated BFT test runs.
| |
| * {{done|}} Windows 2000: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cc7f2f9 functional] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cc85a9c restart]
| |
| * {{done|}} Windows XP: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cc83bf4 functional] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cc882b4 restart]
| |
| * {{done|}} Windows Vista: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cc7d8a6 functional] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cc84b37 restart]
| |
| * {{done|}} Windows 7: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cc84025 functional] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cc88601 restart]
| |
| * {{done|}} Windows 7 64-bit: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cc7e01d functional] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cc86815 restart]
| |
| * {{done|}} Mac OS X 10.6: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cc83158 functional] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cc883fd restart]
| |
| * {{done|}} Ubuntu 11.04: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cc7e55c functional] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cc87574 restart]
| |
| * {{done|}} Ubuntu 11.04 64-bit: [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cc7bf03 functional] | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cc7d93c restart]
| |
| | |
| ; Issues Found
| |
| | |
| === Smoketests ===
| |
| The following are the detailed results of the manual smoketests.
| |
| * {{done|}} [simona] Windows 2000: no issues found
| |
| * {{done|}} [andreid] Windows XP: no issues found
| |
| * {{done|}} [simona] Windows Vista: no issues found
| |
| * {{done|}} [vladg] Windows 7: no issues found
| |
| * {{done|}} [andreid] Windows 7 64-bit: no issues found
| |
| * {{done|}} [vladg] Mac OS X 10.6: no issues found
| |
| * {{done|}} [george] Ubuntu 11.04: no issues found
| |
| * {{done|}} [george] Ubuntu 11.04 32-bit: no issues found
| |
| * '''Litmus''' [https://litmus.mozilla.org/run_tests.cgi?test_run_id=315 testrun], [https://litmus.mozilla.org/test_run_report.cgi?test_run_id=315 report]
| |
| * '''Firefox 7''' [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-aurora/ builds]
| |
| | |
| ; Issues Found
| |
| | |
| === Feature Spotchecks ===
| |
| The following are the details about spotchecks performed on landed features
| |
| * {{done|}} [whimboo] Embedded Add-on Preferences: no issues found
| |
| * {{done|}} [andreid] Azure Direct2D Backend for Canvas: no issues found
| |
| * {{done|}} [george] Sharing Textures Cross-process for Electrolysis: not landed
| |
| * {{done|}} [vladmaniac] Web Console 7: not landed
| |
| | |
| ; Issues Found
| |
| | |
| === Bug Verifications ===
| |
| The following are the details of bug verifications for Firefox 7
| |
| * '''Bugs''': [http://bit.ly/orlThe tracking-firefox7+]
| |
| * '''Results''': [http://bit.ly/ozZqNi VERIFIED], [http://bit.ly/rdjahm REOPENED]
| |
| | |
| ; Issues Found
| |
| | |
| === Updates ===
| |
| The following are details of update verifications for Firefox 7 merge to Aurora
| |
| | |
| ; Auroratest
| |
| * {{done|}} [ashughes] Windows XP: '''PASS'''
| |
| ** Source: Mozilla/5.0 (Windows NT 5.1; rv:6.0a2) Gecko/20110705 Firefox/6.0a2
| |
| ** Target: Mozilla/5.0 (Windows NT 5.1; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
| |
| * {{done|}} [ashughes] Windows 7: '''PASS'''
| |
| ** Source: Mozilla/5.0 (Windows NT 6.1; rv:6.0a2) Gecko/20110705 Firefox/6.0a2
| |
| ** Target: Mozilla/5.0 (Windows NT 6.1; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
| |
| * {{done|}} [ashughes] Mac OS X 10.6: '''PASS'''
| |
| ** Source: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:6.0a2) Gecko/20110702 Firefox/6.0a2
| |
| ** Target: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
| |
| * {{done|}} [ashughes] Ubuntu 11.04: '''PASS'''
| |
| ** Source: Mozilla/5.0 (X11; Linux x86_64; rv:6.0a2) Gecko/20110628 Firefox/6.0a2
| |
| ** Target: Mozilla/5.0 (X11; Linux x86_64; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
| |
| ; Aurora
| |
| * {{done|}} [ashughes] Windows XP: '''PASS'''
| |
| ** Source: Mozilla/5.0 (Windows NT 5.1; rv:6.0a2) Gecko/20110705 Firefox/6.0a2
| |
| ** Target: Mozilla/5.0 (Windows NT 5.1; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
| |
| * {{done|}} [ashughes] Windows 7: '''PASS'''
| |
| ** Source: Mozilla/5.0 (Windows NT 6.1; rv:6.0a2) Gecko/20110705 Firefox/6.0a2
| |
| ** Target: Mozilla/5.0 (Windows NT 6.1; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
| |
| * {{done|}} [ashughes] Mac OS X 10.6: '''PASS'''
| |
| ** Source: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:6.0a2) Gecko/20110702 Firefox/6.0a2
| |
| ** Target: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
| |
| * {{done|}} [ashughes] Ubuntu 11.04: '''PASS'''
| |
| ** Source: Mozilla/5.0 (X11; Linux x86_64; rv:6.0a2) Gecko/20110628 Firefox/6.0a2
| |
| ** Target: Mozilla/5.0 (X11; Linux x86_64; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
| |
| ; Issues Found
| |