QA/mozilla-beta/fx6/Beta 4: Difference between revisions
(→Manual) |
|||
(22 intermediate revisions by 7 users not shown) | |||
Line 10: | Line 10: | ||
We should complete the following tasks before signing off on the next weekly beta. We should be focusing on bug verifications and testing around fixes. | We should complete the following tasks before signing off on the next weekly beta. We should be focusing on bug verifications and testing around fixes. | ||
* {{ | * {{done|}} Automation (Mozmill BFTs) - ashughes/whimboo: [[QA/mozilla-beta/fx6/Beta_4#Automation|'''Signed Off''']] | ||
* {{ | * {{done|}} [http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=FIREFOX_6_0b3_RELEASE&tochange=FIREFOX_6_0b4_RELEASE Bug Verifications] - Waverley Team: [http://bit.ly/q3SERG '''Signed Off'''] | ||
* {{ | * {{done|}} Manual Testing - Waverley Team: [[QA/mozilla-beta/fx6/Beta_4#Manual|'''Signed Off''']] | ||
* {{ | * {{done|}} Automation (Updates) - ashughes/whimboo: [[QA/mozilla-beta/fx6/Beta_4#Updates|'''Signed Off (betatest)''']] | ||
Spotchecks should be performed across all supported operating systems, including Mac OSX 10.7: | Spotchecks should be performed across all supported operating systems, including Mac OSX 10.7: | ||
Line 20: | Line 19: | ||
* Windows 7 (32-bit & 64-bit) | * Windows 7 (32-bit & 64-bit) | ||
* Windows Vista | * Windows Vista | ||
* Mac | * Mac OS X 10.6 | ||
* Ubuntu | * Mac OS X 10.7 | ||
* Ubuntu 11.04 (32-bit and 64-bit) | |||
* Windows 2000 | * Windows 2000 | ||
== Results == | == Results == | ||
Line 30: | Line 29: | ||
=== Automation === | === Automation === | ||
Run automated BFTs and link reports from dashboard | Run automated BFTs and link reports from dashboard: [https://litmus.mozilla.org/test_run_report.cgi?test_run_id=326 Litmus], [http://mozmill-release.brasstacks.mozilla.com/#/functional/reports?branch=6.0&platform=All&from=2011-08-01&to=2011-08-01 Mozmill] | ||
{| class="fullwidth-table" | {| class="fullwidth-table" | ||
| style="background:#EFEFEF; width:20%" | '''Platform''' | | style="background:#EFEFEF; width:20%" | '''Platform''' | ||
Line 38: | Line 36: | ||
|- | |- | ||
| Windows 2000 | | Windows 2000 | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/f0cbd82133cb0b19acd7d615d0af4cbd '''PASS'''] | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/f0cbd82133cb0b19acd7d615d0af88fa '''PASS'''] | ||
|- | |- | ||
| Windows XP | | Windows XP | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/f0cbd82133cb0b19acd7d615d0af32e0 '''PASS'''] | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/f0cbd82133cb0b19acd7d615d0af5cb3 '''PASS'''] | ||
|- | |- | ||
| Windows Vista | | Windows Vista | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/f0cbd82133cb0b19acd7d615d0af6eb5 '''PASS'''] | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/f0cbd82133cb0b19acd7d615d0afbf1e '''PASS'''] | ||
|- | |- | ||
| Windows 7 | | Windows 7 | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/f0cbd82133cb0b19acd7d615d0af45d7 '''PASS'''] | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/f0cbd82133cb0b19acd7d615d0afa18b '''PASS'''] | ||
|- | |- | ||
| Windows 7 64-bit | | Windows 7 64-bit | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/f0cbd82133cb0b19acd7d615d0af60e6 '''PASS'''] | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/f0cbd82133cb0b19acd7d615d0afb5dd '''PASS'''] | ||
|- | |- | ||
| Mac OS X 10.6 | | Mac OS X 10.6 | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/f0cbd82133cb0b19acd7d615d0b01113 '''PASS'''] | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/f0cbd82133cb0b19acd7d615d0b01fec '''PASS'''] | ||
|- | |- | ||
| Mac OS X 10.7 | | Mac OS X 10.7 | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/f0cbd82133cb0b19acd7d615d0afd1fa '''PASS'''] | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/f0cbd82133cb0b19acd7d615d0afd553 '''PASS'''] | ||
|- | |- | ||
| Ubuntu | | Ubuntu 11.04 | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/f0cbd82133cb0b19acd7d615d0af285d '''PASS'''] | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/f0cbd82133cb0b19acd7d615d0af67d6 '''PASS'''] | ||
|- | |- | ||
| Ubuntu | | Ubuntu 11.04 64-bit | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/f0cbd82133cb0b19acd7d615d0af379e '''PASS'''] | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/f0cbd82133cb0b19acd7d615d0af7dbd '''PASS'''] | ||
|} | |} | ||
; Issues Found | ; Issues Found | ||
=== Manual === | === Manual === | ||
;Smoketests | ;Smoketests | ||
Litmus Testrun: https://litmus.mozilla.org/run_tests.cgi?test_run_id=326 | * Litmus Testrun: https://litmus.mozilla.org/run_tests.cgi?test_run_id=326 | ||
Litmus Report: https://litmus.mozilla.org/test_run_report.cgi?test_run_id=326 | * Litmus Report: https://litmus.mozilla.org/test_run_report.cgi?test_run_id=326 | ||
{| class="fullwidth-table" | {| class="fullwidth-table" | ||
Line 90: | Line 87: | ||
| Windows 2000 | | Windows 2000 | ||
| Simona | | Simona | ||
| | | PASS | ||
|- | |- | ||
| Windows XP | | Windows XP | ||
| Andrei | | Andrei | ||
| | | PASS | ||
|- | |- | ||
| Windows Vista | | Windows Vista | ||
| Simona | | Simona | ||
| | | PASS | ||
|- | |- | ||
| Windows 7 | | Windows 7 | ||
| Andrei | | Andrei | ||
| | | PASS | ||
|- | |- | ||
| Windows 7 64-bit | | Windows 7 64-bit | ||
Line 110: | Line 107: | ||
| Mac OS X 10.6 | | Mac OS X 10.6 | ||
| Vlad G | | Vlad G | ||
| | | PASS | ||
|- | |- | ||
| Ubuntu 11.04 | | Ubuntu 11.04 | ||
Line 118: | Line 115: | ||
| Ubuntu 11.04 64-bit | | Ubuntu 11.04 64-bit | ||
| George | | George | ||
| | | PASS | ||
|- | |- | ||
| Mac OS X 10.7 | | Mac OS X 10.7 | ||
| Vlad G | | Vlad G | ||
| | | PASS | ||
|- | |||
|} | |} | ||
Line 128: | Line 126: | ||
* Any major issues? | * Any major issues? | ||
** n/a | ** n/a | ||
;l10n spotchecking | |||
* German | |||
**Name of tester: Alin | |||
**PASS | |||
* French | |||
**Name of tester: Alin | |||
**PASS | |||
* Spanish | |||
**Name of tester: Alin | |||
**PASS | |||
* Italian | |||
**Name of tester: Alin | |||
**PASS | |||
* Russian | |||
**Name of tester: Alin | |||
**PASS | |||
;Exploratory Testing Around Changes - On Applicable Platforms | ;Exploratory Testing Around Changes - On Applicable Platforms | ||
* | * WebGL | ||
* | ** {{Bug|675625}} - Firefox crashes when performing WebGL conformance tests on Ubuntu 11.04 | ||
* | ** {{Bug|675634}} - Firefox crashes while on a page that uses WebGL | ||
* Flash | |||
** {{Bug|675645}} - Firefox freezes when using Pop out while in Fullscreen mode on Youtube | |||
* Silverlight | |||
** n/a | |||
;Issues Found | ;Issues Found | ||
Line 138: | Line 158: | ||
=== Updates === | === Updates === | ||
'''Target Build ID:''' Mozilla/5.0 (Windows NT 5.0; rv:6.0) Gecko/20100101 Firefox/6.0 | '''Source Builds:''' 4.0b*, 4.0, 4.0.1, 5.0b*, 5.0, 6.0b1, 6.0b2, 6.0b3 <br> | ||
'''Source Locales:''' en-US + one random P1 locale per platform <br> | |||
'''Update Types:''' complete+fallback, partial+fallback <br> | |||
'''Target Build ID:''' Mozilla/5.0 (Windows NT 5.0; rv:6.0) Gecko/20100101 Firefox/6.0 | |||
{| class="fullwidth-table" | {| class="fullwidth-table" | ||
| style="background:#EFEFEF; width: | | style="background:#EFEFEF; width:25%" | '''Platform''' | ||
| style="background:#EFEFEF; width: | | style="background:#EFEFEF; width:65%" | '''Builds''' | ||
| style="background:#EFEFEF; width:10%" | '''Result''' | | style="background:#EFEFEF; width:10%" | '''Result''' | ||
|- | |- | ||
| colspan="4" | '''Betatest''': [http://mozmill-release.brasstacks.mozilla.com/#/update/ | | colspan="4" | '''Betatest''': [http://mozmill-release.brasstacks.mozilla.com/#/update/overview?branch=6.0&channel=betatest&from=2011-08-01&to=2011-08-01 REPORT] | ||
|- | |- | ||
| Windows 2000 | | Windows 2000 | ||
| | | Mozilla/5.0 (Windows NT 5.0; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | ||
| ''' | | '''PASS''' | ||
|- | |- | ||
| Windows XP | | Windows XP | ||
| | | Mozilla/5.0 (Windows NT 5.1; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | ||
| ''' | | '''PASS''' | ||
|- | |- | ||
| Windows Vista | | Windows Vista | ||
| | | Mozilla/5.0 (Windows NT 6.0; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | ||
| ''' | | '''PASS''' | ||
|- | |- | ||
| Windows 7 | | Windows 7 | ||
| | | Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | ||
| ''' | | '''PASS''' | ||
|- | |- | ||
| Windows 7 64-bit | | Windows 7 64-bit | ||
| | | Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | ||
| ''' | | '''PASS''' | ||
|- | |- | ||
| Mac OS X 10.6 | | Mac OS X 10.6 | ||
| | | Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | ||
| ''' | | '''PASS''' | ||
|- | |- | ||
| Mac OS X 10.7 | | Mac OS X 10.7 | ||
| | | Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | ||
| | | '''PASS''' | ||
|- | |- | ||
| Ubuntu 11.04 | | Ubuntu 11.04 | ||
| | | Mozilla/5.0 (X11; Linux i686; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | ||
| ''' | | '''PASS''' | ||
|- | |- | ||
| Ubuntu 11.04 64-bit | | Ubuntu 11.04 64-bit | ||
| | | Mozilla/5.0 (X11; Linux i686 on x86_64; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | ||
| ''' | | '''PASS''' | ||
|- | |- | ||
| colspan="4" | '''Beta''': [http://mozmill-release.brasstacks.mozilla.com/#/update/ | | colspan="4" | '''Beta''': [http://mozmill-release.brasstacks.mozilla.com/#/update/overview?branch=6.0&channel=beta&from=2011-08-02&to=2011-08-02 report] | ||
|- | |- | ||
| Windows 2000 | | Windows 2000 | ||
| | | Mozilla/5.0 (Windows NT 5.0; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | ||
| ''' | | '''PASS''' | ||
|- | |- | ||
| Windows XP | | Windows XP | ||
| | | Mozilla/5.0 (Windows NT 5.1; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | ||
| ''' | | '''PASS''' | ||
|- | |- | ||
| Windows Vista | | Windows Vista | ||
| | | Mozilla/5.0 (Windows NT 6.0; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | ||
| ''' | | '''PASS''' | ||
|- | |- | ||
| Windows 7 | | Windows 7 | ||
| | | Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | ||
| ''' | | '''PASS''' | ||
|- | |- | ||
| Windows 7 64-bit | | Windows 7 64-bit | ||
| | | Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | ||
| ''' | | '''PASS''' | ||
|- | |- | ||
| Mac OS X 10.6 | | Mac OS X 10.6 | ||
| | | Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | ||
| ''' | | '''PASS''' | ||
|- | |- | ||
| Mac OS X 10.7 | | Mac OS X 10.7 | ||
| | | Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | ||
| | | '''PASS''' | ||
|- | |- | ||
| Ubuntu 11.04 | | Ubuntu 11.04 | ||
| | | Mozilla/5.0 (X11; Linux i686; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | ||
| ''' | | '''PASS''' | ||
|- | |- | ||
| Ubuntu 11.04 64-bit | | Ubuntu 11.04 64-bit | ||
| | | Mozilla/5.0 (X11; Linux i686 on x86_64; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | ||
| ''' | | '''PASS''' | ||
|- | |- | ||
|} | |} | ||
; Issues Found | ; Issues Found | ||
Latest revision as of 06:36, 2 August 2011
« mozilla-beta:fx6 | mozilla-beta:fx6:Beta 3 | mozilla-beta:fx6:Beta 4 | mozilla-beta:fx6:Beta 5 »
Beta 4 Test Plan and Results
- Release: Thursday/Friday July 28th/30th
- QA Contact: Juan Becerra
- IRC: irc://irc.mozilla.org#qa juanb, ashughes, whimboo
- Builds: Firefox 6.0b4 | pushlog
Tasks
We should complete the following tasks before signing off on the next weekly beta. We should be focusing on bug verifications and testing around fixes.
- [DONE] Automation (Mozmill BFTs) - ashughes/whimboo: Signed Off
- [DONE] Bug Verifications - Waverley Team: Signed Off
- [DONE] Manual Testing - Waverley Team: Signed Off
- [DONE] Automation (Updates) - ashughes/whimboo: Signed Off (betatest)
Spotchecks should be performed across all supported operating systems, including Mac OSX 10.7:
- Windows XP
- Windows 7 (32-bit & 64-bit)
- Windows Vista
- Mac OS X 10.6
- Mac OS X 10.7
- Ubuntu 11.04 (32-bit and 64-bit)
- Windows 2000
Results
Problems Found
List any found issues or bugs here. Use [6b4] whiteboard entry to keep track.
Automation
Run automated BFTs and link reports from dashboard: Litmus, Mozmill
Platform | General Tests | Restart Tests |
Windows 2000 | PASS | PASS |
Windows XP | PASS | PASS |
Windows Vista | PASS | PASS |
Windows 7 | PASS | PASS |
Windows 7 64-bit | PASS | PASS |
Mac OS X 10.6 | PASS | PASS |
Mac OS X 10.7 | PASS | PASS |
Ubuntu 11.04 | PASS | PASS |
Ubuntu 11.04 64-bit | PASS | PASS |
- Issues Found
Manual
- Smoketests
- Litmus Testrun: https://litmus.mozilla.org/run_tests.cgi?test_run_id=326
- Litmus Report: https://litmus.mozilla.org/test_run_report.cgi?test_run_id=326
Platform | Name of tester | Results |
Windows 2000 | Simona | PASS |
Windows XP | Andrei | PASS |
Windows Vista | Simona | PASS |
Windows 7 | Andrei | PASS |
Windows 7 64-bit | Alin | PASS |
Mac OS X 10.6 | Vlad G | PASS |
Ubuntu 11.04 | George | PASS |
Ubuntu 11.04 64-bit | George | PASS |
Mac OS X 10.7 | Vlad G | PASS |
- Issues Found
- Any major issues?
- n/a
- l10n spotchecking
- German
- Name of tester: Alin
- PASS
- French
- Name of tester: Alin
- PASS
- Spanish
- Name of tester: Alin
- PASS
- Italian
- Name of tester: Alin
- PASS
- Russian
- Name of tester: Alin
- PASS
- Exploratory Testing Around Changes - On Applicable Platforms
- WebGL
- bug 675625 - Firefox crashes when performing WebGL conformance tests on Ubuntu 11.04
- bug 675634 - Firefox crashes while on a page that uses WebGL
- Flash
- bug 675645 - Firefox freezes when using Pop out while in Fullscreen mode on Youtube
- Silverlight
- n/a
- Issues Found
- n/a
Updates
Source Builds: 4.0b*, 4.0, 4.0.1, 5.0b*, 5.0, 6.0b1, 6.0b2, 6.0b3
Source Locales: en-US + one random P1 locale per platform
Update Types: complete+fallback, partial+fallback
Target Build ID: Mozilla/5.0 (Windows NT 5.0; rv:6.0) Gecko/20100101 Firefox/6.0
Platform | Builds | Result | |
Betatest: REPORT | |||
Windows 2000 | Mozilla/5.0 (Windows NT 5.0; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | PASS | |
Windows XP | Mozilla/5.0 (Windows NT 5.1; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | PASS | |
Windows Vista | Mozilla/5.0 (Windows NT 6.0; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | PASS | |
Windows 7 | Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | PASS | |
Windows 7 64-bit | Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | PASS | |
Mac OS X 10.6 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | PASS | |
Mac OS X 10.7 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | PASS | |
Ubuntu 11.04 | Mozilla/5.0 (X11; Linux i686; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | PASS | |
Ubuntu 11.04 64-bit | Mozilla/5.0 (X11; Linux i686 on x86_64; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | PASS | |
Beta: report | |||
Windows 2000 | Mozilla/5.0 (Windows NT 5.0; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | PASS | |
Windows XP | Mozilla/5.0 (Windows NT 5.1; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | PASS | |
Windows Vista | Mozilla/5.0 (Windows NT 6.0; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | PASS | |
Windows 7 | Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | PASS | |
Windows 7 64-bit | Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | PASS | |
Mac OS X 10.6 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | PASS | |
Mac OS X 10.7 | Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | PASS | |
Ubuntu 11.04 | Mozilla/5.0 (X11; Linux i686; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | PASS | |
Ubuntu 11.04 64-bit | Mozilla/5.0 (X11; Linux i686 on x86_64; rv:6.0) Gecko/20100101 Firefox/6.0 (20110729080751) | PASS |
- Issues Found