QA/mozilla-beta/fx6/Beta 2: Difference between revisions
< QA | mozilla-beta/fx6
Jump to navigation
Jump to search
(→Tasks) |
|||
(20 intermediate revisions by 8 users not shown) | |||
Line 10: | Line 10: | ||
We should complete the following tasks before signing off on the next weekly beta. We should be focusing on bug verifications and testing around fixes. | We should complete the following tasks before signing off on the next weekly beta. We should be focusing on bug verifications and testing around fixes. | ||
* {{ | * {{done|}} Automation (Mozmill BFTs) - ashughes/whimboo: [[QA/mozilla-beta/fx6/Beta_2#Automation|'''Signed Off''']] | ||
* {{ | * {{done|}} [https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=0d82a53ffaa6&tochange=ea7034005498 Bug Verifications] - Waverley Team: [http://bit.ly/qr7afL '''Signed Off'''] | ||
* {{ | * {{done|}} Manual Testing - Waverley Team: [[QA/mozilla-beta/fx6/Beta_2/Manual_Testing|'''Signed Off''']] | ||
* {{ | * {{done|}} Automation (Updates) - ashughes/whimboo: [[QA/mozilla-beta/fx6/Beta_2#Updates|'''Signed Off''']] | ||
* {{ | * {{done|}} Crash Stats / Feedback / Web Content Sign Off - juanb, marcia: '''Signed Off''' | ||
Spotchecks should be performed across all supported operating systems, including Mac OSX 10.7 which is close to shipping: | Spotchecks should be performed across all supported operating systems, including Mac OSX 10.7 which is close to shipping: | ||
Line 29: | Line 29: | ||
List any found issues or bugs here. Use '''[6b2]''' whiteboard entry to keep track. | List any found issues or bugs here. Use '''[6b2]''' whiteboard entry to keep track. | ||
=== Automation | === Automation === | ||
Run automated BFTs and link reports from dashboard | Run automated BFTs and link reports from dashboard | ||
* Reports: [https://litmus.mozilla.org/test_run_report.cgi?test_run_id=321 Litmus], [http://mozmill-release.brasstacks.mozilla.com/#/functional/reports?branch=6.0&platform=All&from=2011-07-13&to=2011-07-14 Mozmill] | * Reports: [https://litmus.mozilla.org/test_run_report.cgi?test_run_id=321 Litmus], [http://mozmill-release.brasstacks.mozilla.com/#/functional/reports?branch=6.0&platform=All&from=2011-07-13&to=2011-07-14 Mozmill] | ||
{| class="fullwidth-table" | {| class="fullwidth-table" | ||
| style="background:#EFEFEF; width:20%" | '''Platform''' | | style="background:#EFEFEF; width:20%" | '''Platform''' | ||
| style="background:#EFEFEF; width:25%" | '''General Tests''' | | style="background:#EFEFEF; width:25%" | '''General Tests''' | ||
| style="background:#EFEFEF; width:25%" | '''Restart Tests''' | | style="background:#EFEFEF; width:25%" | '''Restart Tests''' | ||
|- | |- | ||
| Windows 2000 | | Windows 2000 | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/7e20178158194d04f7df0ce952de11e3 PASS] | |||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/7e20178158194d04f7df0ce952de30b6 PASS] | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | |||
|- | |- | ||
| Windows XP | | Windows XP | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/7e20178158194d04f7df0ce952dde81c PASS] (known random failure) | |||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/7e20178158194d04f7df0ce952ddfd5e PASS] | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | |||
|- | |- | ||
| Windows Vista | | Windows Vista | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/7e20178158194d04f7df0ce952de0593 PASS] | |||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/7e20178158194d04f7df0ce952de4b56 PASS] | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | |||
|- | |- | ||
| Windows 7 | | Windows 7 | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/7e20178158194d04f7df0ce952ddd2cd PASS] (known failure) | |||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/7e20178158194d04f7df0ce952ddfda1 PASS] | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | |||
|- | |- | ||
| Windows 7 64-bit | | Windows 7 64-bit | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/7e20178158194d04f7df0ce952ddef1c PASS] (known failure) | |||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/7e20178158194d04f7df0ce952de4349 PASS] | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | |||
|- | |- | ||
| Mac OS X 10.6 | | Mac OS X 10.6 | ||
| | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/7e20178158194d04f7df0ce952ddfa66 PASS] (known failure) | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/7e20178158194d04f7df0ce952de3fca PASS] | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | |- | ||
| Mac OS X 10.7 | |||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/7e20178158194d04f7df0ce952de66ec PASS] | |||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/7e20178158194d04f7df0ce952de6b63 PASS] | |||
|- | |- | ||
| Ubuntu 10.10 | | Ubuntu 10.10 | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/7e20178158194d04f7df0ce952dde86a PASS] | |||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/7e20178158194d04f7df0ce952de2935 PASS] | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | |||
|- | |- | ||
| Ubuntu 10.10 64-bit | | Ubuntu 10.10 64-bit | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/7e20178158194d04f7df0ce952dddb52 PASS] (i686) (known failure)<br>[http://mozmill-release.brasstacks.mozilla.com/#/functional/report/7e20178158194d04f7df0ce952de5856 PASS] (x86_64) | |||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/7e20178158194d04f7df0ce952de194e PASS] (i686)<br>[http://mozmill-release.brasstacks.mozilla.com/#/functional/report/7e20178158194d04f7df0ce952de5c7c PASS] (x86_64) | ||
| [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/ | |||
|} | |} | ||
Line 87: | Line 77: | ||
* Any major issues? | * Any major issues? | ||
=== Manual | === Manual === | ||
;Smoketests | ;Smoketests | ||
Litmus Testrun: https://litmus.mozilla.org/run_tests.cgi?test_run_id=321 | Litmus Testrun: https://litmus.mozilla.org/run_tests.cgi?test_run_id=321 | ||
Line 98: | Line 88: | ||
|- | |- | ||
| Windows 2000 | | Windows 2000 | ||
| | | SimonaB | ||
| | | {{bug|671535}} - Ogv videos fail to play on Windows 2000 | ||
|- | |- | ||
| Windows XP | | Windows XP | ||
| | | SimonaB | ||
| | | PASS | ||
|- | |- | ||
| Windows Vista | | Windows Vista | ||
| | | SimonaB | ||
| | | PASS | ||
|- | |- | ||
| Windows 7 | | Windows 7 | ||
| | | Andrei | ||
| | | PASS | ||
|- | |- | ||
| Windows 7 64-bit | | Windows 7 64-bit | ||
| | | Andrei | ||
| | | PASS | ||
|- | |- | ||
| Mac OS X 10.6 | | Mac OS X 10.6 | ||
| | | VladG | ||
| | | PASS | ||
|- | |- | ||
| Ubuntu | | Ubuntu 11.04 | ||
| | | George | ||
| | | PASS | ||
|- | |- | ||
| Ubuntu | | Ubuntu 11.04 64-bit | ||
| | | George | ||
| | | PASS | ||
|- | |- | ||
| Mac OS X 10.7 | | Mac OS X 10.7 | ||
| | | VladG | ||
| | | PASS | ||
|} | |} | ||
;Issues Found | ;Issues Found | ||
* Any major issues? | * Any major issues? | ||
**{{bug|671535}} - Ogv videos fail to play on Windows 2000 | |||
;BFTs | ;BFTs | ||
* WinXP | * WinXP | ||
** Testers: | ** Testers: Andrei, George, Simona, Vlad | ||
** Results: ''' | ** Results: '''PASS''' in Litmus with known issues | ||
;Issues Found | ;Issues Found | ||
* n/a | * n/a | ||
=== Updates | === Updates === | ||
; | '''Target Build ID:''' Mozilla/5.0 (Windows NT 5.0; rv:6.0) Gecko/20100101 Firefox/6.0 (20110713171652) | ||
{| class="fullwidth-table" | {| class="fullwidth-table" | ||
| style="background:#EFEFEF; width: | | style="background:#EFEFEF; width:15%" | '''Platform''' | ||
| style="background:#EFEFEF; width: | | style="background:#EFEFEF; width:75%" | '''Builds''' | ||
| style="background:#EFEFEF; width:10%" | '''Result''' | |||
|- | |||
| colspan="4" | '''Betatest''': [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=betatest&from=2011-07-14&to=2011-07-14&target=6.0 report] | |||
|- | |- | ||
| Windows 2000 | | Windows 2000 | ||
| en-US | | 4.0b12 en-US/es-ES, 4.0 en-US/de, 5.0b7 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru | ||
| '''PASS''' | |||
|- | |- | ||
| Windows XP | | Windows XP | ||
| en-US | | 4.0b9 en-US/es-ES, 4.0 en-US/ko, 5.0b2 en-US/fr, 5.0 en-US/it, 6.0b1 en-US/ru | ||
| '''PASS''' | |||
|- | |- | ||
| Windows Vista | | Windows Vista | ||
| en-US | | 4.0b8 en-US/es-ES, 4.0.1 en-US/it, 5.0b3 en-US/it, 5.0 en-US/zh-TW, 6.0b1 en-US/ru | ||
| '''PASS''' | |||
|- | |- | ||
| Windows 7 | | Windows 7 | ||
| en-US | | 4.0b8 en-US/es-ES, 4.0.1 en-US/de, 5.0b1 en-US, 5.0 en-US/zh-TW, 6.0b1 en-US/ru | ||
| '''PASS''' | |||
|- | |- | ||
| Windows 7 64-bit | | Windows 7 64-bit | ||
| en-US | | 4.0b8 en-US/es-ES, 4.0 en-US/de, 5.0b6 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru | ||
| '''PASS''' | |||
|- | |- | ||
| Mac OS X 10.6 | | Mac OS X 10.6 | ||
| en-US | | 4.0b10 en-US/de, 4.0.1 en-US/ja-JP-mac, 5.0b3 en-US/ja-JP-mac, 5.0 en-US/zh-TW, 6.0b1 en-US/es-ES | ||
| en-US | | '''PASS''' | ||
| en-US | |- | ||
| Mac OS X 10.7 | |||
| | |||
| | |||
|- | |||
| Ubuntu 11.04 | |||
| 4.0b11 en-US/es-ES, 4.0.1 en-US/de, 5.0b5 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ko | |||
| '''PASS''' | |||
|- | |||
| Ubuntu 11.04 64-bit | |||
| 4.0b12 en-US/es-ES, 4.0 en-US/de, 5.0b7 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru | |||
| '''PASS''' | |||
|- | |||
| colspan="4" | '''Beta''': [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=beta&from=2011-07-15&to=2011-07-15&target=6.0 report] | |||
|- | |||
| Windows 2000 | |||
| 4.0b12 en-US/es-ES, 4.0 en-US/de, 5.0b7 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru | |||
| '''PASS''' | |||
|- | |||
| Windows XP | |||
| 4.0b9 en-US/es-ES, 4.0 en-US/ko, 5.0b2 en-US/fr, 5.0 en-US/it, 6.0b1 en-US/ru | |||
| '''PASS''' | |||
|- | |||
| Windows Vista | |||
| 4.0b8 en-US/es-ES, 4.0.1 en-US/it, 5.0b3 en-US/it, 5.0 en-US/zh-TW, 6.0b1 en-US/ru | |||
| '''PASS''' | |||
|- | |- | ||
| | | Windows 7 | ||
| en-US | | 4.0b8 en-US/es-ES, 4.0.1 en-US/de, 5.0b1 en-US, 5.0 en-US/zh-TW, 6.0b1 en-US/ru | ||
| en-US | | '''PASS''' | ||
|- | |||
| Windows 7 64-bit | |||
| 4.0b8 en-US/es-ES, 4.0 en-US/de, 5.0b6 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru | |||
| '''PASS''' | |||
|- | |- | ||
| | | Mac OS X 10.6 | ||
| en-US | | 4.0b10 en-US/de, 4.0.1 en-US/ja-JP-mac, 5.0b3 en-US/ja-JP-mac, 5.0 en-US/zh-TW, 6.0b1 en-US/es-ES | ||
| '''PASS''' | |||
|- | |- | ||
| Mac OS X 10.7 | | Mac OS X 10.7 | ||
| en-US | | | ||
| en-US | | | ||
| | |- | ||
| Ubuntu 11.04 | |||
| 4.0b11 en-US/es-ES, 4.0.1 en-US/de, 5.0b5 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ko | |||
| '''PASS''' | |||
|- | |||
| Ubuntu 11.04 64-bit | |||
| 4.0b12 en-US/es-ES, 4.0 en-US/de, 5.0b7 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru | |||
| '''PASS''' | |||
|- | |||
|} | |} | ||
; Issues Found | ; Issues Found | ||
* Any major issues? | * Any major issues? |
Latest revision as of 22:21, 15 July 2011
« mozilla-beta:fx6 | mozilla-beta:fx6:Beta 2 | mozilla-beta:fx6:Beta 3 »
Beta 2 Test Plan and Results
- Release: Thursday/Friday July 15th/16th
- QA Contact: Juan Becerra
- IRC: irc://irc.mozilla.org#qa juanb, ashughes, whimboo
- Builds: Firefox 6.0b2 | pushlog
Tasks
We should complete the following tasks before signing off on the next weekly beta. We should be focusing on bug verifications and testing around fixes.
- [DONE] Automation (Mozmill BFTs) - ashughes/whimboo: Signed Off
- [DONE] Bug Verifications - Waverley Team: Signed Off
- [DONE] Manual Testing - Waverley Team: Signed Off
- [DONE] Automation (Updates) - ashughes/whimboo: Signed Off
- [DONE] Crash Stats / Feedback / Web Content Sign Off - juanb, marcia: Signed Off
Spotchecks should be performed across all supported operating systems, including Mac OSX 10.7 which is close to shipping:
- Windows XP
- Windows 7 (32-bit & 64-bit)
- Windows Vista
- Mac OSX 10.6
- Ubuntu 10.10 (32-bit and 64-bit)
- Windows 2000
- Mac OSX 10.7
Results
Problems Found
List any found issues or bugs here. Use [6b2] whiteboard entry to keep track.
Automation
Run automated BFTs and link reports from dashboard
Platform | General Tests | Restart Tests |
Windows 2000 | PASS | PASS |
Windows XP | PASS (known random failure) | PASS |
Windows Vista | PASS | PASS |
Windows 7 | PASS (known failure) | PASS |
Windows 7 64-bit | PASS (known failure) | PASS |
Mac OS X 10.6 | PASS (known failure) | PASS |
Mac OS X 10.7 | PASS | PASS |
Ubuntu 10.10 | PASS | PASS |
Ubuntu 10.10 64-bit | PASS (i686) (known failure) PASS (x86_64) |
PASS (i686) PASS (x86_64) |
- Issues Found
- Any major issues?
Manual
- Smoketests
Litmus Testrun: https://litmus.mozilla.org/run_tests.cgi?test_run_id=321
Platform | Name of tester | Results |
Windows 2000 | SimonaB | bug 671535 - Ogv videos fail to play on Windows 2000 |
Windows XP | SimonaB | PASS |
Windows Vista | SimonaB | PASS |
Windows 7 | Andrei | PASS |
Windows 7 64-bit | Andrei | PASS |
Mac OS X 10.6 | VladG | PASS |
Ubuntu 11.04 | George | PASS |
Ubuntu 11.04 64-bit | George | PASS |
Mac OS X 10.7 | VladG | PASS |
- Issues Found
- Any major issues?
- bug 671535 - Ogv videos fail to play on Windows 2000
- BFTs
- WinXP
- Testers: Andrei, George, Simona, Vlad
- Results: PASS in Litmus with known issues
- Issues Found
- n/a
Updates
Target Build ID: Mozilla/5.0 (Windows NT 5.0; rv:6.0) Gecko/20100101 Firefox/6.0 (20110713171652)
Platform | Builds | Result | |
Betatest: report | |||
Windows 2000 | 4.0b12 en-US/es-ES, 4.0 en-US/de, 5.0b7 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru | PASS | |
Windows XP | 4.0b9 en-US/es-ES, 4.0 en-US/ko, 5.0b2 en-US/fr, 5.0 en-US/it, 6.0b1 en-US/ru | PASS | |
Windows Vista | 4.0b8 en-US/es-ES, 4.0.1 en-US/it, 5.0b3 en-US/it, 5.0 en-US/zh-TW, 6.0b1 en-US/ru | PASS | |
Windows 7 | 4.0b8 en-US/es-ES, 4.0.1 en-US/de, 5.0b1 en-US, 5.0 en-US/zh-TW, 6.0b1 en-US/ru | PASS | |
Windows 7 64-bit | 4.0b8 en-US/es-ES, 4.0 en-US/de, 5.0b6 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru | PASS | |
Mac OS X 10.6 | 4.0b10 en-US/de, 4.0.1 en-US/ja-JP-mac, 5.0b3 en-US/ja-JP-mac, 5.0 en-US/zh-TW, 6.0b1 en-US/es-ES | PASS | |
Mac OS X 10.7 | |||
Ubuntu 11.04 | 4.0b11 en-US/es-ES, 4.0.1 en-US/de, 5.0b5 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ko | PASS | |
Ubuntu 11.04 64-bit | 4.0b12 en-US/es-ES, 4.0 en-US/de, 5.0b7 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru | PASS | |
Beta: report | |||
Windows 2000 | 4.0b12 en-US/es-ES, 4.0 en-US/de, 5.0b7 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru | PASS | |
Windows XP | 4.0b9 en-US/es-ES, 4.0 en-US/ko, 5.0b2 en-US/fr, 5.0 en-US/it, 6.0b1 en-US/ru | PASS | |
Windows Vista | 4.0b8 en-US/es-ES, 4.0.1 en-US/it, 5.0b3 en-US/it, 5.0 en-US/zh-TW, 6.0b1 en-US/ru | PASS | |
Windows 7 | 4.0b8 en-US/es-ES, 4.0.1 en-US/de, 5.0b1 en-US, 5.0 en-US/zh-TW, 6.0b1 en-US/ru | PASS | |
Windows 7 64-bit | 4.0b8 en-US/es-ES, 4.0 en-US/de, 5.0b6 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru | PASS | |
Mac OS X 10.6 | 4.0b10 en-US/de, 4.0.1 en-US/ja-JP-mac, 5.0b3 en-US/ja-JP-mac, 5.0 en-US/zh-TW, 6.0b1 en-US/es-ES | PASS | |
Mac OS X 10.7 | |||
Ubuntu 11.04 | 4.0b11 en-US/es-ES, 4.0.1 en-US/de, 5.0b5 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ko | PASS | |
Ubuntu 11.04 64-bit | 4.0b12 en-US/es-ES, 4.0 en-US/de, 5.0b7 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru | PASS |
- Issues Found
- Any major issues?