QA/mozilla-beta/fx6/Beta 3: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(11 intermediate revisions by 5 users not shown)
Line 10: Line 10:
We should complete the following tasks before signing off on the next weekly beta. We should be focusing on bug verifications and testing around fixes.
We should complete the following tasks before signing off on the next weekly beta. We should be focusing on bug verifications and testing around fixes.


* {{ok|}} Automation (Mozmill BFTs) - ashughes/whimboo: [[QA/mozilla-beta/fx6/Beta_3#Automation|'''result''']]
* {{done|}} Automation (Mozmill BFTs) - ashughes/whimboo: [[QA/mozilla-beta/fx6/Beta_3#Automation|'''Signed Off''']]
* {{ok|}} [http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=FIREFOX_6_0b2_RELEASE&tochange=FIREFOX_6_0b3_RELEASE Bug Verifications] - Waverley Team: [http://bit.ly/qr7afL '''result''']
* {{done|}} [http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=FIREFOX_6_0b2_RELEASE&tochange=FIREFOX_6_0b3_RELEASE Bug Verifications] - Waverley Team: [http://bit.ly/qsyEKl '''Signed Off''']
* {{ok|}} Manual Testing - Waverley Team: [[QA/mozilla-beta/fx6/Beta_3#Manual|'''result''']]
* {{done|}} Manual Testing - Waverley Team: [[QA/mozilla-beta/fx6/Beta_3#Manual|'''Signed Off''']]
* {{ok|}} Automation (Updates) - ashughes/whimboo: [[QA/mozilla-beta/fx6/Beta_3#Updates|'''result''']]
* {{ok|}} Automation (Updates) - ashughes/whimboo: [[QA/mozilla-beta/fx6/Beta_3#Updates|'''result''']]
* {{ok|}} Crash Stats / Feedback / Web Content Sign Off - juanb, marcia: '''result'''
* {{ok|}} Crash Stats / Feedback / Web Content Sign Off - juanb, marcia: '''result'''
Line 89: Line 89:
| Windows 2000  
| Windows 2000  
| Simona
| Simona
| RESULT
| PASS
|-
|-
| Windows XP  
| Windows XP  
| Simona
| Simona
| RESULT
| PASS
|-
|-
| Windows Vista  
| Windows Vista  
| Simona
| Simona
| RESULT
| PASS
|-
|-
| Windows 7  
| Windows 7  
| Andrei
| Andrei
| RESULT
| PASS
|-
|-
| Windows 7 64-bit  
| Windows 7 64-bit  
| Andrei
| Andrei
| RESULT
| PASS
|-
|-
| Mac OS X 10.6  
| Mac OS X 10.6  
| George
| George
| RESULT
| PASS
|-
|-
| Ubuntu 11.04
| Ubuntu 11.04
| George
| George
| RESULT
| PASS
|-
|-
| Ubuntu 11.04 64-bit  
| Ubuntu 11.04 64-bit  
| George
| George
| RESULT
| PASS
|-
|-
| Mac OS X 10.7
| Mac OS X 10.7
| George
| George
| RESULT
| PASS
|}
|}


Line 130: Line 130:
;BFTs
;BFTs
* WinXP  
* WinXP  
** Testers: Simona, Andrei, Vlad, George
** Testers: Simona, Andrei, George
** Results: '''RESULT''' in Litmus
*** [PASS] - Installation - History (Simona)
*** [PASS] - Sync - Add-ons (Andrei)
*** [PASS] - Plug Ins - About:permissions (George)
** Results: '''PASS''' in Litmus with known issues


;Issues Found
;Issues Found
Line 144: Line 147:
  | style="background:#EFEFEF; width:10%" | '''Result'''
  | style="background:#EFEFEF; width:10%" | '''Result'''
  |-
  |-
  | colspan="4" | '''Betatest''': [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=betatest&from=2011-07-20&to=2011-07-22&target=6.0 report]
  | colspan="4" | '''Betatest''': [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=betatest&from=2011-07-22&to=2011-07-23&target=6.0 report]
  |-
  |-
  | Windows 2000
  | Windows 2000
  | 4.0b12 en-US/es-ES, 4.0 en-US/de, 5.0b7 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru
  | 4.0b12 en-US/es-ES, 4.0 en-US/de, 5.0b7 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru
  | '''RESULT'''
  | '''PASS'''
  |-
  |-
  | Windows XP
  | Windows XP
  | 4.0b9 en-US/es-ES, 4.0 en-US/ko, 5.0b2 en-US/fr, 5.0 en-US/it, 6.0b1 en-US/ru
  | 4.0b9 en-US/es-ES, 4.0 en-US/ko, 5.0b2 en-US/fr, 5.0 en-US/it, 6.0b1 en-US/ru
  | '''RESULT'''
  | '''PASS'''
  |-
  |-
  | Windows Vista
  | Windows Vista
  | 4.0b8 en-US/es-ES, 4.0.1 en-US/it, 5.0b3 en-US/it, 5.0 en-US/zh-TW, 6.0b1 en-US/ru
  | 4.0b8 en-US/es-ES, 4.0.1 en-US/it, 5.0b3 en-US/it, 5.0 en-US/zh-TW, 6.0b1 en-US/ru
  | '''RESULT'''
  | '''PASS'''
  |-
  |-
  | Windows 7
  | Windows 7
  | 4.0b8 en-US/es-ES, 4.0.1 en-US/de, 5.0b1 en-US, 5.0 en-US/zh-TW, 6.0b1 en-US/ru
  | 4.0b8 en-US/es-ES, 4.0.1 en-US/de, 5.0b1 en-US, 5.0 en-US/zh-TW, 6.0b1 en-US/ru
  | '''RESULT'''
  | '''PASS'''
  |-
  |-
  | Windows 7 64-bit
  | Windows 7 64-bit
  | 4.0b8 en-US/es-ES, 4.0 en-US/de, 5.0b6 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru
  | 4.0b8 en-US/es-ES, 4.0 en-US/de, 5.0b6 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru
  | '''RESULT'''
  | '''PASS'''
  |-
  |-
  | Mac OS X 10.6
  | Mac OS X 10.6
  | 4.0b10 en-US/de, 4.0.1 en-US/ja-JP-mac, 5.0b3 en-US/ja-JP-mac, 5.0 en-US/zh-TW, 6.0b1 en-US/es-ES
  | 4.0b10 en-US/de, 4.0.1 en-US/ja-JP-mac, 5.0b3 en-US/ja-JP-mac, 5.0 en-US/zh-TW, 6.0b1 en-US/es-ES
  | '''RESULT'''
  | '''PASS'''
  |-
  |-
  | Mac OS X 10.7
  | Mac OS X 10.7
Line 176: Line 179:
  | Ubuntu 11.04
  | Ubuntu 11.04
  | 4.0b11 en-US/es-ES, 4.0.1 en-US/de, 5.0b5 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ko
  | 4.0b11 en-US/es-ES, 4.0.1 en-US/de, 5.0b5 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ko
  | '''RESULT'''
  | '''PASS'''
  |-
  |-
  | Ubuntu 11.04 64-bit
  | Ubuntu 11.04 64-bit
  | 4.0b12 en-US/es-ES, 4.0 en-US/de, 5.0b7 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru  
  | 4.0b12 en-US/es-ES, 4.0 en-US/de, 5.0b7 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru  
  | '''RESULT'''
  | '''PASS'''
  |-
  |-
  | colspan="4" | '''Beta''': [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=beta&from=2011-07-20&to=2011-07-22&target=6.0 report]
  | colspan="4" | '''Beta''': [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=beta&from=2011-07-25&to=2011-07-25&target=6.0 report]
  |-
  |-
  | Windows 2000
  | Windows 2000
  | 4.0b12 en-US/es-ES, 4.0 en-US/de, 5.0b7 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru
  | 4.0b12 en-US/es-ES, 4.0 en-US/de, 5.0b7 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru
  | '''RESULT'''
  | '''PASS'''
  |-
  |-
  | Windows XP
  | Windows XP
  | 4.0b9 en-US/es-ES, 4.0 en-US/ko, 5.0b2 en-US/fr, 5.0 en-US/it, 6.0b1 en-US/ru
  | 4.0b9 en-US/es-ES, 4.0 en-US/ko, 5.0b2 en-US/fr, 5.0 en-US/it, 6.0b1 en-US/ru
  | '''RESULT'''
  | '''PASS'''
  |-
  |-
  | Windows Vista
  | Windows Vista
  | 4.0b8 en-US/es-ES, 4.0.1 en-US/it, 5.0b3 en-US/it, 5.0 en-US/zh-TW, 6.0b1 en-US/ru
  | 4.0b8 en-US/es-ES, 4.0.1 en-US/it, 5.0b3 en-US/it, 5.0 en-US/zh-TW, 6.0b1 en-US/ru
  | '''RESULT'''
  | '''PASS'''
  |-
  |-
  | Windows 7
  | Windows 7
  | 4.0b8 en-US/es-ES, 4.0.1 en-US/de, 5.0b1 en-US, 5.0 en-US/zh-TW, 6.0b1 en-US/ru
  | 4.0b8 en-US/es-ES, 4.0.1 en-US/de, 5.0b1 en-US, 5.0 en-US/zh-TW, 6.0b1 en-US/ru
  | '''RESULT'''
  | '''PASS'''
  |-
  |-
  | Windows 7 64-bit
  | Windows 7 64-bit
  | 4.0b8 en-US/es-ES, 4.0 en-US/de, 5.0b6 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru
  | 4.0b8 en-US/es-ES, 4.0 en-US/de, 5.0b6 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru
  | '''RESULT'''
  | '''PASS'''
  |-
  |-
  | Mac OS X 10.6
  | Mac OS X 10.6
  | 4.0b10 en-US/de, 4.0.1 en-US/ja-JP-mac, 5.0b3 en-US/ja-JP-mac, 5.0 en-US/zh-TW, 6.0b1 en-US/es-ES
  | 4.0b10 en-US/de, 4.0.1 en-US/ja-JP-mac, 5.0b3 en-US/ja-JP-mac, 5.0 en-US/zh-TW, 6.0b1 en-US/es-ES
  | '''RESULT'''
  | '''PASS'''
  |-
  |-
  | Mac OS X 10.7
  | Mac OS X 10.7
Line 214: Line 217:
  | Ubuntu 11.04
  | Ubuntu 11.04
  | 4.0b11 en-US/es-ES, 4.0.1 en-US/de, 5.0b5 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ko
  | 4.0b11 en-US/es-ES, 4.0.1 en-US/de, 5.0b5 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ko
  | '''RESULT'''
  | '''PASS'''
  |-
  |-
  | Ubuntu 11.04 64-bit
  | Ubuntu 11.04 64-bit
  | 4.0b12 en-US/es-ES, 4.0 en-US/de, 5.0b7 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru  
  | 4.0b12 en-US/es-ES, 4.0 en-US/de, 5.0b7 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru  
  | '''RESULT'''
  | '''PASS'''
  |-
  |-
  |}
  |}

Latest revision as of 22:56, 25 July 2011

« mozilla-beta:fx6 | mozilla-beta:fx6:Beta 2 | mozilla-beta:fx6:Beta 3 | mozilla-beta:fx6:Beta 4 »

Beta 3 Test Plan and Results

Tasks

We should complete the following tasks before signing off on the next weekly beta. We should be focusing on bug verifications and testing around fixes.

  • [DONE] Automation (Mozmill BFTs) - ashughes/whimboo: Signed Off
  • [DONE] Bug Verifications - Waverley Team: Signed Off
  • [DONE] Manual Testing - Waverley Team: Signed Off
  • [ON TRACK] Automation (Updates) - ashughes/whimboo: result
  • [ON TRACK] Crash Stats / Feedback / Web Content Sign Off - juanb, marcia: result

Spotchecks should be performed across all supported operating systems, including Mac OSX 10.7:

  • Windows XP
  • Windows 7 (32-bit & 64-bit)
  • Windows Vista
  • Mac OSX 10.6
  • Ubuntu 10.10 (32-bit and 64-bit)
  • Windows 2000
  • Mac OSX 10.7

Results

Problems Found

List any found issues or bugs here. Use [6b3] whiteboard entry to keep track.

Automation

Run automated BFTs and link reports from dashboard

Platform General Tests Restart Tests
Windows 2000 PASS PASS
Windows XP PASS PASS
Windows Vista PASS PASS
Windows 7 PASS PASS
Windows 7 64-bit PASS PASS
Mac OS X 10.6 PASS PASS
Mac OS X 10.7 PASS PASS
Ubuntu 11.04 PASS PASS
Ubuntu 11.04 64-bit PASS PASS
Issues Found
  • No NEW issues were found

Manual

Smoketests

Litmus Testrun: https://litmus.mozilla.org/run_tests.cgi?test_run_id=325

Platform Name of tester Results
Windows 2000 Simona PASS
Windows XP Simona PASS
Windows Vista Simona PASS
Windows 7 Andrei PASS
Windows 7 64-bit Andrei PASS
Mac OS X 10.6 George PASS
Ubuntu 11.04 George PASS
Ubuntu 11.04 64-bit George PASS
Mac OS X 10.7 George PASS
Issues Found
  • Any major issues?
    • n/a
BFTs
  • WinXP
    • Testers: Simona, Andrei, George
      • [PASS] - Installation - History (Simona)
      • [PASS] - Sync - Add-ons (Andrei)
      • [PASS] - Plug Ins - About:permissions (George)
    • Results: PASS in Litmus with known issues
Issues Found
  • n/a

Updates

Target Build ID: Mozilla/5.0 (Windows NT 5.0; rv:6.0) Gecko/20100101 Firefox/6.0 (???)

Platform Builds Result
Betatest: report
Windows 2000 4.0b12 en-US/es-ES, 4.0 en-US/de, 5.0b7 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru PASS
Windows XP 4.0b9 en-US/es-ES, 4.0 en-US/ko, 5.0b2 en-US/fr, 5.0 en-US/it, 6.0b1 en-US/ru PASS
Windows Vista 4.0b8 en-US/es-ES, 4.0.1 en-US/it, 5.0b3 en-US/it, 5.0 en-US/zh-TW, 6.0b1 en-US/ru PASS
Windows 7 4.0b8 en-US/es-ES, 4.0.1 en-US/de, 5.0b1 en-US, 5.0 en-US/zh-TW, 6.0b1 en-US/ru PASS
Windows 7 64-bit 4.0b8 en-US/es-ES, 4.0 en-US/de, 5.0b6 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru PASS
Mac OS X 10.6 4.0b10 en-US/de, 4.0.1 en-US/ja-JP-mac, 5.0b3 en-US/ja-JP-mac, 5.0 en-US/zh-TW, 6.0b1 en-US/es-ES PASS
Mac OS X 10.7
Ubuntu 11.04 4.0b11 en-US/es-ES, 4.0.1 en-US/de, 5.0b5 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ko PASS
Ubuntu 11.04 64-bit 4.0b12 en-US/es-ES, 4.0 en-US/de, 5.0b7 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru PASS
Beta: report
Windows 2000 4.0b12 en-US/es-ES, 4.0 en-US/de, 5.0b7 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru PASS
Windows XP 4.0b9 en-US/es-ES, 4.0 en-US/ko, 5.0b2 en-US/fr, 5.0 en-US/it, 6.0b1 en-US/ru PASS
Windows Vista 4.0b8 en-US/es-ES, 4.0.1 en-US/it, 5.0b3 en-US/it, 5.0 en-US/zh-TW, 6.0b1 en-US/ru PASS
Windows 7 4.0b8 en-US/es-ES, 4.0.1 en-US/de, 5.0b1 en-US, 5.0 en-US/zh-TW, 6.0b1 en-US/ru PASS
Windows 7 64-bit 4.0b8 en-US/es-ES, 4.0 en-US/de, 5.0b6 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru PASS
Mac OS X 10.6 4.0b10 en-US/de, 4.0.1 en-US/ja-JP-mac, 5.0b3 en-US/ja-JP-mac, 5.0 en-US/zh-TW, 6.0b1 en-US/es-ES PASS
Mac OS X 10.7
Ubuntu 11.04 4.0b11 en-US/es-ES, 4.0.1 en-US/de, 5.0b5 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ko PASS
Ubuntu 11.04 64-bit 4.0b12 en-US/es-ES, 4.0 en-US/de, 5.0b7 en-US/fr, 5.0 en-US/zh-TW, 6.0b1 en-US/ru PASS
Issues Found
  • Any major issues?
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/mozilla-beta/fx6/Beta_3&oldid=333259"