QA/mozilla-beta/fx6: Difference between revisions

From MozillaWiki
< QA
Jump to navigation Jump to search
 
(16 intermediate revisions by 5 users not shown)
Line 1: Line 1:
<small>[[QA/mozilla-beta/fx6|« mozilla-beta:fx5]] | [[QA/mozilla-beta/fx6|mozilla-beta:fx6]] | [[QA/mozilla-beta/fx7|mozilla-beta:fx7 »]] </small>  
<small>[[QA/mozilla-beta/fx6|« mozilla-beta:fx5]] | [[QA/mozilla-beta/fx6|mozilla-beta:fx6]] | [[QA/mozilla-beta/fx6/Beta_2|mozilla-beta:fx6:Beta 2 »]] </small>  


= Mozilla Beta Test Plan =
= Mozilla Beta Test Plan =
Line 14: Line 14:
* Feature sign-off
* Feature sign-off
* [http://google.com/ Verifying unconfirmed bugs] - [http://bit.ly/m33EyY Bugzilla query]  
* [http://google.com/ Verifying unconfirmed bugs] - [http://bit.ly/m33EyY Bugzilla query]  
* [http://bit.ly/irfY3Z Validating fixes] - [http://bit.ly/mJSjJG Bugzilla query]  
* [http://bit.ly/irfY3Z Validating fixes] - [http://bit.ly/py2wjd Bugzilla query]  
* Crash-stats
* Crash-stats
* Automation strategy
* Automation strategy
Line 27: Line 27:
* [[QA/mozilla-beta/fx6/Bug Verifications | Bug/Litmus Verifications]]
* [[QA/mozilla-beta/fx6/Bug Verifications | Bug/Litmus Verifications]]
* [[QA/mozilla-beta/fx6/Feature Criteria | Feature Criteria]]
* [[QA/mozilla-beta/fx6/Feature Criteria | Feature Criteria]]
* [https://wiki.mozilla.org/QA/mozilla-beta/fx6/Manual_Testing  Manual Testing]
* [[QA/mozilla-beta/fx6/Crash Stats and Feedback | Crash-stats/Feedback/Support]]
* [[QA/mozilla-beta/fx6/Crash Stats and Feedback | Crash-stats/Feedback/Support]]
* [[QA/mozilla-beta/fx6/Test Day | Test Day]]
* [[QA/mozilla-beta/fx6/Test Day | Test Day]]
Line 43: Line 44:
  | Branding
  | Branding
  | juanb
  | juanb
  | ''sign-off ?''
  | SIGNED-OFF
  |-
  |-
  | Automated BFTs
  | Automated BFTs
  | ashughes
  | ashughes
  |  
  | SIGNED-OFF
  |-
  |-
  | Bug Fix Verifications
  | Bug Fix Verifications
  |  
  | team
  |  
  |  
  |-
  |-
  | Accessibility
  | Accessibility
  | marco
  | marco
  |  
  | SIGNED-OFF
  |-
  |-
  | Updates
  | Updates
  | ashughes
  | ashughes
  |  
  | '''in-progress'''
  |-
  |-
  | style="background:#EFEFEF;" colspan="3" | '''Feature Spotchecks'''
  | style="background:#EFEFEF;" colspan="3" | '''Feature Spotchecks'''
Line 158: Line 159:
  | Windows XP
  | Windows XP
  | '''result'''
  | '''result'''
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/ff85e49735292723d78d8a7fa0ef376c REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cb7c9bb PASS]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/ff85e49735292723d78d8a7fa0ef5761 REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cb82e0c PASS]
  |-
  |-
  | Windows Vista
  | Windows Vista
  | '''result'''
  | '''result'''
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/ff85e49735292723d78d8a7fa0ef3eee REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cb79552 PASS]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/ff85e49735292723d78d8a7fa0ef7eeb REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cb8142e PASS]
  |-
  |-
  | Windows 7
  | Windows 7
  | '''result'''
  | '''result'''
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/ff85e49735292723d78d8a7fa0ef42b5 REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cb7b105 PASS]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/ff85e49735292723d78d8a7fa0ef73b0 REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cb83239 PASS]
  |-
  |-
  | Mac OS X 10.6
  | Mac OS X 10.6
  | '''result'''
  | '''result'''
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/ff85e49735292723d78d8a7fa0ef6863 REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cb7f150 PASS] (failures to analyze)
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/ff85e49735292723d78d8a7fa0ef9ae2 REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cb86117 PASS]
  |-
  |-
  | Ubuntu 10.10
  | Ubuntu 10.10
  | '''result'''
  | '''result'''
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/ff85e49735292723d78d8a7fa0ef622e REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cb7c10e PASS]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/ff85e49735292723d78d8a7fa0ef9050 REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cb84728 PASS]
  |-
  |-
  | Ubuntu 10.10 64-bit
  | Ubuntu 10.10 64-bit
  | '''result'''
  | '''result'''
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/ff85e49735292723d78d8a7fa0ef4ec5 REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cb8a9c0 PASS]
  | [http://mozmill-release.brasstacks.mozilla.com/#/general/report/ff85e49735292723d78d8a7fa0ef8161 REPORT]
  | [http://mozmill-release.brasstacks.mozilla.com/#/functional/report/91c2d0c17ff04e7bd83158c06cb901bc PASS]
  |}
  |}


Line 198: Line 199:
  | style="background:#EFEFEF; width:15%" | '''Result'''
  | style="background:#EFEFEF; width:15%" | '''Result'''
  |-
  |-
  | colspan="4" | '''betatest'''
  | colspan="4" | '''Betatest''': [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=betatest&from=2011-07-08&to=2011-07-08&target=6.0 report]
|-
| Windows 2000
| ''Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US''<br>''Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US''
| '''PASS'''<br>'''PASS'''
  |-
  |-
  | Windows XP
  | Windows XP
  | ''Path''
  | ''Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US''<br>''Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US''
  | '''result'''
  | '''PASS'''<br>'''PASS'''
  |-
  |-
  | Windows Vista
  | Windows Vista
  | ''Path''
  | ''Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US''<br>''Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US''
  | '''result'''
  | '''PASS'''<br>'''PASS'''
  |-
  |-
  | Windows 7
  | Windows 7
  | ''Path''
  | ''Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US''<br>''Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US''
  | '''result'''
| '''PASS'''<br>'''PASS'''
|-
| Mac OS X 10.6
| ''Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US''<br>''Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US''
| '''PASS'''<br>'''PASS'''
  |-
| Mac OS X 10.7
| ''Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US''<br>''Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US''
| '''PASS'''<br>'''PASS'''
  |-
  |-
  | Mac OSX 10.6
  | Ubuntu 11.04
  | ''Path''
  | ''Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US''<br>''Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US''
  | '''result'''
  | '''PASS'''<br>'''PASS'''
  |-
  |-
  | Ubuntu 10.10
  | Ubuntu 11.04 64-bit
  | ''Path''
  | ''Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US''<br>''Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US''
  | '''result'''
  | '''PASS'''<br>'''PASS'''
  |-
  |-
  | Ubuntu 10.10 64-bit
  | colspan="4" | '''Beta''': [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=6.0&channel=beta&from=2011-07-01&to=2011-07-08&target=6.0 report]
| ''Path''
| '''result'''
  |-
  |-
  | colspan="4" | '''Beta'''
  | Windows 2000
| ''Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US''<br>''Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US''
| '''PASS'''<br>'''PASS'''
  |-
  |-
  | Windows XP
  | Windows XP
  | ''Path''
  | ''Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US''<br>''Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US''
  | '''result'''
  | '''PASS'''<br>'''PASS'''
  |-
  |-
  | Windows Vista
  | Windows Vista
  | ''Path''
  | ''Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US''<br>''Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US''
  | '''result'''
  | '''PASS'''<br>'''PASS'''
  |-
  |-
  | Windows 7
  | Windows 7
  | ''Path''
  | ''Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US''<br>''Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US''
  | '''result'''
| '''PASS'''<br>'''PASS'''
|-
| Mac OS X 10.6
  | ''Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US''<br>''Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US''
| '''PASS'''<br>'''PASS'''
  |-
  |-
  | Mac OSX 10.6
  | Mac OS X 10.7
  | ''Path''
  | ''Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US''<br>''Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US''
  | '''result'''
  | '''PASS'''<br>'''PASS'''
  |-
  |-
  | Ubuntu 10.10
  | Ubuntu 11.04
  | ''Path''
  | ''Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US''<br>''Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US''
  | '''result'''
  | '''PASS'''<br>'''PASS'''
  |-
  |-
  | Ubuntu 10.10 64-bit
  | Ubuntu 11.04 64-bit
  | ''Path''
  | ''Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US''<br>''Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US''
  | '''result'''
  | '''PASS'''<br>'''PASS'''
  |}
  |}


Line 255: Line 272:


; Issues Found:
; Issues Found:
* n/a
* Crash: 5.0b7 -> 6.0b1 Mac OS X 10.7 on beta channel ([https://crash-stats.mozilla.com/report/index/bp-d093b000-ea9a-489a-be86-672652110708 report])
** Crash not reproducible, reported by juanb


=== Features ===
=== Features ===
Line 264: Line 282:
* Testing Performed: [http://bit.ly/ittE8V Exploratory spreadsheet]
* Testing Performed: [http://bit.ly/ittE8V Exploratory spreadsheet]
* Explored around existing tests: [http://bit.ly/l7kXAn spreadsheet_1] [http://bit.ly/mjsJC2 spreadsheet_2]
* Explored around existing tests: [http://bit.ly/l7kXAn spreadsheet_1] [http://bit.ly/mjsJC2 spreadsheet_2]
* '''Outstanding Issues:''' ....
* '''Outstanding Issues:'''  
** {{bug|669638}} - web console freezes when using firebug


; Web Console Autocompletion Popup(Vlad, Andrei)
; Web Console Autocompletion Popup(Vlad, Andrei)
Line 289: Line 308:
* Platforms Tested: MacOS X 10.6, MacOS X 10.7, Windows Vista x86, Ubuntu 11.04 x86, Win XP x86
* Platforms Tested: MacOS X 10.6, MacOS X 10.7, Windows Vista x86, Ubuntu 11.04 x86, Win XP x86
* Testing Performed: [http://bit.ly/llI8Fg Exploratory spreadsheet]
* Testing Performed: [http://bit.ly/llI8Fg Exploratory spreadsheet]
* '''Outstanding Issues:''' ...
* '''Outstanding Issues:''' No new issues were found


; AOM Plugin Check
; AOM Plugin Check

Latest revision as of 08:55, 19 July 2011

« mozilla-beta:fx5 | mozilla-beta:fx6 | mozilla-beta:fx6:Beta 2 »

Mozilla Beta Test Plan

Description

This is the general mozilla-beta test plan approach. We'll list the testing tasks for this branch as well as their status in relation to the current "version" features are targeted for, and we'll keep an archive of the testing cycles for mozilla-beta.

Current Cycle

The current cycle for mozilla-beta is focused on features and fixes that will go out with Firefox 6 beta 1.

Test Results

Test results:

Bugs found:

  • Track using [fx6] in the whiteboard

Merge Sign-off

Task Owner Status
Branding juanb SIGNED-OFF
Automated BFTs ashughes SIGNED-OFF
Bug Fix Verifications team
Accessibility marco SIGNED-OFF
Updates ashughes in-progress
Feature Spotchecks
Scratchpad Alex Lakatos SIGN-OFF
MatchMedia ashughes
Web Console Vlad Maniac SIGN-OFF
Web Console Autocomplete Popup Vlad Maniac SIGN-OFF
Web Developer Menu Simona Badau SIGN-OFF
About:memory Vlad Ghetiu SIGN-OFF
AOM Plugin Check tester
Site-based Data Management George Carstoiu SIGN-OFF
Tab move/detach Animation Andrei Domuta Not landed yet

Milestone: Firefox 6.0b1

Builds: 6.0b1

Known Issues: The following issues are already known and should not be reported again

  • AMO version check:
    • installed add-ons which are incompatible will be disabled
    • if checkCompatibility is disabled, you may see some add-on related instability
  • About:permission
    • the following issues are still present: bug 660217, bug 661780. No new issues found.

Branding

Check installer, folder, icons, about:, about dialog, and home page for Beta branding

Platform Result Tester
Windows XP PASS AndreiD
Windows Vista PASS Simona
Windows 7 PASS AndreiD
Mac OS X 10.6 PASS Vlad G
Ubuntu 10.10 PASS George
Ubuntu 10.10 64-bit PASS George
Issues Found
  • Mac OS X: Minefield branding is still present in the profile directory and also in Firefox/contents/Mac Os/extensions/"directory"/install.rdf (Vlad G)

Automation

Run automated BFTs and link reports from dashboard

Platform Result General Tests Restart Tests
Windows XP result PASS PASS
Windows Vista result PASS PASS
Windows 7 result PASS PASS
Mac OS X 10.6 result PASS (failures to analyze) PASS
Ubuntu 10.10 result PASS PASS
Ubuntu 10.10 64-bit result PASS PASS
Issues Found
  • n/a

Updates

Check a previous beta build (5.0bX) gets updated to Firefox 6.0b1

Platform Build IDs Result
Betatest: report
Windows 2000 Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US
Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US 		PASS
PASS
Windows XP Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US
Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US 		PASS
PASS
Windows Vista Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US
Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US 		PASS
PASS
Windows 7 Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US
Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US 		PASS
PASS
Mac OS X 10.6 Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US
Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US 		PASS
PASS
Mac OS X 10.7 Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US
Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US 		PASS
PASS
Ubuntu 11.04 Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US
Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US 		PASS
PASS
Ubuntu 11.04 64-bit Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US
Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US 		PASS
PASS
Beta: report
Windows 2000 Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US
Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US 		PASS
PASS
Windows XP Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US
Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US 		PASS
PASS
Windows Vista Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US
Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US 		PASS
PASS
Windows 7 Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US
Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US 		PASS
PASS
Mac OS X 10.6 Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US
Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US 		PASS
PASS
Mac OS X 10.7 Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US
Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US 		PASS
PASS
Ubuntu 11.04 Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US
Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US 		PASS
PASS
Ubuntu 11.04 64-bit Firefox 5.0b6 en-US -> Firefox 6.0b1 en-US
Firefox 5.0b7 en-US -> Firefox 6.0b1 en-US 		PASS
PASS
Known Issues
  • n/a
Issues Found
  • Crash: 5.0b7 -> 6.0b1 Mac OS X 10.7 on beta channel (report)
    • Crash not reproducible, reported by juanb

Features

Identify the feature, description the testing you performed, the platforms you tested on, and any bugs filed

Web Console (Vlad, George)
Web Console Autocompletion Popup(Vlad, Andrei)
  • Platforms Tested: Ubuntu 10.04 x86, Mac OS 10.6, Windows 7 x86, Windows XP x86
  • Testing Performed: Exploratory spreadsheet
  • Outstanding Issues: No new issues were found
Site based Data management (George)
  • Platforms Tested: Ubuntu 11.04 x86, Win XP, Mac OS X 10.6, Win 7 x86
  • Testing Performed: Exploratory spreadsheet
  • Outstanding Issues: No new issues were found
About Memory (VladG)
  • Platforms Tested: Win 7 x86, MacOS X, Win XP x86, Ubuntu 11.04 x86
  • Testing Performed: Exploratory spreadsheet
  • Outstanding Issues: None
Web Developer Menu (Simona)
  • Platforms Tested: Win 7 x86, MacOS X, Win XP x86, Ubuntu 10.04 x86
  • Testing Performed: Exploratory spreadsheet
  • Outstanding Issues: No new issues were found
Scratchpad (Alex)
  • Platforms Tested: MacOS X 10.6, MacOS X 10.7, Windows Vista x86, Ubuntu 11.04 x86, Win XP x86
  • Testing Performed: Exploratory spreadsheet
  • Outstanding Issues: No new issues were found
AOM Plugin Check
  • Platforms Tested: ...
  • Testing Performed: ...
  • Outstanding Issues: ...

Bug Fixes

Verify the bug fixes which landed for this merge.

Bugzilla Query: link

Major Issues Found

Retrieved from "https://wiki.mozilla.org/index.php?title=QA/mozilla-beta/fx6&oldid=330809"