MozillaWiki

Releases/Firefox 17/Test Plan: Difference between revisions

Page Discussion

Releases/Firefox 17/Test Plan (view source)

Revision as of 23:43, 30 November 2012

262 bytes added ,  30 November 2012
→‎Update Testing
 
(24 intermediate revisions by 5 users not shown)
Line 9: Line 9:
  |-
  |-
  | '''Status'''
  | '''Status'''
  | Currently in '''Beta''' -- merging to Released on November 20, 2012
  | '''Released''' on Tuesday, November 20, 2012
  |-
  |-
  | '''Lead'''
  | '''Lead'''
Line 1,851: Line 1,851:
  |-
  |-
  | '''ID'''
  | '''ID'''
  | id
  | 20121128204232
  |-
  |-
  | '''Changeset'''
  | '''Changeset'''
  | [http://hg.mozilla.org/releases/mozilla-release/rev/id id] ([http://hg.mozilla.org/releases/mozilla-release/pushloghtml?fromchange=919435c6f654&tochange=id pushlog])
  | [http://hg.mozilla.org/releases/mozilla-release/rev/c23c45132139 c23c45132139] ([http://hg.mozilla.org/releases/mozilla-release/pushloghtml?fromchange=919435c6f654&tochange=c23c45132139 pushlog])
  |}
  |}


; Checklist (respin for {{bug|813264}})
; Checklist
* {{|}} [[Releases/Firefox_17/Test_Plan#Automated_Testing_9|Automated Testing]]
* {{done|}} [[Releases/Firefox_17/Test_Plan#Automated_Testing_9|Automated Testing]]
* {{|}} [[Releases/Firefox_17/Test_Plan#Manual_Testing_9|Manual Testing]]
* {{done|}} [[Releases/Firefox_17/Test_Plan#Manual_Testing_9|Manual Testing]]
* {{|}} [[Releases/Firefox_17/Test_Plan#Update_Testing_9|Update Testing]] on betatest
* {{done|}} [[Releases/Firefox_17/Test_Plan#Manual_Testing_9|Bug Fix Verifications]]
* {{|}} Functional sign-off
* {{done|}} [[Releases/Firefox_17/Test_Plan#Update_Testing_9|Update Testing]] on betatest
* {{done|}} Sign-off email sent to drivers
* {{|}} [[Releases/Firefox_17/Test_Plan#Update_Testing_9|Update Testing]] on release
* {{|}} [[Releases/Firefox_17/Test_Plan#Update_Testing_9|Update Testing]] on release


Line 1,869: Line 1,870:
  |-
  |-
  | class="header" style="width:20%" | Platform
  | class="header" style="width:20%" | Platform
  | class="header" style="width:30%" | Build#1 Results ([ report])
  | class="header" style="width:30%" | Build#1 Results
  |-
  |-
  | Windows XP
  | Windows XP
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-29 ([http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d3716718499d functional], [http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d3716718b5ca restart])
  |-
  |-
  | Windows Vista
  | Windows Vista
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-29 ([http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d3716717e614 functional], [http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d37167187a06 restart])
  |-
  |-
  | Windows 7 32-bit
  | Windows 7 32-bit
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-29 ([http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d3716717cb86 functional], [http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d371671815ed restart])
  |-
  |-
  | Windows 7 64-bit
  | Windows 7 64-bit
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-29 ([http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d3716717cac0 functional], [http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d3716717f43d restart])
  |-
  |-
  | Windows 8 32-bit
  | Windows 8 32-bit
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-29 ([http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d3716717deed functional], [http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d37167181013 restart])
  |-
  |-
  | Windows 8 64-bit
  | Windows 8 64-bit
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-29 ([http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d3716717da97 functional], [http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d371671800a9 restart])
  |-
  |-
  | Mac OS X 10.6
  | Mac OS X 10.6
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-29 ([http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d3716716a9ef functional], [http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d371671725a9 restart])
  |-
  |-
  | Mac OS X 10.7
  | Mac OS X 10.7
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-29 ([http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d3716716edf4 functional], [http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d3716717372a restart])
  |-
  |-
  | Mac OS X 10.8
  | Mac OS X 10.8
  | '''result''' - YYYY-MM-DD
  | '''1 failure''' - 2012-11-29 ([http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d371671688af functional], [http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d3716716fa4b restart])
  |-
  |-
  | Ubuntu 12.04 32-bit
  | Ubuntu 12.04 32-bit
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-29 ([http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d37167168403 functional], [http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d3716716dea6 restart])
  |-
  |-
  | Ubuntu 12.04 64-bit
  | Ubuntu 12.04 64-bit
  | '''result''' - YYYY-MM-DD
  | '''1 failure''' - 2012-11-29 ([http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d371671667fd functional], [http://mozmill-ci.blargon7.com/#/functional/report/352218d7e3125c857fc1d37167169a92 restart])
  |}
  |}


; Test Failures ([https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA known bugs])
; Test Failures ([https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA known bugs])
* '''[known]''' restartTests/testAddons_uninstallExtension/test4.js ({{bug|783484}})
* '''[known]''' restartTests/testAddons_pluginDisabledAfterRestart/test2.js ({{bug|780957}})


=== Manual Testing ===
=== Manual Testing ===
Line 1,920: Line 1,923:
  | Windows XP
  | Windows XP
  | Paul
  | Paul
  | '''In progress''' - 2012-11-29
  | '''PASS''' - 2012-11-29
  |  
  |  
  |-
  |-
  | Windows 8
  | Windows 8
  | Ioana
  | Ioana
  | '''In progress''' - 2012-11-29
  | '''PASS''' - 2012-11-29
  |  
  |  
  |-
  |-
  | Mac OSX 10.8
  | Mac OSX 10.8
  | Virgil
  | Virgil
  | '''In progress''' - 2012-11-29
  | '''PASS''' - 2012-11-29
  |  
  |  
  |-
  |-
  | Ubuntu 12.04
  | Ubuntu 12.04
  | Simona
  | Simona
  | '''In progress''' - 2012-11-29
  | '''PASS''' - 2012-11-29
  |  
  |  
  |-
  |-
Line 1,944: Line 1,947:
  |-
  |-
  | Update regression tests
  | Update regression tests
  | tester
  |  
  | '''results''' - 2012-11-29
  | '''PASS''' - 2012-11-29
  |  
  |  
  |}
  |}
Additional Spotchecks
* ['''result'''] Mac OSX 10.5 Desupport (tester)


=== Bug Fix Verifications ===
=== Bug Fix Verifications ===
; NOTICE all of these verifications need to be performed against 17.0.1 and [https://wiki.mozilla.org/Releases/Firefox_17_ESR/Test_Plan#Bug_Fix_Verifications_2 17.0.1esr]
==== [VERIFIED FIXED] {{bug|806192}} crash in inDOMUtils::GetBindingURLs with Firebug and ShareMeNot ====
 
==== {{bug|806192}} crash in inDOMUtils::GetBindingURLs with Firebug and ShareMeNot ====
{| class="querytable"
{| class="querytable"
  | class="header" style="width:55% |  
  | class="header" style="width:55% |  
Line 1,982: Line 1,980:
  |}
  |}


==== {{bug|815042}} leaving private browsing with social enabled doesn't reset all social components ====
==== [VERIFIED FIXED] {{bug|815042}} leaving private browsing with social enabled doesn't reset all social components ====
{| class="querytable"
{| class="querytable"
  | class="header" style="width:55% |  
  | class="header" style="width:55% |  
Line 2,001: Line 1,999:
  | colspan="4" | '''Notes'''
  | colspan="4" | '''Notes'''
* Social API can't be enabled from the Facebook activation page. Commented and reopened {{bug|807915}}. Social API can be enabled only by setting the preferences social.active and social.enabled to true in about:config.
* Social API can't be enabled from the Facebook activation page. Commented and reopened {{bug|807915}}. Social API can be enabled only by setting the preferences social.active and social.enabled to true in about:config.
* Social API restores after exiting Private browsing if it was enabled in normal mode. Social API does not restore after exiting Private Browsing mode if it wasn't enabled in normal mode. Social API is not restored in Private Browsing mode if it was previously enabled only in normal mode.
* When exiting Private Browsing mode, Social API restores to the state it was in before entering PB mode.
  |}
  |}


==== {{bug|815743}} Backout the UA changes in bug 588909 ====
==== [VERIFIED FIXED] {{bug|815743}} Backout the UA changes in bug 588909 ====
{| class="querytable"
{| class="querytable"
  | class="header" style="width:55% |  
  | class="header" style="width:55% |  
Line 2,052: Line 2,050:
  |}
  |}


==== {{bug|814101}} Font Issue in FF 17 ====
==== [VERIFIED FIXED] {{bug|814101}} Font Issue in FF 17 ====
{| class="querytable"
{| class="querytable"
  | class="header" |  
  | class="header" |  
Line 2,061: Line 2,059:
  |-
  |-
  | about:support shows Direct2D enabled = true
  | about:support shows Direct2D enabled = true
  | result (Mihaela)
  | PASS (Mihaela)
  | result (Paul)
  | PASS(Paul)
  | result (Mihaela)
  | PASS (Mihaela)
  | result (Paul)
  | PASS(Paul)
  |-
  |-
  | about:support shows GPU Accelerated Windows = 1/1 Direct3D 10
  | about:support shows GPU Accelerated Windows = 1/1 Direct3D 10
  | result (Mihaela)
  | PASS(Mihaela)
  | result (Paul)
  | PASS(Paul)
  | result (Mihaela)
  | PASS (Mihaela)
  | result (Paul)
  | PASS(Paul)
  |-
  |-
  | ClearType ON displays anti-aliased text
  | ClearType ON displays anti-aliased text
  | result (Mihaela)
  | PASS (Mihaela)
  | result (Paul)
  | PASS(Paul)
  | result (Mihaela)
  | PASS (Mihaela)
  | result (Paul)
  | PASS(Paul)
  |-
  |-
  | ClearType OFF does not display anti-aliased text
  | ClearType OFF does not display anti-aliased text
  | result (Mihaela)
  | PASS (Mihaela)
  | result (Paul)
  | PASS(Paul)
  | result (Mihaela)
  | PASS (Mihaela)
  | result (Paul)
  | PASS(Paul)
  |-
  |-
  | colspan="5" | '''Test'''
  | colspan="5" | '''Test'''
Line 2,096: Line 2,094:
  |-
  |-
  | colspan="4" | '''Notes'''
  | colspan="4" | '''Notes'''
*  
* Environments:
** Win 7 32 bit, AMD Radeon HD 6450
** Win 7 64bit, Nvidia GeForce 210
** Win 8 64bit, AMD Radeon HD 6450
** Win 8 32bit, Nvidia GeForce 210
  |}
  |}


==== {{bug|813264}} Bug 809865 or Bug 810726 broke Skype click-to-call ====
==== [VERIFIED FIXED] {{bug|814554}} Firefox 17 silently stops processing permissions.sqlite ====
{| class="querytable"
{| class="querytable"
  | class="header" style="width:55% |  
  | class="header" style="width:55% |  
| class="header" style="width:15% | Windows XP
  | class="header" style="width:15% | Windows 7
  | class="header" style="width:15% | Windows 7
  | class="header" style="width:15% | Windows 8
  | class="header" style="width:15% | Mac OSX
| class="header" style="width:15% | Ubuntu
  |-
  |-
  | Skype Click-to-Call installs and functions in Firefox 17.0.1
  | Cookies permissions persist across Firefox restart
  | FAIL (Paul)
  | PASS (Manuela)
  | FAIL (Manuela)
  | PASS (Virgil)
  | FAIL (Ioana)
  | PASS (Simona)
  |-
  |-
  | colspan="4" | '''Test'''
  | colspan="4" | '''Test'''
* Install the Skype click-to-call toolbar ([http://www.skype.com/intl/en/get-skype/on-your-computer/click-to-call/windows/ download])
* Start Firefox 17.0.1 with a new profile
* Browse the web and click on phone number links
* '''''Note: installation may requiring forcing version compatibility'''''
* '''''Note: broken behaviour can be observed in Firefox 17.0'''''
|-
| colspan="4" | '''Notes'''
* Skype Click-to-Call doesn't get installed on Windows 8 32bit and on Windows XP.
* The add-on could be installed on Windows 7 32bit, but it still doesn't work.
|}
 
==== {{bug|814554}} Firefox 17 silently stops processing permissions.sqlite ====
{| class="querytable"
| class="header" style="width:55% |
| class="header" style="width:15% | Windows 7
| class="header" style="width:15% | Mac OSX
| class="header" style="width:15% | Ubuntu
|-
| Cookies permissions persist across Firefox restart
| PASS (Manuela)
| PASS (Virgil)
| PASS (Simona)
|-
| colspan="4" | '''Test'''
* Start Firefox 17.0.1 with a new profile
* Select Options > Privacy > History > Firefox will: use custom settings for history
* Select Options > Privacy > History > Firefox will: use custom settings for history
* Change ''cookies'' pref to Keep until: ask me every time
* Change ''cookies'' pref to Keep until: ask me every time
Line 2,151: Line 2,130:
  |}
  |}


==== {{bug|816197}} Need hotfix to mitigate TMP bustage in bug 813763 ====
=== Update Testing ===
{| class="querytable"
* Partial: 17.0, 16.0.2, 15.0.1
| class="header" style="width:55% |
* Complete: 16.0
| class="header" style="width:15% | Windows 7
 
| class="header" style="width:15% | Mac OSX
{|class="querytable" style="width:60%"
| class="header" style="width:15% | Ubuntu
  |-
  |-
  | Hotfix removed when TMP >0.4.0.3 installed
  | class="header" | Platform
  | N/A (Manuela)
| class="header" | betatest ([http://mozmill-ondemand.blargon7.com/#/update/detail?branch=17.0&channel=betatest&from=2012-11-29&to=2012-11-29&target=17.0.1 report])
| N/A (Virgil)
  | class="header" | release ([http://mozmill-ondemand.blargon7.com/#/update/detail?branch=17.0&channel=release&from=2012-11-30&to=2012-11-30&target=17.0.1 report])
| N/A (Simona)
  |-
  |-
  | Hotfix removed when TMP not installed
  | Windows XP
  | N/A (Manuela)
  | '''PASS''' - 2012-11-29
  | N/A (Virgil)
  | '''PASS''' - 2012-11-30
| N/A (Simona)
  |-
  |-
| Hotfix displays [https://support.mozilla.org/en-US/kb/tab-mix-plus-incompatible-firefox-17-use-dev-build this page] on first startup with TMP <=0.4.0.3 installed
  | Windows Vista
| N/A (Manuela)
  | '''PASS''' - 2012-11-29
| N/A (Virgil)
  | '''PASS''' - 2012-11-30
| N/A (Simona)
|-
| Hotfix displays [https://support.mozilla.org/en-US/kb/tab-mix-plus-incompatible-firefox-17-use-dev-build this page] on second startup with TMP <=0.4.0.3 installed
| N/A (Manuela)
| N/A (Virgil)
| N/A (Simona)
|-
| Hotfix removed on third startup with TMP <=0.4.0.3 installed
| N/A (Manuela)
| N/A (Virgil)
| N/A (Simona)
|-
| Hotfix removed if TMP <=0.4.0.3 installed but disabled
| N/A (Manuela)
| N/A (Virgil)
| N/A (Simona)
|-
| colspan="4" | '''Test'''
* Be sure to check the hotfix with Firefox 17.0 and 17.0.1
|-
| colspan="4" | '''Notes'''
* Mozilla Firefox Hotfix Add-on not available for Firefox 17.0 and Firefox 17.0.1
* Tab Mix Plus 0.4.0.3 not compatible with Firefox 17.0 and Firefox 17.0.1
|}
 
=== Update Testing ===
* Partial:
* Complete:
 
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | betatest ([ report])
| class="header" | release ([ report])
|-
| Windows XP
| '''result''' - YYYY-MM-DD
| '''result''' - YYYY-MM-DD
|-
  | Windows Vista
  | '''result''' - YYYY-MM-DD
  | '''result''' - YYYY-MM-DD
  |-
  |-
  | Windows 7 32-bit
  | Windows 7 32-bit
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-29
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-30
  |-
  |-
  | Windows 7 64-bit
  | Windows 7 64-bit
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-29
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-30
  |-
  |-
  | Windows 8 32-bit
  | Windows 8 32-bit (manual re:{{bug|803489}})
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-29
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-30
  |-
  |-
  | Windows 8 64-bit
  | Windows 8 64-bit (manual re:{{bug|803489}})
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-29
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-30
  |-
  |-
  | Mac OS X 10.6
  | Mac OS X 10.6
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-29
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-30
  |-
  |-
  | Mac OS X 10.7
  | Mac OS X 10.7
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-29
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-30
  |-
  |-
  | Mac OS X 10.8
  | Mac OS X 10.8
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-29
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-30
  |-
  |-
  | Ubuntu 12.04 32-bit
  | Ubuntu 12.04 32-bit
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-29
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-30
  |-
  |-
  | Ubuntu 12.04 64-bit
  | Ubuntu 12.04 64-bit
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-29
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-11-30
  |}
  |}
; Spotchecks
* {{done|}} Verify background updates still work (chunk size and ping interval changed for 17.0.1)


; Issues
; Issues
* Very intermittent issue with 15.0.1->17.0.1 fallback updates resulting in connection error to update server


== Bugs ==
== Bugs ==
Ashughes
Confirmed users
14,525

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/490778...491289"