MozillaWiki

Releases/Firefox 15/Test Plan: Difference between revisions

Page Discussion

Releases/Firefox 15/Test Plan (view source)

Revision as of 21:35, 11 September 2012

1,972 bytes added ,  11 September 2012
→‎Hotfix
 
(47 intermediate revisions by 5 users not shown)
Line 2,147: Line 2,147:
  |-
  |-
  | '''Builds'''
  | '''Builds'''
  | [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/15.0-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/15.0-candidates/build1/ HTTP]
  | [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/15.0.1-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/15.0.1-candidates/build1/ HTTP]
  |-
  |-
  | '''ID'''
  | '''ID'''
  | 20120824154833
  | 20120905151427
  |-
  |-
  | '''Changeset'''
  | '''Changeset'''
  | [http://hg.mozilla.org/releases/mozilla-release/rev/450143d2d810 450143d2d810] ([http://hg.mozilla.org/releases/mozilla-release/pushloghtml?fromchange=450143d2d810&tochange=e5728a4e106c pushlog])
  | [https://hg.mozilla.org/releases/mozilla-release/rev/0b774a1067fe 0b774a1067fe] ([http://hg.mozilla.org/releases/mozilla-release/pushloghtml?fromchange=e5728a4e106c&tochange=0b774a1067fe pushlog])
  |}
  |}


; Checklist
; Checklist
* {{ |}} [[Releases/Firefox_15/Test_Plan#Automated_Tests_8|Automated tests]]
* {{done|}} [[Releases/Firefox_15/Test_Plan#Automated_Tests_9|Automated tests]]
* {{ |}} [[Releases/Firefox_15/Test_Plan#Manual_Testing_7|Manual Testing]]
* {{done|}} [[Releases/Firefox_15/Test_Plan#Manual_Testing|Manual Testing]]
* {{ |}} [[Releases/Firefox_15/Test_Plan#Update_Testing_7|Update Testing]]
* {{done|}} [[Releases/Firefox_15/Test_Plan#Update_Testing_8|Update Testing]]
* {{ |}} Get final sign-off from peers
* {{done|}} Final sign-off sent and updates qualified on Release channel
* {{ |}} Final sign-off sent and updates qualified on Release channel


=== Automated Tests ===
=== Automated Tests ===
Line 2,168: Line 2,167:
  |-
  |-
  | class="header" style="width:20%" | Platform
  | class="header" style="width:20%" | Platform
  | class="header" style="width:30%" | Results: [ report]
  | class="header" style="width:30%" | Results: [http://mozmill-ci.blargon7.com/#/functional/reports?branch=15.0&platform=All&from=2012-09-05&to=2012-09-06 report]
  |-
  |-
  | Windows XP
  | Windows XP
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Windows Vista
  | Windows Vista
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Windows 7 32-bit
  | Windows 7 32-bit
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Windows 7 64-bit
  | Windows 7 64-bit
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Mac OS X 10.6
  | Mac OS X 10.6
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Mac OS X 10.7
  | Mac OS X 10.7
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Mac OS X 10.8
  | Mac OS X 10.8
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Ubuntu 12.04 32-bit
  | Ubuntu 12.04 32-bit
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Ubuntu 12.04 64-bit
  | Ubuntu 12.04 64-bit
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |}
  |}


Line 2,214: Line 2,213:
# Stop Private Browsing mode
# Stop Private Browsing mode
# Open about:cache and verify the ''Disk Cache'' and ''Memory Cache'' do not contain any entries referring to the sites you visited in Private Browsing mode
# Open about:cache and verify the ''Disk Cache'' and ''Memory Cache'' do not contain any entries referring to the sites you visited in Private Browsing mode
# Grep your profile to make sure no references to the website(s) you visited in Private Browsing mode are discoverable in the profile on your filesystem
* cd ${PROFILE_DIRECTORY}
* grep -r "sitename" .


Other things to watch for:
Other things to watch for:
Line 2,224: Line 2,226:
* Private Browsing data does not leak across multiple subsequent Private Browsing sessions after exiting Private Browsing mode
* Private Browsing data does not leak across multiple subsequent Private Browsing sessions after exiting Private Browsing mode
* Private Browsing data does not leak into Session Restore from restart, crash, about:sessionstore, software update, add-on install restart, system hard reset, system restore from sleep/suspend/hibernate, etc
* Private Browsing data does not leak into Session Restore from restart, crash, about:sessionstore, software update, add-on install restart, system hard reset, system restore from sleep/suspend/hibernate, etc
;Exploratory testing
*Private Browsing - [https://mozqa.etherpad.mozilla.org/private-browsing etherpad]
*Web compatibility, plugins, addons etc - [https://mozqa.etherpad.mozilla.org/Fx15-0-1-exploratory etherpad]


; Results
; Results
Line 2,233: Line 2,239:
  |-
  |-
  | Windows XP
  | Windows XP
  |
  | Ioana
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-05
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Windows Vista
  | Windows Vista
  |
  | VladG
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-05
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Windows 7 32-bit
  | Windows 7 32-bit
  |
  | Mihaela
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-05
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Windows 7 64-bit
  | Windows 7 64-bit
  |
  | Mihaela
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-05
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Mac OSX 10.6
  | Mac OS X 10.6
  |
  | VladG
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-05
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Mac OSX 10.7
  | Mac OS X 10.7
  |
  | VladG
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-05
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Mac OSX 10.8
  | Mac OS X 10.8
  |
  | Virgil
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-05
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Ubuntu 12.04 32-bit
  | Ubuntu 12.04 32-bit
  |
  | Virgil
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-05
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Ubuntu 12.04 64-bit
  | Ubuntu 12.04 64-bit
  |
  | Ioana
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-05
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |}
  |}


Line 2,281: Line 2,287:


=== Update Testing ===
=== Update Testing ===
* Partial: 12.0, 13.0.1, 14.0.1
* Partial: 13.0.1, 14.0.1, 15.0
* Complete: 13.0, 11.0, 10.0, 10.0.1, 10.0.2
* Complete: 12.0, 13.0


{|class="querytable" style="width:60%"
{|class="querytable" style="width:60%"
  |-
  |-
  | class="header" | Platform
  | class="header" | Platform
  | class="header" | Releasetest: [ report]
  | class="header" | Releasetest: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=15.0&channel=releasetest&from=2012-09-06&to=2012-09-07&target=15.0.1 report]
  | class="header" | Release: [ report]
  | class="header" | Release: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=15.0&channel=release&from=2012-09-06&to=2012-09-07&target=15.0.1 report]
  |-
  |-
  | Windows XP
  | Windows XP
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Windows Vista
  | Windows Vista
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Windows 7 32-bit
  | Windows 7 32-bit
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Windows 7 64-bit
  | Windows 7 64-bit
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Mac OS X 10.6
  | Mac OS X 10.6
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Mac OS X 10.7
  | Mac OS X 10.7
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Mac OS X 10.8
  | Mac OS X 10.8
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Ubuntu 11.10 32-bit
  | Ubuntu 12.04 32-bit
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |-
  |-
  | Ubuntu 11.10 64-bit
  | Ubuntu 12.04 64-bit
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  | '''result''' - YYYY-MM-DD
  | '''PASS''' - 2012-09-06
  |}
  |}


; Issues
; Issues
== Hotfix ==
Hotfix to turn off Background Updates (see {{bug|790096}}).
=== Test Case ===
Hotfix installs for:
* Firefox 15 - 18 users on...
* Mac & Linux with...
* add-ons installed in the app-bundle (how do you do this????) -- ex. Comcast software
Hotfixed is active if:
* Hotfix add-on appears in Addons Manager
* Background Update pref is turned off
Updates to check:
* Updates not found via background update
* Updates found via manual check for updates
Questions:
* How do you install add-ons in the app-bundle? What is an app-bundle? and are there specific add-ons?
* Hotfix installer dialog?
* Background Update pref?
* Expectation of pave-over install with "affected" version?
* Expectation of pave-over install with "unaffected" version?
Expectations:
* 15.0.1 release: hotfix applies, won't be able to test updates since none available
* 15.0 release: hotfix applies, no background updates to 15.0.1
* 15.0b6 beta: hotfix applies, no background updates to 16.0b3
* 15.0a2 aurora: hotfix applies, no background updates to 17.0a2, hotfix removed after update
* 16.0b2 beta: hotfix applies, no background updates to 16.0b3
* 17.0a2 aurora: hotfix applies for those with app-bundle add-ons
* 18.0a1 nightly: hotfix applies for those with app-bundle add-ons


== Beta Feature Sign-off ==
== Beta Feature Sign-off ==
Ashughes
Confirmed users
14,525

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/467699...469590"