Releases/Firefox 14/Test Plan: Difference between revisions

Jump to navigation Jump to search

Releases/Firefox 14/Test Plan (view source)

Revision as of 22:20, 10 July 2012

8,579 bytes added ,  10 July 2012
no edit summary
No edit summary
Line 2,156: Line 2,156:
  | 8
  | 8
  | 79%
  | 79%
|-
| '''[http://bit.ly/LjZyWi QA+ SG]''': for the [[Desktop_Firefox#Security_Triage_Team | Security team]] to verify security fixes
|
|
|
|-
| '''[http://bit.ly/NCG2UT status-firefox14:fixed]''': for QA lead to triage for potential QA+ bugs
|
|
|
|}
''[https://docs.google.com/spreadsheet/ccc?key=0AjuYQoXoeC5KdENxZDNoRlYwZENWOXFDN01DUVpfN0E#gid=0 Softvision verifications etherpad]''
=== QAWANTED ===
[http://bit.ly/L7NnJw QAWANTED Bugs]
* Please look through the list of QAWANTED bugs and make sure there's nothing else we can do to advance the bug.
=== Issues Found ===
<bugzilla>
{
"id": ""
}
</bugzilla>
=== Known Issues ===
<bugzilla>
{
"id": ""
}
</bugzilla>
== Beta 12 ==
{| class="querytable" style="width:30%"
| style="width:10%" |
| style="width:20%" |
|-
| '''Builds'''
| [ftp://ftp.mozilla.org/pub/firefox/nightly/14.0b12-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/firefox/nightly/14.0b12-candidates/build1/ HTTP]
|-
| '''ID'''
| 20120710123126
|-
| '''Changeset'''
| [http://icanhascheezburger.com TBD]
|}
; Checklist
* {{ok}|}} Functional automated tests, synced to Litmus
* {{ok|}} Manual tests
* {{ok|}} Bug fix verifications (target 50%)
* {{ok|}} Security fix verifications (target 25%)
* {{ok|}} Update tests on releasetest
* {{ok|}} Get final sign-off from Bob Moss
* {{ok|}} Update tests on beta
=== Automated Tests ===
The following are the detailed results of the automated BFT test runs.
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | Result: [http://mozmill-ci.blargon7.com/#/functional/reports?branch=14.0&platform=All&from=2012-07-10&to=2012-07-10 report]
|-
| Windows XP
| '''RESULT''' - 2012-07-10
|-
| Windows Vista
| '''RESULT''' - 2012-07-10
|-
| Windows 7 32-bit
| '''RESULT''' - 2012-07-10
|-
| Windows 7 64-bit
| '''RESULT''' - 2012-07-10
|-
| Mac OS X 10.6
| '''RESULT''' - 2012-07-10
|-
| Mac OS X 10.7
| '''RESULT''' - 2012-07-10
|-
| Ubuntu 11.10 32-bit
| '''RESULT''' - 2012-07-10
|-
| Ubuntu 11.10 64-bit
| '''RESULT''' - 2012-07-10
|}
; Test Failures ([https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA known bugs])
=== Manual Tests ===
The following are the detailed results of the manual testing.
==== Smoketests ====
Run the '''Firefox 14.0 Smoketests''' in [https://litmus.mozilla.org/run_tests.cgi?test_run_id=480 Litmus] and report results below.
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | Assignee
| class="header" | Result
|-
| Windows XP
| Simona
| '''RESULT''' - 2012-07-10
|-
| Windows Vista
| tester
| '''RESULT''' - 2012-07-10
|-
| Windows 7 32-bit
| tester
| '''RESULT''' - 2012-07-10
|-
| Windows 7 64-bit
| tester
| '''RESULT''' - 2012-07-10
|-
| Mac OS X 10.6.8
| tester
| '''RESULT''' - 2012-07-10
|-
| Mac OS X 10.7.4
| tester
| '''RESULT''' - 2012-07-10
|-
| Ubuntu 12.04 32-bit
| tester
| '''RESULT''' - 2012-07-10
|-
| Ubuntu 11.10 64-bit
| tester
| '''RESULT''' - 2012-07-10
|-
| Windows 7 32-bit (es-AR)
| Gabriela
| '''RESULT''' - 2012-07-10
|}
''Results in [https://litmus.mozilla.org/test_run_report.cgi?test_run_id=480 Litmus]''
==== Regression Testing ====
Run the '''Firefox 14.0 Regression Tests''' and  '''Firefox 14.0 Web App Compatibility''' in [https://litmus.mozilla.org/run_tests.cgi?test_run_id=480 Litmus] and report the results below.
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | Assignee
| class="header" | Result: [https://litmus.mozilla.org/test_run_report.cgi?test_run_id=480 report]
|-
| Windows 7 32-bit
| tester
| '''RESULT''' - 2012-07-10
|-
| Mac OS X 10.6.8
| tester
| '''RESULT''' - 2012-07-10
|-
| Ubuntu 11.10 64-bit
| tester
| '''RESULT''' - 2012-07-10
|-
| Windows XP
| tester
| '''RESULT''' - 2012-07-10
|}
==== Exploratory Testing ====
# {{done|}}Exploratory testing on plugins, addons, bookmarks etc [https://mozqa.etherpad.mozilla.org/Firefox14-exploratory Etherpad]
# {{done|}}Exploratory testing on the new features [https://mozqa.etherpad.mozilla.org/Firefox14-features Etherpad]
==== Feature Testing ====
Run the  '''Firefox 14.0 Features Sign-off''' tests in [https://litmus.mozilla.org/run_tests.cgi?test_run_id=480 Litmus] and report the results below. Additionally do some exploratory testing to make sure no regressions have been introduced to related legacy features.
{| class="querytable"
| class="header" | Feature
| class="header" | Owner
| class="header" | Win32
| class="header" | Mac
| class="header" | Linux
|-
|[[Firefox/Features/URL Autocomplete/Test Plan|URL Autocomplete]]
| Vlad Ghetiu
| '''RESULT''' - 2012-07-10
| '''RESULT''' - 2012-07-10
| '''RESULT''' - 2012-07-10
|-
|[[Opt-in activation for plugins/Test Plan|Opt-in activation for plugins]]
| Paul Silaghi
| '''RESULT''' - 2012-07-10
| '''RESULT''' - 2012-07-10
| '''RESULT''' - 2012-07-10
|-
|[[QA/Firefox migrators |Firefox migrators - Safari Migrator]]
| Paul Silaghi
| '''RESULT''' - 2012-07-10
| '''RESULT''' - 2012-07-10
| '''N/A'''
|-
|[[DevTools/Features/PseudoClassLock/Test Plan |Pseudo-Class Lock ]]
| Ioana Budnar
| '''RESULT''' - 2012-07-10
| '''RESULT''' - 2012-07-10
| '''RESULT''' - 2012-07-10
|-
|[[Platform/Features/MouseCapture/Test Plan |Support for Mouse Capture / Mouse Lock ]]
| Virgil Dicu
| '''RESULT''' - 2012-07-10
| '''RESULT''' - 2012-07-10
| '''RESULT''' - 2012-07-10
|-
|[[Releases/Firefox_14/Test_Plan#Mac_10.8_Signing | Mac OS 10.8 Signing]]
| Required
| n/a
| '''RESULT''' - 2012-07-10
| n/a
|-
|}
===== Mac 10.8 Signing =====
NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test ([https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html more info]).
* Download the .dmg file and drag the Firefox.app file to a location on your computer
* Open a terminal window and cd to the folder Firefox.app is located
* Run ''codesign -vv Firefox.app'' in terminal, output should contain ''valid on disk'' and ''satisfies its Designated Requirement''
* Run ''xattr Firefox.app'' in terminal, output should contain ''com.apple.quarantine''
* Double click the Firefox.app icon, you should see a warning about the program being provided by an ''unknown developer'' or having been ''downloaded from the Internet''
* Click "open" and browse to any website, Firefox should start and you should be able to browse
* Quit Firefox and double click the Firefox.app icon again, you should ''not'' be reprompted for the warning in step 5, firefox should start up and browse without any issues
Spot check build after this test.
=== Update Tests ===
The following are details of update verifications
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | Releasetest: [http://mozmill-ondemand.blargon7.com/#/update/overview?branch=14.0&channel=releasetest&from=2012-07-10&to=2012-07-11 report]
| class="header" | Beta: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=14.0&channel=beta&from=2012-07-10&to=2012-07-11&target=14.0 report]
|-
| Windows XP
| '''RESULT''' - 2012-07-10
| '''RESULT''' - 2012-07-10
|-
| Windows Vista
| '''RESULT''' - 2012-07-10
| '''RESULT''' - 2012-07-10
|-
| Windows 7 32-bit
| '''RESULT''' - 2012-07-10
| '''RESULT''' - 2012-07-10
|-
| Windows 7 64-bit
| '''RESULT''' - 2012-07-10
| '''RESULT''' - 2012-07-10
|-
| Mac OS X 10.6
| '''RESULT''' - 2012-07-10
| '''RESULT''' - 2012-07-10
|-
| Mac OS 10.7
| '''RESULT''' - 2012-07-10
| '''RESULT''' - 2012-07-10
|-
| Ubuntu 11.10 32-bit
| '''RESULT''' - 2012-07-10
| '''RESULT''' - 2012-07-10
|-
| Ubuntu 11.10 64-bit
| '''RESULT''' - 2012-07-10
| '''RESULT''' - 2012-07-10
|}
; Issues
=== Bug Fix Verifications ===
The following are the detailed results of verifications of bug fixes landed in Beta.
* Any fix you very should have the status-firefox14 flag set to ''verified''
* If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
{| class="querytable"
| class="header" | Query
| class="header" | Triaged
| class="header" | Remaining
| class="header" | % Complete
|-
| '''[http://bit.ly/KbYn7h QA+]''': for testers to verify functional fixes
|
|
|
  |-
  |-
  | '''[http://bit.ly/LjZyWi QA+ SG]''': for the [[Desktop_Firefox#Security_Triage_Team | Security team]] to verify security fixes
  | '''[http://bit.ly/LjZyWi QA+ SG]''': for the [[Desktop_Firefox#Security_Triage_Team | Security team]] to verify security fixes
Jbecerra
canmove, Confirmed users
7,810

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/449380"

Navigation menu