Releases/Firefox 14/Test Plan: Difference between revisions

Jump to navigation Jump to search

Releases/Firefox 14/Test Plan (view source)

Revision as of 06:21, 16 July 2012

7,171 bytes added ,  16 July 2012
no edit summary
No edit summary
Line 2,430: Line 2,430:
{
{
  "id": "770207, 711545, 675645, 763237, 763237"
  "id": "770207, 711545, 675645, 763237, 763237"
}
</bugzilla>
== Release Candidate ==
{| class="querytable" style="width:30%"
| style="width:10%" |
| style="width:20%" |
|-
| '''Builds'''
| [ftp://ftp.mozilla.org/pub/firefox/nightly/14.0.1-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/firefox/nightly/14.0.1-candidates/build1/ HTTP]
|-
| '''ID'''
| 20120713134347
|-
| '''Changeset'''
| [http://hg.mozilla.org/releases/mozilla-release/rev/e5728a4e106c e5728a4e106c]
|}
; Checklist
* {{ok|}} Functional automated tests
* {{ok|}} Manual tests
* {{ok|}} Bug fix verifications
* {{ok|}} Security fix verifications
* {{ok|}} Update tests on releasetest
* {{ok|}} Get final sign-off
* {{ok|}} Update tests on release
=== Automated Tests ===
The following are the detailed results of the automated BFT test runs.
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | Result: [http://mozmill-ci.blargon7.com/#/functional/reports?branch=14.0&platform=All&from=2012-07-13&to=2012-07-14 report]
|-
| Windows XP
| '''RESULT''' - 2012-07-15
|-
| Windows Vista
| '''RESULT''' - 2012-07-15
|-
| Windows 7 32-bit
| '''RESULT''' - 2012-07-15
|-
| Windows 7 64-bit
| '''RESULT''' - 2012-07-15
|-
| Mac OS X 10.6
| '''RESULT''' - 2012-07-15
|-
| Mac OS X 10.7
| '''RESULT''' - 2012-07-15
|-
| Ubuntu 11.10 32-bit
| '''RESULT''' - 2012-07-15
|-
| Ubuntu 11.10 64-bit
| '''RESULT''' - 2012-07-15
|}
; Test Failures ([https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA known bugs])
=== Manual Tests ===
The following are the detailed results of the manual testing.
==== Smoketests ====
Run the '''Firefox 14.0 Smoketests''' in [https://litmus.mozilla.org/run_tests.cgi?test_run_id=482 Litmus] and report results below.
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | Assignee
| class="header" | Result
|-
| Windows XP
| Simona
| '''RESULT''' - 2012-07-15
|-
| Windows Vista
| tester
| '''RESULT''' - 2012-07-15
|-
| Windows 7 32-bit
| tester
| '''RESULT''' - 2012-07-15
|-
| Windows 7 64-bit
| tester
| '''RESULT''' - 2012-07-15
|-
| Mac OS X 10.6.8
| tester
| '''RESULT''' - 2012-07-15
|-
| Mac OS X 10.7.4
| tester
| '''RESULT''' - 2012-07-15
|-
| Ubuntu 12.04 32-bit
| tester
| '''RESULT''' - 2012-07-15
|-
| Ubuntu 11.10 64-bit
| tester
| '''RESULT''' - 2012-07-15
|-
| Windows 7 32-bit (es-AR)
| Gabriela
| '''RESULT''' - 2012-07-15
|}
''Results in [https://litmus.mozilla.org/test_run_report.cgi?test_run_id=482 Litmus]''
==== Regression Testing ====
Run the '''Firefox 14.0 Regression Tests''' and  '''Firefox 14.0 Web App Compatibility''' in [https://litmus.mozilla.org/run_tests.cgi?test_run_id=482 Litmus] and report the results below.
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | Assignee
| class="header" | Result: [https://litmus.mozilla.org/test_run_report.cgi?test_run_id=482 report]
|-
| Windows 7 32-bit
| tester
| '''RESULT''' - 2012-07-15
|-
| Mac OS X 10.6.8
| tester
| '''RESULT''' - 2012-07-15
|-
| Ubuntu 11.10 64-bit
| tester
| '''RESULT''' - 2012-07-15
|-
| Windows XP
| tester
| '''RESULT''' - 2012-07-15
|}
==== Exploratory Testing ====
# {{ok|}}Exploratory testing on plugins, addons, bookmarks etc [https://mozqa.etherpad.mozilla.org/Firefox14-exploratory Etherpad]
# {{ok|}}Exploratory testing on the new features [https://mozqa.etherpad.mozilla.org/Firefox14-features Etherpad]
==== Feature Testing ====
By now we should have all features signed off.
===== Mac 10.8 Signing =====
NOTE: Test needs to be run from a clean Mac OSX 10.8 install. Please keep a vanilla image handy so you can restore once completing this test ([https://people.mozilla.com/~stmichaud/Gatekeeper/MLGatekeeper.html more info]).
* Download the .dmg file and drag the Firefox.app file to a location on your computer
* Open a terminal window and cd to the folder Firefox.app is located
* Run ''codesign -vv Firefox.app'' in terminal, output should contain ''valid on disk'' and ''satisfies its Designated Requirement''
* Run ''xattr Firefox.app'' in terminal, output should contain ''com.apple.quarantine''
* Double click the Firefox.app icon, you should see a warning about the program being provided by an ''unknown developer'' or having been ''downloaded from the Internet''
* Click "open" and browse to any website, Firefox should start and you should be able to browse
* Quit Firefox and double click the Firefox.app icon again, you should ''not'' be reprompted for the warning in step 5, firefox should start up and browse without any issues
Spot check build after this test.
=== Update Tests ===
The following are details of update verifications
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | Releasetest: [http://mozmill-ondemand.blargon7.com/#/update/overview?branch=14.0&channel=releasetest&from=2012-07-15&to=2012-07-11 report]
| class="header" | Beta: [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=14.0&channel=beta&from=2012-07-15&to=2012-07-11&target=14.0 report]
|-
| Windows XP
| '''RESULT''' - 2012-07-15
| '''RESULT''' - 2012-07-15
|-
| Windows Vista
| '''RESULT''' - 2012-07-15
| '''RESULT''' - 2012-07-15
|-
| Windows 7 32-bit
| '''RESULT''' - 2012-07-15
| '''RESULT''' - 2012-07-15
|-
| Windows 7 64-bit
| '''RESULT''' - 2012-07-15
| '''RESULT''' - 2012-07-15
|-
| Mac OS X 10.6
| '''RESULT''' - 2012-07-15
| '''RESULT''' - 2012-07-15
|-
| Mac OS 10.7
| '''RESULT''' - 2012-07-15
| '''RESULT''' - 2012-07-15
|-
| Ubuntu 11.10 32-bit
| '''RESULT''' - 2012-07-15
| '''RESULT''' - 2012-07-15
|-
| Ubuntu 11.10 64-bit
| '''RESULT''' - 2012-07-15
| '''RESULT''' - 2012-07-15
|}
; Issues
=== Bug Fix Verifications ===
The following are the detailed results of verifications of bug fixes landed in Beta.
* Any fix you very should have the status-firefox14 flag set to ''verified''
* If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
{| class="querytable"
| class="header" | Query
| class="header" | Triaged
| class="header" | Remaining
| class="header" | % Complete
|-
| '''[http://bit.ly/KbYn7h QA+]''': for testers to verify functional fixes
|
|
|
|-
| '''[http://bit.ly/LjZyWi QA+ SG]''': for the [[Desktop_Firefox#Security_Triage_Team | Security team]] to verify security fixes
|
|
|
|-
| '''[http://bit.ly/NCG2UT status-firefox14:fixed]''': for QA lead to triage for potential QA+ bugs
|
|
|
|}
''[https://docs.google.com/spreadsheet/ccc?key=0AjuYQoXoeC5KdENxZDNoRlYwZENWOXFDN01DUVpfN0E#gid=0 Softvision verifications etherpad]''
=== QAWANTED ===
[http://bit.ly/L7NnJw QAWANTED Bugs]
* Please look through the list of QAWANTED bugs and make sure there's nothing else we can do to advance the bug.
=== Issues Found ===
<bugzilla>
{
"id": ""
}
</bugzilla>
=== Known Issues ===
<bugzilla>
{
"id": ""
}
}
</bugzilla>
</bugzilla>
Jbecerra
canmove, Confirmed users
7,810

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/450902"

Navigation menu