MozillaWiki

Releases/Firefox 10 ESR/Test Plan: Difference between revisions

Page Discussion

Releases/Firefox 10 ESR/Test Plan (view source)

Revision as of 18:42, 8 January 2013

7,856 bytes added ,  8 January 2013
→‎Update Testing
 
(37 intermediate revisions by 5 users not shown)
Line 2,558: Line 2,558:
  | class="header" | Platform
  | class="header" | Platform
  | class="header" | betatest ([http://mozmill-ondemand.blargon7.com/#/update/detail?branch=10.0&channel=esrtest&from=2012-11-16&to=2012-11-17&target=10.0.11 report])
  | class="header" | betatest ([http://mozmill-ondemand.blargon7.com/#/update/detail?branch=10.0&channel=esrtest&from=2012-11-16&to=2012-11-17&target=10.0.11 report])
| class="header" | releasetest ([report])
  | class="header" | esr ([report])
  | class="header" | esr ([report])
  |-
  |-
  | Windows 2000
  | Windows 2000
  | '''PASS''' - 2012-11-16
  | '''PASS''' - 2012-11-16
| '''result''' - YYYY-MM-DD
  | '''result''' - YYYY-MM-DD
  | '''result''' - YYYY-MM-DD
  |-
  |-
  | Windows XP
  | Windows XP
  | '''PASS''' - 2012-11-16
  | '''PASS''' - 2012-11-16
| '''result''' - YYYY-MM-DD
  | '''result''' - YYYY-MM-DD
  | '''result''' - YYYY-MM-DD
  |-
  |-
  | Windows Vista
  | Windows Vista
  | '''PASS''' - 2012-11-16
  | '''PASS''' - 2012-11-16
| '''result''' - YYYY-MM-DD
  | '''result''' - YYYY-MM-DD
  | '''result''' - YYYY-MM-DD
  |-
  |-
  | Windows 7 32-bit
  | Windows 7 32-bit
  | '''PASS''' - 2012-11-16
  | '''PASS''' - 2012-11-16
| '''result''' - YYYY-MM-DD
  | '''result''' - YYYY-MM-DD
  | '''result''' - YYYY-MM-DD
  |-
  |-
  | Windows 7 64-bit
  | Windows 7 64-bit
  | '''PASS''' - 2012-11-16
  | '''PASS''' - 2012-11-16
| '''result''' - YYYY-MM-DD
  | '''result''' - YYYY-MM-DD
  | '''result''' - YYYY-MM-DD
  |-
  |-
  | Mac OS X 10.6
  | Mac OS X 10.6
  | '''PASS''' - 2012-11-16
  | '''PASS''' - 2012-11-16
| '''result''' - YYYY-MM-DD
  | '''result''' - YYYY-MM-DD
  | '''result''' - YYYY-MM-DD
  |-
  |-
  | Mac OS X 10.7
  | Mac OS X 10.7
  | '''PASS''' - 2012-11-16
  | '''PASS''' - 2012-11-16
| '''result''' - YYYY-MM-DD
  | '''result''' - YYYY-MM-DD
  | '''result''' - YYYY-MM-DD
  |-
  |-
  | Ubuntu 12.04 32-bit
  | Ubuntu 12.04 32-bit
  | '''PASS''' - 2012-11-16
  | '''PASS''' - 2012-11-16
| '''result''' - YYYY-MM-DD
  | '''result''' - YYYY-MM-DD
  | '''result''' - YYYY-MM-DD
  |-
  |-
  | Ubuntu 12.04 64-bit
  | Ubuntu 12.04 64-bit
  | '''PASS''' - 2012-11-16
  | '''PASS''' - 2012-11-16
| '''result''' - YYYY-MM-DD
  | '''result''' - YYYY-MM-DD
  | '''result''' - YYYY-MM-DD
  |}
  |}
; ESR Manual Spotchecks due to Infra issues
* '''PASS''' Windows 2000: 10.0.10 en-US & de, 10.0.9 en-US & es-ES
* '''PASS''' Windows 7: 10.0.10 en-US & fr, 10.0.9 en-US & it
* '''PASS''' Ubuntu 12.04: 10.0.10 en-US & zh-TW, 10.0.9 en-US & ro
* '''PASS''' Mac OSX 10.7: 10.0.10 en-US & ja-JP-mac, 10.0.9 en-US & de


; Issues
; Issues
* {{bug|813699}} ondemand_update scripts taking a long time to execute for Firefox 17 updates on release channel


=== New Issues ===
=== New Issues ===
Line 2,625: Line 2,622:
}
}
</bugzilla>
</bugzilla>
== Firefox 10.0.12 ESR ==
{| class="querytable" style="width:30%"
| style="width:10%" |
| style="width:20%" |
|-
| '''Builds'''
| [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/10.0.12esr-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/10.0.12esr-candidates/build1/ HTTP]
|-
| '''ID'''
| 20130103094221
|-
| '''Changeset'''
| [https://hg.mozilla.org/releases/mozilla-esr10/rev/9e1102e58ac8 9e1102e58ac8] ([http://hg.mozilla.org/releases/mozilla-esr10/pushloghtml?fromchange=b0b7f731ed02&tochange=9e1102e58ac8 pushlog])
|}
; Checklist
* {{done|}} Functional automation
* {{done|}} Manual tests
* {{done|}} Bug fix verifications
* {{done|}} Update automation on esrtest channel
* {{done|}} Update automation on releasetest channel
* {{|}} Updates automation on esr channel
=== Automated Tests ===
The following are the detailed results of the automated Mozmill test runs.
{| class="querytable" style="width:80%"
|-
| class="header" style="width:20%" | Platform
| class="header" style="width:30%" | Functional Results
| class="header" style="width:30%" | Restart Results
|-
| Windows 2000
| PASS: [http://mozmill-ci.blargon7.com/#/functional/report/23d8fbdd0190d4b0496d6b129f52c502 0 fails]
| PASS: [http://mozmill-ci.blargon7.com/#/functional/report/23d8fbdd0190d4b0496d6b129f531b67 0 fails]
|-
| Windows XP
| PASS: [http://mozmill-ci.blargon7.com/#/functional/report/23d8fbdd0190d4b0496d6b129f52fdbe 0 fails]
| PASS: [http://mozmill-ci.blargon7.com/#/functional/report/23d8fbdd0190d4b0496d6b129f53428f 0 fails]
|-
| Windows Vista
| PASS: [http://mozmill-ci.blargon7.com/#/functional/report/23d8fbdd0190d4b0496d6b129f52adad 0 fails]
| PASS: [http://mozmill-ci.blargon7.com/#/functional/report/23d8fbdd0190d4b0496d6b129f53242a 0 fails]
|-
| Windows 7 32-bit
| PASS: [http://mozmill-ci.blargon7.com/#/functional/report/23d8fbdd0190d4b0496d6b129f52a0af 0 fails]
| PASS: [http://mozmill-ci.blargon7.com/#/functional/report/23d8fbdd0190d4b0496d6b129f52e01b 0 fails]
|-
| Windows 7 64-bit
| PASS: [http://mozmill-ci.blargon7.com/#/functional/report/23d8fbdd0190d4b0496d6b129f528f40 0 fails]
| PASS: [http://mozmill-ci.blargon7.com/#/functional/report/23d8fbdd0190d4b0496d6b129f52d652 0 fails]
|-
| Mac OS X 10.6
| PASS: [http://mozmill-ci.blargon7.com/#/functional/report/23d8fbdd0190d4b0496d6b129f5209d3 0 fails]
| PASS: [http://mozmill-ci.blargon7.com/#/functional/report/23d8fbdd0190d4b0496d6b129f5272c5 0 fails]
|-
| Mac OS X 10.7
| PASS: [http://mozmill-ci.blargon7.com/#/functional/report/23d8fbdd0190d4b0496d6b129f5218ea 0 fails]
| PASS: [http://mozmill-ci.blargon7.com/#/functional/report/23d8fbdd0190d4b0496d6b129f527a41 0 fails]
|-
| Ubuntu 12.04 32-bit
| PASS: [http://mozmill-ci.blargon7.com/#/functional/report/23d8fbdd0190d4b0496d6b129f5036ec 0 fails]
| PASS: [http://mozmill-ci.blargon7.com/#/functional/report/23d8fbdd0190d4b0496d6b129f5063aa 0 fails]
|-
| Ubuntu 12.04 64-bit
| PASS: [http://mozmill-ci.blargon7.com/#/functional/report/23d8fbdd0190d4b0496d6b129f503030 1 known fail]
| PASS: [http://mozmill-ci.blargon7.com/#/functional/report/23d8fbdd0190d4b0496d6b129f5051ad 0 fails]
|}
; Test Failures ([https://bugzilla.mozilla.org/buglist.cgi?list_id=3007328;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=mozmill-test-failure;bug_status=NEW;bug_status=ASSIGNED;component=Mozmill%20Tests;product=Mozilla%20QA known bugs])
* /testSecurity/testGreyLarry.js::testLarryGrey -> ''Controller.assertJSProperty(ID: page-proxy-favicon)'' ({{bug|785041}})
=== Manual Smoketests ===
Run the '''[https://moztrap.mozilla.org/runtests/environment/590/ Firefox ESR 10.0.12 Smoketests]''' in MozTrap and report the results below.
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | Assignee
| class="header" | Result: [https://moztrap.mozilla.org/results/cases/?filter-run=590 report]
|-
| Windows 2000
| Simona
| '''PASS''' - 2013-01-04
|-
| Windows 7 64-bit
| Ioana
| '''PASS''' - 2013-01-04
|-
| Mac OS X 10.7
| Virgil
| '''PASS''' - 2013-01-04
|-
| Ubuntu 12.04 32-bit
| Ioana
| '''PASS''' - 2013-01-04
|}
=== Bug Fix Verifications ===
* {{done|}} {{bug|795355}} & {{bug|757197}} Batch of root CA changes (Manuela) - [https://etherpad.mozilla.org/Root-CA-Changes-Exploratory exploratory etherpad]
* {{done|}} {{bug|802026}} URL spoofing with credentials info of URL & 204 (ashughes)
* {{done|}} {{bug|811382}} Update OTS to r95 ie. [https://www.google.com/search?&q=web+fonts Web Fonts] (Simona) - [https://etherpad.mozilla.org/Web-Fonts-Exploratory exploratory etherpad]
* {{done|}} {{bug|825022}} Deal with TURKTRUST mis-issued *.google.com certificate (mwobensmith)
=== Update Testing ===
* Partial: 10.0.11esr
* Complete: 10.0esr - 10.0.10esr
{|class="querytable" style="width:60%"
|-
| class="header" | Platform
| class="header" | [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=10.0&channel=esrtest&from=2013-01-03&to=2013-01-04&target=10.0.12 esrtest] (2013-01-03)
| class="header" | [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=10.0&channel=releasetest&from=2013-01-07&to=2013-01-08&target=10.0.12 releasetest] (2012-01-07)
| class="header" | [http://mozmill-ondemand.blargon7.com/#/update/detail?branch=10.0&channel=esr&from=2013-01-08&to=2013-01-09&target=10.0.12 esr] (2012-01-08)
|-
| Windows 2000
| PASS: 0 fails
| PASS: 0 fails
| PASS: 0 fails
|-
| Windows XP
| PASS: 0 fails
| PASS: 0 fails
| PASS: 0 fails
|-
| Windows Vista
| PASS: 0 fails
| PASS: 0 fails
| PASS: 0 fails
|-
| Windows 7 32-bit
| PASS: 0 fails
| PASS: 0 fails
| PASS: 0 fails
|-
| Windows 7 64-bit
| PASS: 0 fails
| PASS: 0 fails
| PASS: 0 fails
|-
| Mac OS X 10.6
| PASS: 0 fails
| PASS: 0 fails
| PASS: 0 fails
|-
| Mac OS X 10.7
| PASS: 0 fails
| PASS: 0 fails
| PASS: 0 fails
|-
| Ubuntu 12.04 32-bit
| PASS: 0 fails
| PASS: 0 fails
| PASS: 0 fails
|-
| Ubuntu 12.04 64-bit
| PASS: 0 fails
| PASS: 0 fails
| PASS: 0 fails
|}
=== New Issues ===
Report any new issues found during testing below. Any items marked in '''bold''' are considered blocking the release.
=== Known Issues ===
Crashers
* {{bug|786742}} Firefox EXCEPTION_INVALID_HANDLE crash in js::GCHelperThread::doSweep (Yandex)
Geolocation
* {{bug|799075}} Failure to determine geolocation on ESR builds on every other try
HTML5 Video
* {{bug|520600}} Video plays automatically if paused when switching to fullscreen mode
* {{bug|708557}} Html5 youtube context menu not persistent when right clicking
* FIXED <strike>{{bug|520160}} No context menu when watching a video in full screen mode</strike>
* {{bug|803057}} HTML5 videos continues to play after Pop Out
Plugins
* {{bug|78414}} Keyboard shortcuts fail if plug-in has focus
* {{bug|667399}} RealPlayer G2 LiveConnect plugin crash
* {{bug|747073}} Reader 10 doesn't work in 64-bit mode
* {{bug|775082}} Flash video controls disappear after dragging the tabs
* {{bug|794924}} Choppy fading out for Quicktime videos
* {{bug|812479}} QuickTime back/forward and mute buttons do not work
* WONTFIX <strike>{{bug|692446}} Firefox prompts to install WMP plugin even though it's installed</strike>
Printing
* {{bug|717178}} Printing stops after 73%, and browser hangs up
Session Restore
* {{bug|791195}} Session is not restored after Firefox crash
Tech Evangelism
* {{bug|747909}} In Full Screen mode - copy actions are not allowed on HTML5 videos
* WONTFIX <strike>{{bug|694288}} Vimeo embedded video player is not displayed on Windows 2000</strike>
* WONTFIX <strike>{{bug|779492}} Youtube Flash pop out video player is opened in the background</strike>
Ashughes
Confirmed users
14,525

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/488678...497495"