Releases/Firefox 10 ESR/Test Plan

From MozillaWiki
Jump to navigation Jump to search

Firefox 10 ESR

The following is a test plan to sign-off Firefox 10 ESR for Release.

Firefox 10.0

Builds: FTP, HTTP

Checklist
  • [DONE] Functional automated tests, synced to Litmus
  • [DONE] Manual smoketests
  • [DONE] Bug fix verifications (N/A for first ESR)
  • [DONE] Update tests (N/A for first ESR)

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Assignee Result: report
Windows 2000 ashughes PASS - 2012-01-30
Windows XP ashughes PASS - 2012-01-30
Windows Vista ashughes PASS - 2012-01-30
Windows 7 32-bit ashughes PASS - 2012-01-30
Windows 7 64-bit ashughes PASS - 2012-01-30
Mac OS X 10.6 ashughes PASS - 2012-01-30
Mac OS X 10.7 ashughes PASS - 2012-01-30
Ubuntu 11.10 32-bit ashughes PASS - 2012-01-30
Ubuntu 11.10 64-bit ashughes PASS - 2012-01-30
Test Failures

Manual Tests

The following are the detailed results of the manual tests.

Platform Assignee Smoketests
Windows 2000 Simona PASS - 2012-01-31
Windows XP Ioana PASS - 2012-01-31
Windows Vista Simona PASS - 2012-01-31
Windows 7 32-bit Mihaela PASS - 2012-01-31
Windows 7 64-bit Mihaela PASS - 2012-01-31
Mac OS X 10.6 VladG PASS - 2012-01-31
Mac OS X 10.7 VladG PASS - 2012-01-31
Ubuntu 11.10 32-bit Ioana PASS - 2012-01-31
Ubuntu 11.10 64-bit Virgil PASS - 2012-01-31

Litmus: testrun, results

Exploratory Testing
  • Check top-sites like Facebook, Youtube, Gmail, Amazon, banking websites, audio/video streaming, etc
  • Check a random assortment of top add-ons to ensure compatibility
    • The tests and results are listed in the Etherpad

Known Issues

Bug ID Summary Reporter Status
bug 719380 Html5 video flickers in full screen mode VladG NEW
bug 520600 Switching into full screen video playback mode while video is paused switches into play mode automatically Henrik Skupin NEW
bug 702894 Using esc to exit full screen video from built-in player breaks player in pre-DOM API full-screen implementation Christopher Blizzard ASSIGNED
bug 585816 no thumbnail when dragging tabs Juan Becerra RESOLVED WONTFIX
bug 671535 Videos fail to play on Windows 2000 Simona B RESOLVED WONTFIX
bug 382124 Crash Reporter (= Breakpad) doesn't write minidumps on Windows 2000 (old DbgHelp.dll / MiniDumpWriteDump()) Steve England RESOLVED WONTFIX

Firefox 10.0.1

Builds: FTP, HTTP
Litmus: testrun, report

Checklist
  • [DONE] Functional automated tests, synced to Litmus
  • [DONE] Manual tests
  • [DONE] Bug fix verifications
  • [ON TRACK] Update tests on releasetest
  • [ON TRACK] Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Assignee Result: report
Windows 2000 ashughes PASS - 2012-02-08
Windows XP ashughes PASS - 2012-02-08
Windows Vista ashughes PASS - 2012-02-08
Windows 7 32-bit ashughes PASS - 2012-02-08
Windows 7 64-bit ashughes PASS - 2012-02-08
Mac OS X 10.6 ashughes PASS - 2012-02-08
Mac OS X 10.7 ashughes PASS - 2012-02-08
Ubuntu 11.10 32-bt ashughes PASS - 2012-02-08
Ubuntu 11.10 64-bit ashughes PASS - 2012-02-08
Test Failures
  • testSecurity/testSafeBrowsingWarningPages (known, bug 705182)
  • testSecurity/testSubmitUnencryptedInfoWarning (bug 725486)

Manual Tests

The following are the detailed results of the manual testing.

Smoketests

Run the manual smoketests and report results below.

Platform Assignee Result
Windows 2000 Simona PASS with known issues - 2012-02-09
Windows XP Ioana PASS - 2012-02-09
Windows Vista Simona PASS with known issues - 2012-02-09
Windows 7 32-bit Mihaela PASS with known issues - 2012-02-09
Windows 7 64-bit Paul PASS with known issues - 2012-02-09
Mac OS X 10.6.8 Mihaela PASS - 2012-02-09
Mac OS X 10.7.2 Virgil PASS - 2012-02-09
Ubuntu 11.10 32-bit Ioana PASS - 2012-02-09
Ubuntu 11.10 64-bit Virgil PASS - 2012-02-09

Results in Litmus

Update Tests

The following are details of update verifications

Platform Releasetest: report Release: report
Windows 2000 PASS - 2012-02-09 PASS - 2012-02-10
Windows XP PASS - 2012-02-09 PASS - 2012-02-10
Windows Vista PASS - 2012-02-09 PASS - 2012-02-10
Windows 7 32-bit PASS - 2012-02-09 PASS - 2012-02-10
Windos 7 64-bit PASS - 2012-02-09 PASS - 2012-02-10
Mac OS X 10.6 PASS - 2012-02-09 PASS - 2012-02-10
Mac OS 10.7 PASS - 2012-02-09 PASS - 2012-02-10
Ubuntu 11.10 32-bit PASS - 2012-02-09 PASS - 2012-02-10
Ubuntu 11.10 64-bit PASS - 2012-02-09 PASS - 2012-02-10
Issues

Bug Fix Verifications

Bug Status
bug 718939
Java applet causes text entry fields to become semi-unresponsive
TIP: See comment 23 and 25 for steps to reproduce 		VERIFIED FIXED (juanb)
bug 724284
Use after free in nsXBLDocumentInfo::ReadPrototypeBindings 		VERIFIED FIXED (marcia)

Issues Found

Bug Notes
bug 725631
Forward button disabled by visiting Youtube before other websites 		DUPE of bug 682115?
not widely reproducible
bug 725614
Keyboard shortcuts fail on pages containing Java/Shockwave/PDF 		Not a regression, reproducible back to 3.6.26
bug 725306
Vimeo doesn't enter full screen correctly via context menu 		Regressed in Firefox 7.0a1 2011-07-28
bug 725296
Browser is unresponsive while Silverlight content is loading 		Not a regression, affects Chrome
bug 709029
RealPlayer plugin crashes when trying to play a video 		Not a regression, Firefox 4.0 (OOPP?)
bug 638770
Clicking on a quicktime video pauses it 		Trivial UX issue, not worth tracking
bug 78414
Keyboard shortcuts fail when plug-in has focus 		Very old bug, no reason to start blocking
bug 692446
Firefox prompts to install windows media player plugin even though it is already installed 		MSI bug

Firefox 10.0.2 ESR

Builds: FTP, HTTP
Litmus: testrun, report
Mozmill: [ report]

Checklist
  • [DONE] Functional automated tests, synced to Litmus
  • [ON TRACK] Manual smoketests
  • [DONE] Bug fix verifications
  • {{ |}} Update tests on releasetest
  • {{ |}} Get final sign-off from Bob Moss
  • {{ |}} Update tests on release

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Result: report
Windows 2000 PASS - 2012-02-16
Windows XP PASS - 2012-02-16
Windows Vista PASS - 2012-02-16
Windows 7 32-bit PASS - 2012-02-16
Windows 7 64-bit PASS - 2012-02-16
Mac OS X 10.6 PASS - 2012-02-16
Mac OS X 10.7 PASS - 2012-02-16
Ubuntu 11.10 32-bit PASS - 2012-02-16
Ubuntu 11.10 64-bit PASS - 2012-02-16
Test Failures

Manual Tests

The following are the detailed results of the manual Smoke and ESR tests.

Platform Assignee Result: report
Windows 2000 tester result - YYYY-MM-DD
Windows XP RemusPop result - YYYY-MM-DD
Windows Vista tester result - YYYY-MM-DD
Windows 7 32-bit AlinT result - YYYY-MM-DD
Windows 7 64-bit AlinT result - YYYY-MM-DD
Mac OS X 10.6.8 AlexLakatos result - YYYY-MM-DD
Mac OS X 10.7.2 AlexLakatos result - YYYY-MM-DD
Ubuntu 11.10 32-bit RemusPop result - YYYY-MM-DD
Ubuntu 11.10 64-bit Vlad Maniac (irc: vladmaniac, #automation) PASS - 2012-02-17

Focus Tests

  1. {{|}PNG handling exploratory testing Etherpad

Update Tests

The following are details of update verifications

Platform Releasetest: [ report] Esr: [ report]
Windows 2000 result - YYYY-MM-DD result - YYYY-MM-DD
Windows XP result - YYYY-MM-DD result - YYYY-MM-DD
Windows Vista result - YYYY-MM-DD result - YYYY-MM-DD
Windows 7 32-bit result - YYYY-MM-DD result - YYYY-MM-DD
Windos 7 64-bit result - YYYY-MM-DD result - YYYY-MM-DD
Mac OS X 10.6 result - YYYY-MM-DD result - YYYY-MM-DD
Mac OS 10.7 result - YYYY-MM-DD result - YYYY-MM-DD
Ubuntu 11.10 32-bit result - YYYY-MM-DD result - YYYY-MM-DD
Ubuntu 11.10 64-bit result - YYYY-MM-DD result - YYYY-MM-DD
Issues

Bug Fix Verifications

[DONE] bug 727401 libpng: integer overflow leading to heap-buffer overflow

Issues Found

Bug Status
Retrieved from "https://wiki.mozilla.org/index.php?title=Releases/Firefox_10_ESR/Test_Plan&oldid=398768"