« Firefox 18 | Firefox 19 | Firefox 20 »

Firefox 19 Test Plan

Summary

Milestone	Firefox 19
Status	Currently in Beta -- merging to Release on February 19th, 2012
Lead	Anthony Hughes (irc: ashughes)
2nd	Juan Becerra (irc: juanb)

The following is the test plan for Firefox 19 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 19. After the release this document will be used as an archive of what was done to validate this release.

If you have some free time, please pick a task below and get in touch with one of the leads.

Details

The following is a description of the tasks which need to be completed throughout this release.

2012-10-08: [DONE] Nightly Merge
2012-11-23: [DONE] Aurora Merge
2013-01-07: [DONE] Pre-Beta feature sign-offs
2013-01-10: [DONE] Beta 1
2013-01-17: [ON TRACK] Beta 2
2013-01-24: {{|}} Beta 3
2013-01-31: {{|}} Beta 4
2013-02-07: {{|}} Beta 5
2013-02-14: {{|}} Beta 6
2013-02-15: {{|}} Release Candidate
2013-02-19: {{|}} Final

Nightly

Merge

Functional Automation - result

Update Spotchecks - result

Features

See the Feature Release Tracking wiki for a general overview

Features

See the Feature Release Tracking wiki for a general overview

Feature	Owner	Status	Tests
Panel-based Download Manager	Simona Badau	Delayed to Firefox 20
Tab Strip Visual Redesign	Virgil Dicu	Delayed
Windows Plugin Hang UI	Manuela Muntean	Delayed
Show PDF inline	Mihaela Velimiroviciu	On Track
Awesomebar Search Provider Reset		On Track

Endurance Automation

The purpose of this section is to track any potential memory regressions via spikes reported in the dashboard.

What to Do

Check the Endurance Charts
If you see an anomalous event (ie. spike or dip), get the report for the specific event
If the event happens twice in a row, file a bug under Mozmill Tests
Notify Anthony Hughes of the issue
Record the event in this wiki below

Current Status (Mihaela Velimiroviciu) - Last Updated - Nov 1st, 2012

Full Query

ID	Summary	Priority	Status
807338	Memory increase discovered by mozmill endurance tests - Mac 10.7&10.8 since Oct25	--	RESOLVED

1 Total; 0 Open (0%); 1 Resolved (100%); 0 Verified (0%);

Functional Automation

The purpose of this section is to track any potential functional regressions via results reported in the dashboard.

What to Do

Check the testrun reports
If you see any failing tests, check that there are not any bugs on file for it already
If not, file a bug under Mozmill Tests
Notify Anthony Hughes of the issue
Record the event in this wiki below

Current Status (Vlad Maniac / Alex Lakatos) - Last Updated

Bug Triage

The purpose of this section is to triage UNCONFIRMED incoming bugs.

What to Do

Check the UNCONFIRMED bugs
Test the bugs in the latest Firefox Nightly
If you can't reproduce the bug, ask for the reporter to provide more information
If you reproduce the bug, add your testing information and mark the bug NEW
Notify Anthony Hughes of the NEW issue

Current Status (SoftVision) - Last Updated

Result Spreadsheet

Aurora Merge


Builds	nightlies
Changeset	tip (pushlog)

Checklist

[DONE] Automated Testing
[DONE] Update Testing on auroratest
[DONE] Peer Sign-off
[DONE] Final sign-off
[DONE] Update Testing on aurora

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	PASS: report
Windows XP	PASS - 2012-11-23
Windows Vista	PASS - 2012-11-23
Windows 7 32-bit	PASS - 2012-11-23
Windows 7 64-bit	PASS - 2012-11-23
Windows 8 32-bit	PASS - 2012-11-23
Windows 8 64-bit	PASS - 2012-11-23
Mac OS X 10.6	PASS - 2012-11-23
Mac OS X 10.7	PASS - 2012-11-23
Mac OS X 10.8	PASS - 2012-11-23
Ubuntu 12.04 32-bit	PASS - 2012-11-23
Ubuntu 12.04 64-bit	PASS - 2012-11-23

Test Failures (known bugs)

Update Testing

Spotcheck updates from a previous Aurora to Aurora 19.

Platform	Aurora channel
Windows 7 64-bit	PASS - 2012-11-26	Firefox 17.0a2 2012-10-11 -> Firefox 19.0a2 2012-11-26
Windows 8 64-bit	PASS - 2012-11-26	Firefox 18.0a2 2012-11-15 -> Firefox 19.0a2 2012-11-26
Mac OS X 10.7	PASS - 2012-11-26	Firefox 18.0a2 2012-11-19 -> Firefox 19.0a2 2012-11-26
Ubuntu 12.04 64-bit	PASS - 2012-11-26	Firefox 17.0a2 2012-10-01 -> Firefox 19.0a2 2012-11-26

Issues

Bug Triage

VERIFYME: verify the fix and do some exploratory testing around the issue
FIXED: triage these bugs and test to ensure the fix is effective
QAWANTED: triage these bugs and test to fulfill the QA request
UNCONFIRMED: triage these bugs for potential new regressions

QAWANTED

Bugs requiring QA's assistance in debugging (steps, regression range, exploratory testing, verification, etc)

Status as of 2012-12-20:

bug 820805 crash in js::ion::IonBailoutIterator::IonBailoutIterator or js::ion::OperatorIn
- WORKSFORME - Unclear if qawanted is still neeeded (marcia)
bug 818626 crash in nsStyleContext::~nsStyleContext
- ASSIGNED - Untracked due to failed attemtps to reproduce, no more leads
bug 823119 crash in mozilla::gfx::DrawTargetD2D::CopySurface with PDF Viewer
- NEW - Reproduced many stability issues but not the specific signature (ashughes)
bug 668583 crash @ js::gc::ScanRope
- NEW - Pending feedback from Engineering about leads for QA to test (ashughes)
bug 698296 Firefox Crash Report @ je_free | js::gc::Arena::finalize
- NEW - So far unreproducible, any other leads? (ashughes)

Pre-beta Feature Sign-off

Firefox 19.0a2 Feature Status

Feature	Owner	Status
In-line PDF Viewer	MihaelaV (SV)	Ready for Beta
Awesomebar Search Provider Reset (bug 718088)	Paul Silaghi (SV), Manuela Muntean (SV)	Will be tested via Testday
Mac OSX 10.8 Notification Center Support (bug 728106, bug 777409)	unassigned	Moved to Firefox 20
Panel-based Download Manager	Simona Bidau (SV)	Moved to Firefox 20
Windows Plugin Hang UI	Manuela Muntean (SV)	Moved to Firefox 20
Tab Strip Visual Redesign	Virgil Dicu (SV)	Moved to Firefox 20

Beta Checklist

The following checklist should be completed before signing off for Release.

Beta 1

[DONE] Breakpad End-to-End functionality
[DONE] Mac OSX 10.8 Signing
[DONE] Add-on compatibility (top AMO, top third-party, toolbars, yandex, avg, norton, yahoo, bandoo, conduit, etc)

Beta 2

[DONE] Breakpad End-to-End functionality
[DONE] Stop accepting MD5 as a hash algorithm in signatures
[DONE] Plug-in compatibility (Flash, Java, Silverlight, Realplayer, Quicktime, Shockwave, Unity [Mac])
[DONE] Firefox Beta testday stressing flagship features and bug triage

Beta 3

[DONE] Breakpad End-to-End functionality
[DONE] Mac OSX 32-bit mode
[DONE] Web compatibility (search, webmail, banking, video, audio, gaming, news, cms, etc.)

Beta 4

[DONE] Breakpad End-to-End functionality
[DONE] Verify update applies with root owned files
[DONE] Add-on compatibility (top AMO, top third-party, toolbars, yandex, avg, norton, yahoo, bandoo, conduit, etc)
[DONE] Firefox Beta testday stressing flagship features and bug triage

Beta 5

{{ |}} Breakpad End-to-End functionality
{{ |}} What's New Page on Firefox update
{{ |}} Plug-in compatibility (Flash, Java, Silverlight, Realplayer, Quicktime, Shockwave, Unity [Mac])

Beta 6

{{ |}} Breakpad End-to-End functionality
{{ |}} Mac OSX 10.5 and Windows 2000 de-support (install and update)
{{ |}} Web compatibility (search, webmail, banking, video, audio, gaming, news, cms, etc.)

Beta 1

Builds	FTP, HTTP
ID	20130109111322
Changeset	222e6877be4b (pushlog)

Checklist

[DONE] Functional Automation
[DONE] Manual Testing
[DONE] Bug Triage
[DONE] Update Automation on betatest channel
[DONE] Update Automation on releasetest channel
[DONE] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Functional Results	Restart Results
Windows XP	[DONE] 0 Fails	[DONE] 0 Fails
Windows Vista	[DONE] 0 Fails	[DONE] 0 Fails
Windows 7 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 7 64-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 8 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 8 64-bit	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.6	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.7	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.8	[DONE] 0 Fails	[DONE] 0 Fails
Ubuntu 12.04 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Ubuntu 12.04 64-bit	[DONE] 0 Fails	[DONE] 0 Fails

Test Failures (known bugs)

Manual Testing

Report any bugs found during exploratory testing in Bugzilla and add them to the Issues Found section below.

Focus Area	Windows 7	Windows 8	Mac OSX 10.7	Mac OSX 10.8	Ubuntu 12.04
Regression Tests: Run the Firefox 19.0b1 Regression Tests in MozTrap. Do some exploratory testing around the testcases to try to exercise some edge-cases.	[DONE] Manuela 1 Bugs	[DONE] Simona 0 Bugs	[DONE] Bogdan 0 Bugs	[DONE] Paul 0 Bugs	[DONE] Alexandra 0 Bugs
PDF.js: Test the new built-in PDF reader to check for issues with viewing and printing PDFs. Check some of the behaviour with other PDF readers side-loaded (ie. Adobe Reader, Foxit Reader, etc).	[DONE] Manuela 0 Bugs	[DONE] Simona 1 Bugs	[DONE] Bogdan 0 Bugs	[DONE] Paul 0 Bugs	[DONE] Cornel 0 Bugs
Add-on Compatibility: Test the top add-ons for stability issues and functional issues. Test featured add-ons from AMO and Third-party toolbars like Yahoo, Yandex, AVG, Amazon, Norton, etc.	[DONE] Manuela 1 Bugs	[DONE] Simona 1 Bugs	[DONE] Bogdan 0 Bugs	[DONE] Paul 0 Bugs	[DONE] Cornel 0 Bugs
Plug-in Compatibility: Test the latest plug-ins for stability issues and visual glitches. Test plugins like Flash, Reader, Java, Quicktime, Realplayer, Windows Media Player, Silverlight, etc.	[DONE] Manuela 0 Bugs	[DONE] Simona 0 Bugs	[DONE] Bogdan 0 Bugs	[DONE] Paul 0 Bugs	[DONE] Cornel 0 Bugs
Web Compatibility: Test some of the top visited websites to look for stability issues and UI glitches. Some suggestions include WebMail (including built in messengers), Social Networks, Video sites, Media streaming websites, News websites, Content Management systems, Web banking, HTML5 video, WebGL/MapsGL, User forums	[DONE] Manuela 0 Bugs	[DONE] Simona 0 Bugs	[DONE] Bogdan 0 Bugs	[DONE] Paul 0 Bugs	[DONE] Cornel 0 Bugs

Update Testing

Results of running the Mozmill update automation.
18.0b6 should get a partial update, <=18.0b5 should get a complete update.

Platform	Betatest (2013-01-10)	Releasetest (2013-01-10)	Beta (2013-01-10)
Windows XP	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows Vista	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 7 32-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 7 64-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 8 32-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 8 64-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Mac OS X 10.6	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Mac OS X 10.7	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Mac OS X 10.8	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Ubuntu 12.04 32-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Ubuntu 12.04 64-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails

Bug Triage

QAWANTED

Bugs: Test to find steps to reproduce, regression range, or testcases	Status
bug 820506 Problem selecting image node in SVG when used with clip-path in certain condition	TESTING (ashughes)
bug 668583 Crash @ js::gc::ScanRope	NEEDINFO: Need more information from developer
bug 698296 Firefox Crash Report @ je_free js::gc::Arena::finalize	NEEDINFO: Need more information from developer
bug 814200 Xt plugins cannot be resized larger in FF17	VERIFYME: To be verified in 19b2
bug 815542 Crash in IncrementalCollectSlice	DROPPED: Nothing left for QA to do

NORMAL FIXES

Bugs: Verify these fixes in the latest Beta candidate	Status
bug 828507 Firefox 19b1 Beta updates fail to apply	TESTING (ashughes)
bug 802476 Printing crashes with a local file	VERIFIED FIXED (Cornel)
bug 789933 Crash in mozilla::gfx::DrawTargetCairo::Stroke	VERIFIED FIXED (Bogdan)
bug 818276 Scrollbar on popoups disappeared after updating	VERIFIED FIXED (Paul)
bug 812828 "Open All in Tabs" fails if Library window is open with no background windows	VERIFIED FIXED (Ioana)
bug 810586 Collapsible table not rendered correctly	NEEDINFO: Bug still reproduces under certain conditions (Paul)
bug 810531 Invalidation artifacts when scrolling w3.org working draft pages	VERIFIED FIXED (Manuela)
bug 799900 Wrong position of East Asian input method candidate box in HiDPI mode

SECURITY FIXES

Bugs: Verify these fixes in the latest Beta candidate (requires privileged access)	Status
bug 814839 WebGL conformance test hang [@mozilla::WebGLContext::Clear]	VERIFIED FIXED (mwobensmith)
bug 756581 JS OOM Testing: Assertion failure	DROPPED: Needs custom test script, reporter to verify (mwobensmith)
bug 812341 Opt-only Crash [@ SuppressDeletedPropertyHelper]	DROPPED: Needs custom shell, reporter to verify (mwobensmith)
bug 814407 WebGL conformance crash with Mesa 8.0 after calling texImage2D	DROPPED: Needs custom hardware, reporter to verify (mwobensmith)
bug 814716 WebGL conformance crash (NVIDIA Noveau Linux driver)	DROPPED: Needs custom hardware, reporter to verify (mwobensmith)
bug 780770 Content can prevent chrome buttons from working	VERIFIED FIXED (mnoorenberghe)

TOPCRASH FIXES

Verify these signatures are no longer present on crashstats in the current Beta

N/A for Beta 1 due to lack of crash data

UNCONFIRMED

Bugs: Triage unconfirmed bugs for potential incoming regressions.	Status
bug 806137 Private browsing wipes regular browsing session	CONFIRMED (Ioana)
bug 803840 keyboard shortcuts are different on each platform	CONFIRMED (Ioana)
bug 800032 Twitter website F5 key not working	NEEDINFO (Ioana)
bug 823722 "Flash Video Downloader - Youtube Downloader" stopped working with update	RESOLVED WORKSFORME (Ioana)
bug 810131 Status bar appears when switching loading tab to new tab	NEEDINFO (Ioana)
bug 812871 Highlighted context menu items font rendering issues on Windows XP	NEEDINFO (Ioana)
bug 809805 GameDesire Pool doesn't open in Nightly	NEEDINFO (Ioana)
bug 818863 Pinned tabs do not work with "don't load tabs until selected" feature	RESOLVED DUPE of bug 713190 (Ioana)
bug 812888 Issue with viewing YouTube videos.	RESOLVED WORKSFORME (Ioana)
bug 805426 Navigator error pops up when clicked on picture	RESOLVED INVALID (Ioana)

Issues Found

Bugs: New issues found during testing the Beta candidate	Status
bug 828372 Yandex Elements toolbar buttons are missplaced	RESOLVED INVALID: Yandex Elements bug
bug 828473 Yahoo Toolbar changes position after Yandex bar is installed	RESOLVED INVALID: Yahoo Toolbar bug
bug 828507 Firefox 19b1 Beta updates fail to apply	RESOLVED FIXED (Simona)
bug 828898 Firefox fails to display all PDF handling options	RESOLVED INVALID: Foxit Reader bug

Items in bold are BLOCKER bugs

Beta 2

Builds	FTP, HTTP
ID	20130116072953
Changeset	8b833c1150b5 (pushlog)

Checklist

[DONE] Functional Automation
[DONE] Manual Testing
[DONE] Bug Triage
[DONE] Update Automation on betatest channel
[DONE] Update Automation on releasetest channel
[DONE] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Functional Results	Restart Results
Windows XP	[DONE] 0 Fails	[DONE] 0 Fails
Windows Vista	[DONE] 0 Fails	[DONE] 0 Fails
Windows 7 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 7 64-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 8 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 8 64-bit	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.6	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.7	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.8	[DONE] 0 Fails	[DONE] 0 Fails
Ubuntu 12.04 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Ubuntu 12.04 64-bit	[DONE] 0 Fails	[DONE] 0 Fails

Test Failures (known bugs)

Manual Testing

Focus testing around bug fixes landed between Beta 1 and Beta 2 (see Bug Triage section below).

Update Testing

Results of running the Mozmill update automation.
19.0b1 should get a partial update, <=18.0b6 should get a complete update.

Platform	Betatest (2013-01-16)	Releasetest (2013-01-17)	Beta 2013-01-17)
Windows XP	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows Vista	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 7 32-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 7 64-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 8 32-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Windows 8 64-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Mac OS X 10.6	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Mac OS X 10.7	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Mac OS X 10.8	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Ubuntu 12.04 32-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails
Ubuntu 12.04 64-bit	[DONE] : 0 Fails	[DONE] : 0 Fails	[DONE] : 0 Fails

Bug Triage

NORMAL FIXES

Verify these fixes in the latest Beta candidate.

[DONE] VERIFIED bug 828202 System proxy settings issue with Firefox 18 [1] (Manuela)
[DONE] VERIFIED bug 814200 Xt plugins cannot be resized larger in FF17 (Virgil)
[DONE] VERIFIED bug 828632 System Proxy Configuration Causes Memory Leak, syslogd load [2] (Simona)
[DONE] VERIFIED bug 828236 issue with NTLM and some SSL Proxies [3] (reporter)
[DONE] VERIFIED bug 829646 Proxy Auto Config calling myIPAddress() in the top level fails (Alexandra)
[DONE] VERIFIED bug 818468 [Linux] Langpacks bundled in distribution/extensions are registered but disabled (reporter)
[DONE] VERIFIED bug 814434 [hidpi/retina] url suggestion box pops up on wrong screen (mwobensmith)
[DONE] VERIFIED bug 830303 Fix plugin-container spawning on BSD after bug 753046 (reporter)
[MISSED] NEEDS CRASHSTATS bug 828184 startup crash in nsPrefBranch::GetIntPref with QIPCAP.dll
[MISSED] UNVERIFIABLE bug 828205 crash in js::DebugScopeProxy::has with Firebug
- Relies on having Ubuntu-provided Firefox 19.0b2 builds for Ubuntu 10.04 which do not exist

These will require setting up a proxy and having Firefox connect to the internet through that proxy. 1. "This only impacts system proxy pac file settings on all platforms. file:// uris are the only type known to be effected. uris pointing to non existant files, using strange syntax, etc..."

2. "This is an OSX specific issue and the patch is OSX specific at compile time, so don't waste time with other platforms. Configure a system proxy using a file:// pac file and you should see the logging entries syslog (using 18.0.0) and verify they are gone with 18.0.1 while verifying that the pac file is honored. The fix generally impacts the pac file handling for os x so some general testing could be redone."

3. "Setup squid to do integrated windows authentication (i.e. NTLM) to access the proxy and then configure proxy use for http and https traffic. Honestly, that's a giant pain in the butt to do but that's what the interop problem was - specifically with accessing https through the proxy. Manual configuration is fine in this case."

4. "This bug is pretty self descriptive and the patch for it includes an automated regression test. The issue involves using the myIPAddress() function in PAC file outside the scope of the FindProxyForURL() function - the test in the patch illustrates such a case."

SECURITY FIXES

Verify these fixes in the latest Beta candidate (requires privileged access)

[DONE] VERIFIED bug 775228 (CVE-2012-5830) use-after-free when loading html file on osx (mwobensmith)
[DONE] VERIFIED bug 812341 Opt-only Crash [@ SuppressDeletedPropertyHelper] (mwobensmith)
[DONE] VERIFIED bug 785734 (CVE-2012-5833) Mesa crashes on certain texImage2D calls (reporter)

TOPCRASH FIXES

Verify these signatures are no longer present on crashstats in the current Beta

QAWANTED

Test the following bugs to find steps to reproduce, regression range, or testcases.

[AT RISK] NEEDS ENGR bug 668583 crash @ js::gc::ScanRope (ashughes)
[AT RISK] NEEDS ENGR bug 698296 crash in js::SweepBackgroundThings @ js::gc::Arena::finalize
[ON TRACK] VERIFYME bug 818468 Langpacks bundled in distribution/extensions are registered but disabled even if shown enabled
[ON TRACK] VERIFYME bug 828184 startup crash in nsPrefBranch::GetIntPref with QIPCAP.dll (Websense Endpoint)
UNASSIGNED bug 783369 Norton Confidential 2012/2013 crashes (couictlr.dll, coffplgn.dll)

UNCONFIRMED

Issues Found

Items in bold are BLOCKER bugs

Known Issues

Full Query

ID	Summary	Priority	Status
747909	Youtube.com In Full Screen mode - copy actions are not allowed on HTML5 videos	--	RESOLVED
797116	[Win8] Flash hang in F_1152915508 while playing Knight Hunters	P3	RESOLVED

2 Total; 0 Open (0%); 2 Resolved (100%); 0 Verified (0%);

Beta 3

Builds	FTP, HTTP
ID	20130123083802
Changeset	19124ba6cd20 (pushlog)

Checklist

[DONE] Functional Automation
[DONE] Manual Testing
[DONE] Bug Triage
[DONE] Update Automation on releasetest channel
[ON TRACK] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Functional Results	Restart Results
Windows XP	PASS	PASS
Windows Vista	PASS	PASS
Windows 7 32-bit	PASS	PASS
Windows 7 64-bit	PASS	PASS
Windows 8 32-bit	PASS	PASS
Windows 8 64-bit	PASS	PASS
Mac OS X 10.6	PASS	PASS
Mac OS X 10.7	PASS	PASS
Mac OS X 10.8	PASS	PASS
Ubuntu 12.04 32-bit	PASS	PASS
Ubuntu 12.04 64-bit	PASS	PASS

Test Failures (known bugs)

Manual Testing

Focus testing around bug fixes landed between Beta 2 and Beta 3 (see Bug Triage section below).

Update Testing

Results of running the Mozmill update automation.
19.0bx should get a partial update, <=xx.x should get a complete update.

Platform	[link Betatest] (2013-01-24)	[link Releasetest] (2013-01-24)	[link Beta] (2013-01-24)
Windows XP	Skipped	http://mozmill-ondemand.blargon7.com/#/update/reports?branch=19.0&platform=All&from=2013-01-24&to=2013-01-25	status: number of fails
Windows Vista	Skipped	http://mozmill-ondemand.blargon7.com/#/update/reports?branch=19.0&platform=All&from=2013-01-24&to=2013-01-25	status: number of fails
Windows 7 32-bit	Skipped	PASS: results	status: number of fails
Windows 7 64-bit	Skipped	PASS: results	status: number of fails
Windows 8 32-bit	Skipped	PASS: results	status: number of fails
Windows 8 64-bit	Skipped	PASS: results	status: number of fails
Mac OS X 10.6	Skipped	PASS: results	status: number of fails
Mac OS X 10.7	Skipped	PASS: results	status: number of fails
Mac OS X 10.8	Skipped	PASS: results	status: number of fails
Ubuntu 12.04 32-bit	Skipped	PASS: results	status: number of fails
Ubuntu 12.04 64-bit	Skipped	PASS: results	status: number of fails

Bug Triage

NORMAL FIXES

Verify these fixes in the latest Beta candidate. (change link above for beta 3)

[DONE] VERIFIED bug 828286 center of svg animation is incorrect on industrial demo at zamek.pmmf.hu, ever since SVG display lists became enabled (Ioana)
[DONE] VERIFIED bug 829921 click events not delivered with trivial CSS transform, margin, overflow: hidden (Ioana)
[DONE] VERIFIED bug 827217 webgl.can-lose-context-in-foreground=false crashes in desktop Firefox (Ioana)
[DONE] NEEDINFO bug 826115 warnings about rollups too noisy (Manuela)
[DONE] VERIFIED bug 818023 IonMonkey: function.caller does not always work (and in interpreter and JM) (Paul)
[DONE] [QA-] bug 829858 Null dereference in nsHttpChannel::InitOfflineCacheEntry() (Paul)
[DONE] NEEDINFO bug 789975 Extremely long pause while collecting telemetry information on the main thread (Paul)
[DONE] VERIFIED bug 830066 Drag & drop links into toolkit downloads manager to download files fails (TypeError: aInitiatingDocument is undefined) (Cornel)
[DONE] DROPPED bug 804605 Kerberos authentication does not work with CNAME (Cornel)
- Asked the reporter to verify.
[DONE] NEEDINFO bug 812647 Startup error: Could not open any libc. (osfile_unix_allthreads.jsm) (Virgil)
[DONE] VERIFIED bug 827225 "Assertion failure: obj->isTypedArray()" with WebGL readPixels (Virgil)
[DONE] VERIFIED bug 825544 Clicking on a duckduckgo result link while the page is loading prevents the result page from being recorded (Alexandra)
[DONE] DROPPED bug 821329 Input widgets don't accept character input with MacOS's build-in handwriting feature (Bogdan)
- Asked the reporter to verify.
[DONE] DROPPED bug 825620 Regression: Flash is triggering the discrete GPU (Bogdan)
- Asked the reporter to verify.
[DONE] VERIFIED bug 831780 scrollWidth, scrollHeight of DIV containing IMG reported as 40x40px until the image's actual size is known, regardless of CSS sizing of the IMG (Simona)
[DONE] REOPENED bug 830214 crash when printing selection of a PDF with many pages in pdf.js (Simona)
[DONE] VERIFIED bug 829435 Japanese characters appear completely garbled in PDF viewer (Mihaela)

SECURITY FIXES

Verify these fixes in the latest Beta candidate (requires privileged access) (change link above for beta 3)

[DONE] VERIFIED bug 831095 Use-After-Free crash @xul!nsImageLoadingContent::OnStopContainer (mwobensmith)
[DONE] VERIFIED bug 827070 Heap-buffer-overflow WRITE in nsSaveAsCharset::DoCharsetConversion (mwobensmith)
[DONE] DROPPED bug 690970 cycle collector can be made to Unlink live objects (by using weak maps or watchpoints) (mwobensmith)
- Verification of fix not possible and developer feels issue is theoretical

TOPCRASH FIXES

Verify these signatures are no longer present on crashstats in the current Beta (change link above for beta 3)

[DONE] VERIFIED bug 819888 crash in nsWindow::DealWithPopups (Paul)
[DONE] DROPPED bug 803568 Nightly crashed when see pdf file in fullscreen (presentation mode) or zooming (Cornel)
- Verification impeded by bug 728571.
[DONE] VERIFIED bug 806206 crash in JSObject::global (Ioana)
[DONE] VERIFIED bug 810618 crash in mozilla::dom::EventHandlerNonNull::`.* deleting destructor' (Alexandra)
[DONE] VERIFIED bug 803125 crash in imgStatusTrackerObserver::OnStartDecode (Alexandra)

QAWANTED

Test the following bugs to find steps to reproduce, regression range, or testcases.

{{|}} VERIFIED bug bugID Bug description (verifier)

UNCONFIRMED

[DONE] UNCONFIRMED bug 834014 Edit menu greyed when ("select all" or "copy" is available) (Manuela - changed the component)
[DONE] UNCONFIRMED bug 834237 pdf.js: Chapter headlines in some PDFs are ugly because one headline has an overlap with another. (Manuela - changed the component)
[DONE] RESOLVED INCOMPLETE bug 829540 Firefox would not load any pages in normal mode. Had to restrst in safe mode with extensions and add-ons disabled (Manuela)
[DONE] UNCONFIRMED bug 807991 FireFTP causes manu js sinc reflows (Simona - changed the component)
[DONE] RESOLVED DUPLICATE bug 829414 Clear List button broken in Beta 19 (Virgil)

Issues Found

Bugs: New issues found during testing the Beta candidate	Status
bug 834256 Firefox crashes while viewing PDF when many PDF documents are opened	NEW

Items in bold are BLOCKER bugs

Beta 4

Builds	FTP, HTTP
ID	20130130080006
Changeset	74d5479be250 (pushlog)

Checklist

[DONE] Functional Automation
[ON TRACK] Manual Testing
[ON TRACK] Bug Triage
[ON TRACK] Update Automation on betatest channel
[ON TRACK] Update Automation on releasetest channel
[ON TRACK] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Functional Results	Restart Results
Windows XP	[DONE] 0 Fails	[DONE] 0 Fails
Windows Vista	[DONE] 0 Fails	[DONE] 0 Fails
Windows 7 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 7 64-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 8 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 8 64-bit	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.6	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.7	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.8	[DONE] 0 Fails	[DONE] 0 Fails
Ubuntu 12.04 32-bit	[DONE] 0 Fails	[DONE] 1 Fails
Ubuntu 12.04 64-bit	[DONE] 0 Fails	[DONE] 1 Fails

Test Failures (known bugs)

Manual Testing

Report any bugs found during exploratory testing in Bugzilla and add them to the Issues Found section below.

Focus Area	Windows 7	Windows 8	Mac OSX 10.7	Mac OSX 10.8	Ubuntu 12.04
Regression Tests: Run the Firefox 19.0b4 Regression Tests in MozTrap. Do some exploratory testing around the testcases to try to exercise some edge-cases.	[DONE] Alexandra 0 Bugs	[DONE] Bogdan 0 Bugs	[DONE] Manuela 0 Bugs	[DONE] Virgil 0 Bugs	[DONE] Simona 0 Bugs
PDF.js: Test the new built-in PDF reader to check for issues with viewing and printing PDFs. Check some of the behaviour with other PDF readers side-loaded (ie. Adobe Reader, Foxit Reader, etc). Several bugs were marked for tracking in 19 for this feature. We may disable it. Pay close attention to this.	[DONE] Alexandra 1 Bugs	[DONE] Bogdan 1 Bugs	[DONE] Manuela 1 Bugs	[DONE] Virgil 0 Bugs	[DONE] Simona 0 Bugs

Update Testing

Results of running the Mozmill update automation.
19.0b3 should get a partial update, <= 19.0b2 should get a complete update.

Platform	Betatest (2013-01-30)	Releasetest (2013-01-30)	Beta (2013-01-30)
Windows XP	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails
Windows Vista	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails
Windows 7 32-bit	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails
Windows 7 64-bit	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails
Windows 8 32-bit	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails
Windows 8 64-bit	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails
Mac OS X 10.6	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails
Mac OS X 10.7	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails
Mac OS X 10.8	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails
Ubuntu 12.04 32-bit	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails
Ubuntu 12.04 64-bit	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails	[ON TRACK] : 0 Fails

Bug Triage

QAWANTED

NORMAL FIXES

[DONE] VERIFIED bug 832998 web page is MUCH slower in FF than in Chrome (Ioana)
[DONE] VERIFIED bug 828205 crash in js::DebugScopeProxy::has with Firebug (Ioana)
[DONE] DROPPED bug 835090 Where possible xpcshell tests should run on xulrunner (Ioana)
- Automated tests changes to run on XULrunner.
[DONE] VERIFIED bug 832576 WebGL uniform setters got 3-4x slower due to refactoring (Ioana)
[DONE] VERIFIED bug 809846 Click to Play doesn't play music on jango.com or other sites using the SoundManager2 library (Paul)
[DONE] VERIFIED bug 820497 click-to-play: asynchronously adding different plugins won't update the popup notification (Paul)
[DONE] VERIFIED bug 744745 Click to play overlay not appearing on vimeo.com (Paul)
[DONE] VERIFIED bug 819871 Nightly: can not login at Mail.com (Paul)
[DONE] VERIFIED bug 813550 Location bar's placeholder in an RTL chrome should be aligned on the right (Paul)
[DONE] VERIFIED bug 830086 NullPointerException in my webapp in Firefox 18 (Virgil)

SECURITY FIXES

[DONE] VERIFIED bug 761448 "ASSERTION: We should never be reusing a shared inner window" with pagehide event (mwobensmith)
[DONE] VERIFIED bug 812893 Heap-use-after-free in nsOverflowContinuationTracker::Finish, with -moz-columns (mwobensmith)
[DONE] DROPPED bug 814716 WebGL conformance crash (NVIDIA Noveau Linux driver) (mwobensmith)
- Needs custom hardware, reporter to verify

TOPCRASH FIXES

Verify these signatures are no longer present on crashstats in the current Beta

[DONE] UPDATED bug 812776 crash in nsCSSRendering::PrepareBackgroundLayer @ nsIFrame::GetStyleVisibility (Ioana)
Recent crashes available in Socorro.
[DONE] UPDATED bug 813489 crash in nsHttpTransaction::SetSecurityCallbacks (Ioana)
Recent crashes available in Socorro.
[DONE] UPDATED bug 812203 crash in nsHttpConnection::SetSecurityCallbacks (Ioana)
Recent crashes available in Socorro.
[DONE] VERIFIED bug 814953 crash in nsWindow::ProcessMessage (Ioana)
[DONE] VERIFIED bug 803568 Nightly crashed when see pdf file in fullscreen (presentation mode) or zooming (Ioana)

UNCONFIRMED

Issues Found

Items in bold are BLOCKER bugs PDF Viewer

[DONE] NEW bug 836732 Buttons in pdf.js remain in hover state after selection (Virgil)
[DONE] NEW bug 836736 Selected text font changes to default when firefox is left in the background in pdf.js (Alexandra)
[DONE] NEW bug 836748 Loading stuck on a very large PDF file (Bogdan)

Beta 5

Builds	FTP, HTTP
ID	20130206083616
Changeset	3e382bb14817 (pushlog)

Checklist

{{ |}} Functional Automation
{{ |}} Manual Testing
{{ |}} Bug Triage
{{ |}} Update Automation on betatest channel
{{ |}} Update Automation on releasetest channel
{{ |}} Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform	Functional Results	Restart Results
Windows XP	[DONE] 0 Fails	[DONE] 0 Fails
Windows Vista	[DONE] 0 Fails	[DONE] 0 Fails
Windows 7 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 7 64-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 8 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Windows 8 64-bit	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.6	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.7	[DONE] 0 Fails	[DONE] 0 Fails
Mac OS X 10.8	[DONE] 0 Fails	[DONE] 0 Fails
Ubuntu 12.04 32-bit	[DONE] 0 Fails	[DONE] 0 Fails
Ubuntu 12.04 64-bit	[DONE] 0 Fails	[DONE] 0 Fails

Test Failures (known bugs)

Manual Testing

Report any bugs found during exploratory testing in Bugzilla and add them to the Issues Found section below.

Regression Tests

Run the following regression tests from the above linked testrun. Do some exploratory testing and try to exercise some edge-cases around these scenarios.

Breakpad End-to-End functionality
- [DONE] Windows 7 (Bogdan)
- [DONE] Mac OSX 10.8 (Alexandra)
- [DONE] Ubuntu 12.04 (Manuela)
What's New Page on Firefox update
- [DONE] Windows 7 (Bogdan)
- [DONE] Mac OSX 10.8 (Alexandra)
- [DONE] Ubuntu 12.04 (Manuela)

PDF.js (if time permits)

Test the new built-in PDF reader to check for issues with viewing and printing PDFs. Check some of the behaviour with other PDF readers side-loaded (ie. Adobe Reader, Foxit Reader, etc). Several bugs were marked for tracking in 19 for this feature. We may disable it. Pay close attention to this.

[DONE] Windows 7 (Bogdan)
[DONE] Mac OSX 10.8 (Alexandra)
[DONE] Ubuntu 12.04 (Manuela)

PDF Viewer Blocker Triage

The following bugs need triaging for potentially blocking PDF.js from shipping in Firefox 19. Try to reproduce these bugs in Firefox 19, set the tracking-firefox19:? flag if it reproduces and you believe it should block. Add your justification in the bug and in this wiki below.

Add a bullet-item below for any bugs you think are potential feature blockers. Be sure to include a reason you believe it should block. Easily reproducible crashes, hangs, or inability to load or download a PDF during a common user scenario would be examples of potential blockers.

bug 738967 pdf.js doesn't work for embedded PDFs

Update Testing

Results of running the Mozmill update automation.
19.0b3 should get a partial update, <= 19.0b2 should get a complete update.

Platform	Betatest	[ Releasetest]	[ Beta]
Windows XP	[DONE] : 0 Fails	}}: N Fails	}}: N Fails
Windows Vista	[DONE] : 0 Fails	}}: N Fails	}}: N Fails
Windows 7 32-bit	[DONE] : 0 Fails	}}: N Fails	}}: N Fails
Windows 7 64-bit	[DONE] : 0 Fails	}}: N Fails	}}: N Fails
Windows 8 32-bit	[DONE] : 0 Fails	}}: N Fails	}}: N Fails
Windows 8 64-bit	[DONE] : 0 Fails	}}: N Fails	}}: N Fails
Mac OS X 10.6	[DONE] : 0 Fails	}}: N Fails	}}: N Fails
Mac OS X 10.7	[DONE] : 0 Fails	}}: N Fails	}}: N Fails
Mac OS X 10.8	[DONE] : 0 Fails	}}: N Fails	}}: N Fails
Ubuntu 12.04 32-bit	[DONE] : 0 Fails	}}: N Fails	}}: N Fails
Ubuntu 12.04 64-bit	[DONE] : 0 Fails	}}: N Fails	}}: N Fails

Bug Fix Verifications

The following fixes require verification before sign-off.

[DONE] COMMENTED bug 824162 Strange focus switching with chat windows (Virgil)
[DONE] REOPENED bug 830278 Printing selections in pdf.js don't work and can cause OOM errors in large PDFs (Simona)
[DONE] VERIFIED bug 834075 PDF url in the address bar becomes wrong in a certain situation (Alexandra)
bug 836887 Zoom-In button using wrong icon on HiDPI screens
[DONE] VERIFIED bug 797288 Pdf.js download behavior inconsistent (Manuela)
[DONE] COMMENTED bug 837371 Crash in nsContainerFrame::BuildDisplayListForNonBlockChildren (AMD Radeon HD 6xxx) (Bogdan)
[DONE] VERIFIED bug 825035 Blocklisted click-to-play notification only fades in/out once per window per session (Paul)
[DONE] VERIFIED bug 836463 PDF.js Broken Forms (Bogdan)
bug 835814 Heap-use-after-free in nsAsyncDOMEvent::Run
- Restricted access.
[DONE] VERIFIED bug 837845 Multi-GB memory spike when using regular expressions (Virgil)
[ON TRACK] bug 838001 Input Method can't be opened to type to input fields in a panel on Firefox 19+ (pzhang)
[ON TRACK] bug 799900 Wrong position of East Asian input method candidate box in HiDPI mode (haozhun)

Issues Found

The following are new issues found while testing the Beta candidate. Items in bold are BLOCKER bugs.

bug 839074 With pdf.js enabled, selected text doesn't stay selected after zoom (Manuela)
bug 839092 Clicking on titles from Show Document Outline does not always bring up on top of the page the title selected (Bogdan)
bug 839089 Moving with arrow keys left and right changes page on a pdf zoomed page (Bogdan)
bug 839096 Searching for consecutive words is broken in pdf.js (Alexandra)

Bug Triage

QAWANTED

These bugs require help finding steps to reproduce, regression ranges, and/or test cases.

[AT RISK] bug 836236 Crash in _ZThn184_N22nsComboboxControlFrame15GetRollupWidgetEv (juanb)
- Multiple unsuccessful attempts to reproduce the crash, though developer has made progress.

CRITICAL FIXES

These bugs need their fixes test and verified. These bugs should be tested before any other fixed bugs. Add a bullet item for any bugs you test.

[DONE] VERIFIED bug 802355 crash spike in mozilla::plugins::PluginModuleParent::StreamCast with abort message: "Corrupted plugin stream data.: file e:/builds/moz2_slave/m-cen-w32-ntly/build/dom/plugins/ipc/PluginModuleParent.cpp, line 844" (Virgil)
- No crashes found for the last 4 weeks in crash reports.
[DONE] VERIFIED bug 817284 new dom binding for windowless xhr events are broken (Paul)
[DONE] VERIFIED bug 830948 Crash [@ nsGlobalWindow::GetInnerHeight(int*)] while clicking View -> Zoom -> Zoom In on Firefox Aurora 20 (Paul)
[DONE] VERIFIED bug 798045 (CVE-2012-4191) crash in mozilla::net::FailDelayManager::Lookup (Virgil)

NORMAL FIXES

These bugs need their fixes tested and verified. Add a bullet item for any bugs you test.

SECURITY FIXES

These bugs need their fixes tested and verified. This will require elevated Bugzilla privileges. Add a bullet item for any bugs you test.

TOPCRASH FIXES

These bugs need to be tested to confirm the crash no longer reproduces. You will need to confirm you can reproduce the crash in a known broken build before verifying it's fixed in the latest build. If the crash is not reproducible for you in a known broken build, check crashstats for instances of the crash signature in the latest Firefox versions. Add a bullet item for any bugs you test.

UNCONFIRMED

These bugs need confirmation that they reproduce and need to be moved into an appropriate Bugzilla component. Add a bullet item for any bugs you test.

Known Issues

Full Query

ID	Summary	Priority	Status
828991	In a page with wide content (triggering shrink-to-fit), "print selection" prints very tiny output, even if the wide stuff isn't selected	--	NEW

1 Total; 1 Open (100%); 0 Resolved (0%); 0 Verified (0%);

Full Query

ID	Summary	Priority	Status
803456	Highlighting from Find within a PDF remains after closing Find bar	P2	RESOLVED

1 Total; 0 Open (0%); 1 Resolved (100%); 0 Verified (0%);

Releases/Firefox 19/Test Plan

Firefox 19 Test Plan

Summary

Details

Nightly

Merge

Features

Features

Endurance Automation

Functional Automation

Bug Triage

Aurora Merge

Automated Testing

Update Testing

Bug Triage

Pre-beta Feature Sign-off

Beta Checklist

Beta 1

Automated Testing

Manual Testing

Update Testing

Bug Triage

Issues Found

Beta 2

Automated Testing

Manual Testing

Update Testing

Bug Triage

Issues Found

Known Issues

Beta 3

Automated Testing

Manual Testing

Update Testing

Bug Triage

Issues Found

Beta 4

Automated Testing

Manual Testing

Update Testing

Bug Triage

Issues Found

Beta 5

Automated Testing

Manual Testing

PDF Viewer Blocker Triage

Update Testing

Bug Fix Verifications

Issues Found

Bug Triage

QAWANTED

CRITICAL FIXES

NORMAL FIXES

SECURITY FIXES

TOPCRASH FIXES

UNCONFIRMED

Known Issues

Navigation menu

Search