Releases/Firefox 11/Test Plan

From MozillaWiki
Jump to: navigation, search

« Firefox 10 | Firefox 11 | Firefox 12 »

Contents

Firefox 11 Test Plan

Summary

Milestone Firefox 11
Status Currently in Beta -- Release on March 13, 2012
Lead Anthony Hughes (irc: ashughes)
2nd Juan Becerra (irc: juanb)
Builds HTTPS

The following is the test plan for Firefox 11 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 11. After the release this document will be used as an archive of what was done to validate this release.

If you have some free time, please pick a task below and get in touch with one of the leads.

Details

The following is a description of the tasks which need to be completed throughout this release.

  • 2011-11-10: [DONE] Nightly Merge
  • 2011-12-22: [DONE] Aurora Merge
  • 2012-02-03: [DONE] Beta 1
  • 2012-02-07: [ON TRACK] Beta 2
  • 2012-02-14: {{|}} Beta 3
  • 2012-02:21: {{|}} Beta 4
  • 2012-02-28: {{|}} Beta 5
  • 2012-03-06: {{|}} Beta 6
  • 2012-03-09: {{|}} Release Candidate
  • 2012-03-13: {{|}} Final

Features

See the Feature Release Tracking wiki for a general overview

[DONE]
  • Chrome Migration (Paul Silaghi): SIGNED OFF on Firefox 11 RC build 1 on 2012-03-12
  • SPDY (Ioana Budnar): SIGNED OFF on Firefox 11 RC build 2 on 2012-03-13
  • Page Inspector Tilt (Alin Trif): SIGNED OFF 2012-01-25
  • Add-on Sync (Tracy Walker): SIGNED OFF 2012-01-25
    • NOTE: will not sync across application versions, add-ons outside of AMO, plugins, personas, and search plugins
[ON TRACK]
[AT RISK]
[MISSED]
Final Sign-off
  • Litmus testrun is being built (to be run in week leading up to Beta merge

Endurance Automation

Endurance automation is run every day during Nightly. The purpose of this section is to track any potential memory regressions via spikes reported in the dashboard.

Functional Automation

Functional automation is run every day on Nightly and Aurora. The purpose of this section is to track any potential functional regressions via results reported in the dashboard.

  • Owner: Vlad Maniac (2nd - Alex Lakatos)
  • Status: on track
  • Reports: dashboard
  • Issues: none found

Bug Triage

This section tracks bug triage work. The purpose is to triage UNCONFIRMED incoming bugs during Nightly and Aurora. Once fixed, they will be triaged and verified during Beta.

Aurora Merge

Functional Automation

Platform Result: report
Windows 2000 PASS - 2011-12-22
Windows XP PASS - 2011-12-22
Windows Vista PASS - 2011-12-22
Windows 7 32-bit PASS - 2011-12-22
Windows 7 64-bit PASS - 2011-12-22
Mac OS X 10.6 PASS - 2011-12-22
Mac OS X 10.7 PASS - 2011-12-22
Ubuntu 11.04 32-bit PASS - 2011-12-22
Ubuntu 11.04 64-bit PASS - 2011-12-22

Updates

Platform Result
Auroratest
Win32 Mozilla/5.0 (Windows NT 5.1; rv:10.0a2) Gecko/20111211 Firefox/10.0a2
Mozilla/5.0 (Windows NT 5.1; rv:11.0a2) Gecko/20111222 Firefox/11.0a2
Mac Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:10.0a2) Gecko/20111215 Firefox/10.0a2
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:11.0a2) Gecko/20111222 Firefox/11.0a2
Linux Mozilla/5.0 (X11; Linux i686 on x86_64; rv:10.0a2) Gecko/20111125 Firefox/10.0a2
Mozilla/5.0 (X11; Linux i686 on x86_64; rv:11.0a2) Gecko/20111222 Firefox/11.0a2
Aurora
Win32 Mozilla/5.0 (Windows NT 5.1; rv:10.0a2) Gecko/20111211 Firefox/10.0a2
Mozilla/5.0 (Windows NT 5.1; rv:11.0a2) Gecko/20111222 Firefox/11.0a2
Mac Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:10.0a2) Gecko/20111215 Firefox/10.0a2
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:11.0a2) Gecko/20111222 Firefox/11.0a2
Linux Mozilla/5.0 (X11; Linux i686 on x86_64; rv:10.0a2) Gecko/20111125 Firefox/10.0a2
Mozilla/5.0 (X11; Linux i686 on x86_64; rv:11.0a2) Gecko/20111222 Firefox/11.0a2

Beta 1

Builds: HTTPS
Litmus: testrun, report

Checklist
  • [DONE] Functional automated tests, synced to Litmus
  • [DONE] Manual tests
  • [DONE] Bug fix verifications (10%)
  • [DONE] Check with Support for top issues from previous Beta/Release
  • [DONE] Check with Marcia for top-crashers spiking
  • [DONE] Update tests on releasetest
  • [DONE] Get final sign-off from Bob Moss
  • [DONE] Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Assignee Result: report
Windows 2000 ashughes PASS - 2012-02-01
Windows XP ashughes PASS - 2012-02-01
Windows Vista ashughes PASS - 2012-02-01
Windows 7 32-bit ashughes PASS - 2012-02-01
Windows 7 64-bit ashughes PASS - 2012-02-01
Mac OS X 10.6 ashughes PASS - 2012-02-01
Mac OS X 10.7 ashughes PASS - 2012-02-01
Ubuntu 11.10 32-bit ashughes PASS - 2012-02-01
Ubuntu 11.10 64-bit ashughes PASS - 2012-02-01
Test Failures

Manual Tests

The following are the detailed results of the manual testing.

Smoketests

Run the manual smoketests and report results below.

Platform Assignee Result
Windows 2000 Simona PASS - 2012-02-02
Windows XP Ioana PASS - 2012-02-02
Windows Vista Simona PASS - 2012-02-02
Windows 7 32-bit Mihaela PASS - 2012-02-02
Windows 7 64-bit Paul PASS - 2012-02-02
Mac OS X 10.6.8 VladG PASS - 2012-02-02
Mac OS X 10.7.2 VladG PASS - 2012-02-02
Ubuntu 11.10 32-bit Ioana PASS - 2012-02-02
Ubuntu 11.10 64-bit Virgil PASS - 2012-02-02

Results in Litmus

Focus Testing

The following are the detailed results of the focused feature testing. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc

  1. [DONE] Rich Profile Updates Etherpad
  2. [DONE] Add-on Compatibility (including default compatible) Etherpad
  3. [DONE] Performance (typing and scrolling on intense pages -- like Facebook timeline) Etherpad
  4. [DONE] Web Compatibility (bank sites, social networking, video and audio streaming, etc) Etherpad
  5. [DONE] Anti-virus Compatibility (checking that AV software does not interfere, degrade Firefox performance) Etherpad
  6. [DONE] Plugin Stability (games & videos, flash, java, silverlight, etc) Etherpad
  7. [DONE] Run the Software Update and Silent Update FFTs
Feature Sign-off Testing

The following are the results of feature testing for those features which are shipping. Use the Firefox 11.0 Feature Sign-off tests in the Litmus testrun to qualify the features. Additionally, if time permits, do some exploratory testing around these features.

Feature Owner Win32 Mac Linux
Chrome Migration Paul Silaghi PASS - 2012-02-02 PASS - 2012-02-02 PASS - 2012-02-02
SPDY Ioana Budnar PASS - 2012-02-02 PASS - 2012-02-02 PASS - 2012-02-02
Page Inspector Tilt Alin Trif PASS - 2012-02-02 PASS - 2012-02-02 PASS - 2012-02-02
Add-on Sync Tracy Walker PASS - 2012-02-02 PASS - 2012-02-02 PASS - 2012-02-02

Update Tests

The following are details of update verifications

Platform Releasetest: report Beta: report
Windows 2000 PASS - 2012-02-02 PASS - 2012-02-03
Windows XP PASS - 2012-02-02 PASS - 2012-02-03
Windows Vista PASS - 2012-02-02 PASS - 2012-02-03
Windows 7 32-bit PASS - 2012-02-02 PASS - 2012-02-03
Windos 7 64-bit PASS - 2012-02-02 PASS - 2012-02-03
Mac OS X 10.6 PASS - 2012-02-02 PASS - 2012-02-03
Mac OS 10.7 PASS - 2012-02-02 PASS - 2012-02-03
Ubuntu 11.10 32-bit PASS - 2012-02-02 PASS - 2012-02-03
Ubuntu 11.10 64-bit PASS - 2012-02-02 PASS - 2012-02-03
Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

  • Any fix you very should have the status-firefox11 flag set to verified
  • If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
Queries

Issues Found

Bug Status
bug 723444 - Angry Birds is running jerky Need help investigating, should not block 11b1
bug 723615 - Fix Mozilla UA sniffing results "update to Firefox 10" what's new page
bug 723920 - Firefox 10beta does not auto-update INVALID: updates download in the background, prompt on completion

Known Issues

Bug Status
bug 681922 In google docs Upload -> Files has no highlighting Not a Firefox bug, regressed due to Google back-out
bug 697696 Highlight issue in amazon's password box Needs reduced testcase for debugging, no functional deficiency
bug 692446 Firefox prompts to install WMP plug-in though it's installed Still investigating, might be issue with Microsoft Installer
bug 702045 Make Firefox able to survive a Windows System Restore Development work still ongoing
bug 278860 confusing "profile in use"/"already running" error when profile is missing (not found) NEW

Feedback

If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.

Builds can be found here: HTTPS

You can post your feedback below or in this etherpad.

Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.

Beta 2

Builds: HTTPS
Litmus: testrun, report

Checklist
  • [DONE] Functional automated tests, synced to Litmus
  • [DONE] Manual tests
  • [DONE] Bug fix verifications (20%)
  • [DONE] Check with Support for top issues from previous Beta/Release
  • [DONE] Check with Marcia for top-crashers spiking
  • [DONE] Update tests on releasetest
  • [DONE] Get final sign-off from Bob Moss
  • [DONE] Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Assignee Result: report
Windows 2000 ashughes PASS - 2012-02-08
Windows XP ashughes PASS - 2012-02-08
Windows Vista ashughes PASS - 2012-02-08
Windows 7 32-bit ashughes PASS - 2012-02-08
Windows 7 64-bit ashughes PASS - 2012-02-08
Mac OS X 10.6 ashughes PASS - 2012-02-08
Mac OS X 10.7 ashughes PASS - 2012-02-08
Ubuntu 11.10 32-bit ashughes PASS - 2012-02-08
Ubuntu 11.10 64-bit ashughes PASS - 2012-02-08
Test Failures

Manual Tests

The following are the detailed results of the manual testing.

Smoketests

Run the manual smoketests and report results below.

Platform Assignee Result
Windows 2000 Simona PASS - 2012-02-10
Windows XP Remus PASS - 2012-02-09
Windows Vista Simona PASS - 2012-02-10
Windows 7 32-bit AlinT PASS - 2012-02-09
Windows 7 64-bit Paul PASS - 2012-02-10
Mac OS X 10.6.8 Alex PASS - 2012-02-09
Mac OS X 10.7.2 Alex PASS - 2012-02-09
Ubuntu 11.10 32-bit Ioana PASS - 2012-02-10
Ubuntu 11.10 64-bit Virgil PASS - 2012-02-10

Results in Litmus

Focus Testing

Run the Fx Firefox 11.0 BFT - Regression Tests from the Litmus testrun and then do exploratory testing around those areas. Indicate any detailed results or issues found below.

Platform Assignee Result
Windows XP Virgil PASS - 2012-02-10
Windows Vista Simona PASS - 2012-02-10
Windows 7 32-bit Ioana PASS - 2012-02-10
Windows 7 64-bit Paul PASS - 2012-02-10
Mac OS X 10.6.8 Mihaela PASS - 2012-02-10
Ubuntu 11.10 32-bit Ioana PASS - 2012-02-10
Ubuntu 11.10 64-bit Virgil PASS - 2012-02-10

Update Tests

The following are details of update verifications

Platform Releasetest: report Beta: report
Windows 2000 PASS - 2012-02-09 PASS - 2012-02-10
Windows XP PASS - 2012-02-09 PASS - 2012-02-10
Windows Vista PASS - 2012-02-09 PASS - 2012-02-10
Windows 7 32-bit PASS - 2012-02-09 PASS - 2012-02-10
Windos 7 64-bit PASS - 2012-02-09 PASS - 2012-02-10
Mac OS X 10.6 PASS - 2012-02-09 PASS - 2012-02-10
Mac OS 10.7 PASS - 2012-02-09 PASS - 2012-02-10
Ubuntu 11.10 32-bit PASS - 2012-02-09 PASS - 2012-02-10
Ubuntu 11.10 64-bit PASS - 2012-02-09 PASS - 2012-02-10
Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

  • Any fix you very should have the status-firefox11 flag set to verified
  • If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
  • Softvision verifications spreadsheet
Queries
Bugs Status
QA+: non-security bugs which need verification VERIFIED: 7 / 90
QA+ SG: security bugs which need verification VERIFIED: 0 / 31
status-firefox11:fixed: bugs needing triage for QA+ TRIAGED: 24 / 24
P1 Verifications
bug 718939
Java applet causes text entry fields to become semi-unresponsive
VERIFIED FIXED (juanb)
bug 724284 Use after free in nsXBLDocumentInfo::ReadPrototypeBindings PENDING (marcia)

Issues Found

Bug Status
bug 726028
Crash in firefox with nssCertificate_Destroy-not reproducible
Needs investigation
bug 725975
Adobe flash plugin crashes when swithching full screen mode on/off (CMD+SHIFT+F) with focus on flash content (video) (Single Core Processor)
Investigating, reproducible back Firefox 6
bug 725296
Browser is unresponsive while Silverlight content is loading
Investigating, reproducible back to 3.6
bug 725631
Forward button disabled after navigating back to Youtube
Not widely reproducible, geo-locale specific?
bug 725614
Application shortcut keys fail to work on pages containig Java/Shockwave/PDF apps after focus is restored to other browser content
Not a regression, reproducible back to 3.6.26
bug 723444
Angry Birds is running jerky
Firefox 11 regression, mozilla-central 2011-11-21
bug 702045
Make Firefox able to survive a Windows System Restore
Development work ongoing
bug 697696
Highlight issue in amazon's password box
Tech Evangelism: Amazon's bug
bug 681922
In google docs Upload -> Files has no highlighting
Tech Evangelism: Google's bug
bug 692446
Firefox prompts to install windows media player plugin even though it is already installed
Issue with MSI, not strictly a Firefox bug
bug 638770
clicking on a quicktime video pauses it
Trivial UX issue
bug 709029
The RealPlayer plugin crashes when trying to play a RealPlayer video
Very old regression going back to 4.0 (OOPP?)
bug 590963
Google Maps - Earth View Causes Freeze
Very old regression going back to 4.0b4

Feedback

If you want to help test and just send us general feedback, please do so below. Areas we are looking for feedback include but are not limited to hangs, crashes, add-on compatibility, youtube, facebook, twitter, etc.

Builds can be found here: HTTPS

You can post your feedback below or in this etherpad.

Be sure to install the Add-on Compatibility Reporter if you want to report incompatibilities.

Beta 3 build 2

Builds: HTTPS
Litmus: testrun, report
Mozmill: [ report]

Checklist
  • [DONE] Functional automated tests, synced to Litmus
  • [DONE] Manual tests
  • [DONE] Bug fix verifications
  • [DONE] Update tests on releasetest
  • {{ |}} Check with Marcia for top-crashers spiking
  • {{ |}} Get final sign-off from Bob Moss
  • {{ |}} Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Result: report
Windows 2000 PASS - 2012-02-16
Windows XP PASS - 2012-02-16
Windows Vista PASS - 2012-02-16
Windows 7 32-bit PASS - 2012-02-16
Windows 7 64-bit PASS - 2012-02-16
Mac OS X 10.6 PASS - 2012-02-16
Mac OS X 10.7 PASS - 2012-02-16
Ubuntu 11.10 32-bit PASS - 2012-02-16
Ubuntu 11.10 64-bit PASS - 2012-02-16
Test Failures

Manual Tests

Smoketests

The following are the detailed results of the manual Smoktests tests.

Platform Assignee Result: report
Windows 2000 Virgil PASS - 2012-02-16
Windows XP Ioana PASS - 2012-02-16
Windows Vista Simona PASS - 2012-02-16
Windows 7 32-bit Mihaela PASS - 2012-02-16
Windows 7 64-bit ashughes PASS - 2012-02-16
Mac OS X 10.6.8 Vlad PASS - 2012-02-16
Mac OS X 10.7.3 Vlad PASS - 2012-02-16
Ubuntu 11.10 32-bit Ioana PASS - 2012-02-16
Ubuntu 11.10 64-bit Virgil PASS - 2012-02-16

Focus Tests

The following are the detailed results of the focused testing and running the regression tests. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc

  1. [DONE] PNG handling exploratory testing Etherpad
Platform Assignee Result: report
Windows 2000 tester result - YYYY-MM-DD
Windows XP tester result - YYYY-MM-DD
Windows Vista tester result - YYYY-MM-DD
Windows 7 32-bit tester result - YYYY-MM-DD
Windows 7 64-bit tester result - YYYY-MM-DD
Mac OS X 10.6.8 tester result - YYYY-MM-DD
Mac OS X 10.7.3 tester result - YYYY-MM-DD
Ubuntu 11.10 32-bit tester result - YYYY-MM-DD
Ubuntu 11.10 64-bit tester result - YYYY-MM-DD

NOTE: Prioritize Windows 7, XP, Mac OS X 10.6, and Ubuntu 32-bit -- do other platforms only as time permits

Update Tests

The following are details of update verifications

Platform Releasetest: report Release: report
Windows 2000 PASS - 2012-02-16 PASS - 2012-02-17
Windows XP PASS - 2012-02-16 PASS - 2012-02-17
Windows Vista PASS - 2012-02-16 PASS - 2012-02-17
Windows 7 32-bit PASS - 2012-02-16 PASS - 2012-02-17
Windos 7 64-bit PASS - 2012-02-16 PASS - 2012-02-17
Mac OS X 10.6 PASS - 2012-02-16 PASS - 2012-02-17
Mac OS 10.7 PASS - 2012-02-16 PASS - 2012-02-17
Ubuntu 11.10 32-bit PASS - 2012-02-16 PASS - 2012-02-17
Ubuntu 11.10 64-bit PASS - 2012-02-16 PASS - 2012-02-17
Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

  • Any fix you very should have the status-firefox11 flag set to verified
  • If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
  • Softvision verifications spreadsheet
Queries
Bugs Status
P1 bug 727401 libpng: integer overflow leading to heap-buffer overflow VERIFIED FIXED
QA+: non-security bugs which need verification VERIFIED: 9 / 90
QA+ SG: security bugs which need verification VERIFIED: 2 / 31
status-firefox11:fixed: bugs needing triage for QA+ TRIAGED: 0

Issues Found

Bug Status
Previously Known
bug 725982
Flash player stops rendering on right click
NEW.Needs investigation
bug 726028
Crash in firefox with nssCertificate_Destroy-not reproducible
Needs investigation
bug 725975
Adobe flash plugin crashes when swithching full screen mode on/off (CMD+SHIFT+F) with focus on flash content (video) (Single Core Processor)
Investigating, reproducible back Firefox 6
bug 725296
Browser is unresponsive while Silverlight content is loading
Investigating, reproducible back to 3.6
bug 725631
Forward button disabled after navigating back to Youtube
Not widely reproducible, geo-locale specific?
bug 725614
Application shortcut keys fail to work on pages containig Java/Shockwave/PDF apps after focus is restored to other browser content
Not a regression, reproducible back to 3.6.26
bug 723444
Angry Birds is running jerky
Firefox 11 regression, mozilla-central 2011-11-21
bug 702045
Make Firefox able to survive a Windows System Restore
Development work ongoing
bug 697696
Highlight issue in amazon's password box
Tech Evangelism: Amazon's bug
bug 681922
In google docs Upload -> Files has no highlighting
Tech Evangelism: Google's bug
bug 692446
Firefox prompts to install windows media player plugin even though it is already installed
Issue with MSI, not strictly a Firefox bug
bug 638770
clicking on a quicktime video pauses it
Trivial UX issue
bug 709029
The RealPlayer plugin crashes when trying to play a RealPlayer video
Very old regression going back to 4.0 (OOPP?)
bug 590963
Google Maps - Earth View Causes Freeze
Very old regression going back to 4.0b4
bug 717521
Moving the map in Bing Maps locks my X11
NEW

Beta 4

Builds: HTTPS
Litmus: testrun, report
Mozmill: dashboard

Checklist
  • [DONE] Functional automated tests, synced to Litmus
  • [DONE] Manual tests
  • [DONE] Bug fix verifications
  • [DONE] Update tests on releasetest
  • [DONE] Check with Marcia for top-crashers spiking
  • [MISSED] Get final sign-off from Bob Moss
  • {{ |}} Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Result: report
Windows 2000 PASS - 2012-02-22
Windows XP PASS - 2012-02-22
Windows Vista PASS - 2012-02-22
Windows 7 32-bit PASS - 2012-02-22
Windows 7 64-bit PASS - 2012-02-22
Mac OS X 10.6 PASS - 2012-02-22
Mac OS X 10.7 PASS - 2012-02-22
Ubuntu 11.10 32-bit PASS - 2012-02-22
Ubuntu 11.10 64-bit PASS - 2012-02-22
Test Failures

Manual Tests

Smoketests

The following are the detailed results of the manual Smoktests tests.

Platform Assignee Result: report
Windows 2000 Simona PASS - 2012-02-23
Windows XP Ioana PASS - 2012-02-23
Windows Vista Simona PASS - 2012-02-23
Windows 7 32-bit Mihaela PASS - 2012-02-23
Windows 7 64-bit Paul PASS - 2012-02-23
Mac OS X 10.6.8 Vlad PASS - 2012-02-23
Mac OS X 10.7.3 Vlad PASS - 2012-02-23
Ubuntu 11.10 32-bit Ioana PASS - 2012-02-23
Ubuntu 11.10 64-bit Ioana PASS - 2012-02-23

Focus Tests

The following are the detailed results of the focused testing and running the regression tests. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc

Platform Assignee Result: report
Windows XP Paul PASS - 2012-02-23
Windows 7 32-bit Mihaela PASS - 2012-02-23
Mac OS X 10.6.8 Vlad PASS - 2012-02-23
Ubuntu 11.10 32-bit Ioana PASS - 2012-02-23

NOTE: Prioritize Windows 7, XP, Mac OS X 10.6, and Ubuntu 32-bit -- do other platforms only as time permits

Update Tests

The following are details of update verifications

Note: bug 717214 disables new updates for 9.0b5 and earlier -- they update to 11.0b2 then get 11.0b2->11.0b4. This is a temporary stop-gap to ease burden on RelEng resources.

Platform Releasetest: report Beta: report
Windows 2000 PASS - 2012-02-23 PASS - 2012-02-24
Windows XP PASS - 2012-02-23 PASS - 2012-02-24
Windows Vista PASS - 2012-02-23 PASS - 2012-02-24
Windows 7 32-bit PASS - 2012-02-23 PASS - 2012-02-24
Windows 7 64-bit PASS - 2012-02-23 PASS - 2012-02-24
Mac OS X 10.6 PASS - 2012-02-23 PASS - 2012-02-24
Mac OS 10.7 PASS - 2012-02-23 PASS - 2012-02-24
Ubuntu 11.10 32-bit PASS - 2012-02-23 PASS - 2012-02-24
Ubuntu 11.10 64-bit PASS - 2012-02-23 PASS - 2012-02-24
Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

  • Any fix you very should have the status-firefox11 flag set to verified
  • If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
  • Softvision verifications spreadsheet
Queries
Bugs Status
QA-WANTED: bugs which need QA to reproduce TRIAGED: 12 / 12
QA+: non-security bugs needing verification VERIFIED: 22 / 72
QA+ SG: security bugs which need verification VERIFIED: 1 / 36
status-firefox11:fixed: bugs needing triage for QA+ TRIAGED: 25

Issues

New

If you find any bugs which are not listed in the Tracking section below, please file them in bugzilla and link them below.

  • bug 730285 - No update channel is found after restoring the OS to a previous point
    • INVESTIGATING: Seems to only affect those who change their channel
  • bug 729872 - HTML5 fullscreen loosing focus causes text entry fields to become unresponsive
    • INVESTIGATING: Regression in Nightly caused by bug 685402 (2011-11-04)
  • bug 729907 - IMDB videos lose their image and controls after restart
    • TRACKING-: Affects back to Firefox 3.6, reloading the page works around the issue
  • bug 547152 - Stray RealPlayer plugin frame appears after switching to another tab
    • TRACKING-: Affects back to Firefox 2.0

Tracking

investigating
  • [RESOLVED] bug 725982 - Shockwave player stops rendering on right click
    • Fixed by updating Shockwave plugin to the latest
  • bug 726028 - Crash in nssCertificate_Destroy
    • Still investigating, so far unreproducible (2012-02-22)
  • bug 725975 - Flash crashes switching full screen mode
    • Still investigating, so far unreproducible (2012-02-22)
  • bug 725631 - Forward button disabled after navigating back to Youtube
    • WONTFIX? (2012-02-22)
tracking?
  • bug 725296 - Hang while Silverlight content is loading
    • Also affects 3.6, Chrome (2012-02-22)
  • bug 725614 - Shortcut keys fail for pages containing Java/Shockwave/PDF after focus on browser content
    • Also affects 3.6 (2012-02-22)
  • bug 692446 - Firefox prompts to install windows media player plugin even though it is already installed
    • Not a Firefox bug, devs working on workaround (2012-02-22)
  • bug 709029 - RealPlayer plugin crashes
    • Started with OOPP (2012-02-22)
  • bug 590963 - Google Maps Earth View causes hang
    • Firefox 4 regression (2012-02-22)
  • bug 717521 - Moving the map in Bing Maps locks X11
tracking+
  • bug 723444 - Angry Birds is running jerky [ASSIGNED]
    • Working with devs by testing TRY builds
  • bug 702045 - Make Firefox able to survive a Windows System Restore [ASSIGNED]
    • Last status update was Nov'11 (2012-02-22)
  • bug 715401 - Firefox Crash @ nsCanvasRenderingContext2DAzure::ClearRect with Yahoo Toolbar
    • QA-WANTED

Beta 5

Builds: HTTPS
Litmus: testrun, report
Mozmill: dashboard

Checklist
  • [DONE] Functional automated tests, synced to Litmus
  • [DONE] Manual tests
  • [DONE] Bug fix verifications
  • [DONE] Update tests on releasetest
  • [MISSED] Check with Marcia for top-crashers spiking (PTO)
  • [DONE] Get final sign-off from Bob Moss
  • [DONE] Update tests on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Result: report
Windows 2000 PASS - 2012-02-29
Windows XP PASS - 2012-02-29
Windows Vista PASS - 2012-02-29
Windows 7 32-bit PASS - 2012-02-29
Windows 7 64-bit PASS - 2012-02-29
Mac OS X 10.6 PASS - 2012-02-29
Mac OS X 10.7 PASS - 2012-02-29
Ubuntu 11.10 32-bit PASS - 2012-02-29
Ubuntu 11.10 64-bit PASS - 2012-02-29
Test Failures
  • bug 731690 - testPreferences/testPreferredLanguage.js: possible Google regression

Manual Tests

Smoketests

The following are the detailed results of the manual Smoktests tests.

Platform Assignee Result: report
Windows 2000 Simona PASS - 2012-03-01
Windows XP Ioana PASS - 2012-03-01
Windows Vista Simona PASS - 2012-03-01
Windows 7 32-bit Mihaela PASS - 2012-03-01
Windows 7 64-bit Paul PASS - 2012-03-01
Mac OS X 10.6.8 VladG PASS - 2012-03-01
Mac OS X 10.7.3 VladG PASS - 2012-03-01
Ubuntu 11.10 32-bit Ioana PASS - 2012-03-01
Ubuntu 11.10 64-bit Virgil PASS - 2012-03-01

Focus Tests

The following are the detailed results of the focused testing and running the regression tests. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc

Platform Assignee Result: report
Windows 7 32-bit Mihaela PASS - 2012-03-01
Mac OS X 10.6.8 VladG PASS - 2012-03-01
Ubuntu 11.10 32-bit Ioana PASS - 2012-03-01
Windows XP Ioana, Mihaela PASS - 2012-03-02
MacOS X 10.7.3 VladG PASS - 2012-03-02
Ubuntu 11.10 64-bit Virgil PASS - 2012-03-02
Windows 7 64-bit Paul PASS - 2012-03-02

NOTE: Prioritize Windows 7, Mac OS X 10.6, and Ubuntu 32-bit -- do other platforms only as time permits

Update Tests

The following are details of update verifications

Note: bug 717214 disables new updates for 9.0b5 and earlier -- they update to 11.0b2 then get 11.0b2->11.0b4. This is a temporary stop-gap to ease burden on RelEng resources.

Platform Releasetest: report Beta: report
Windows 2000 PASS - 2012-03-01 PASS - 2012-03-02
Windows XP PASS - 2012-03-01 PASS - 2012-03-02
Windows Vista PASS - 2012-03-01 PASS - 2012-03-02
Windows 7 32-bit PASS - 2012-03-01 PASS - 2012-03-02
Windows 7 64-bit PASS - 2012-03-01 PASS - 2012-03-02
Mac OS X 10.6 PASS - 2012-03-01 PASS - 2012-03-02
Mac OS 10.7 PASS - 2012-03-01 PASS - 2012-03-02
Ubuntu 11.10 32-bit PASS - 2012-03-01 PASS - 2012-03-02
Ubuntu 11.10 64-bit PASS - 2012-03-01 PASS - 2012-03-02
Issues

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

  • Any fix you very should have the status-firefox11 flag set to verified
  • If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
  • Softvision verifications spreadsheet
Queries
  • QA+: non-security bugs needing verification (10 / 11 VERIFIED)
  • QA+ SG: security bugs which need verification (0 / 33 VERIFIED)
  • status-firefox11:fixed: bugs needing triage for QA+ (0 / ? TRIAGED)
  • UNCO: unconfirmed bugs filed during Firefox 11 (0 / 38 TRIAGED)

Issues

New
Bug Status
bug 731981 - Missing steps for missing Shockwave plugin notification Tech Evangelism - website bug due to user-agent
bug 732388 - Firefox crash ffi_call (Roboform Toolbar 7.7.3.0) Need investigating
QA Investigating (QA-WANTED)
Bug Status
bug 720679 - Crash @ mozilla::dom::workers::WorkerPrivate::CancelAllTimeouts Marcia was investigating, does bsmedbergs feedback help?
bug 701404 - Silverlight crashiness using Bing Video Vlad needs to follow-up with testing Silverlight 5 64-bit
bug 716391 - Crash @je_free xul.dll@0xbb...f Marcia was investigating, status update?
bug 723444 - Angry Birds is running jerky Need to do comparison with Firefox 10 (Paul:SV)
Engineering Investigating
Bug Status
bug 729872 - HTML5 fullscreen breaks text entry carat ASSIGNED, Regression from bug 685402
bug 692446 - WMP plugin repeatedly asks to install ASSIGNED
bug 717521 - Moving the map in Bing Maps locks X11 ASSIGNED, Regression from bug 702739
bug 730285 - Can't update Firefox after Windows system restore ASSIGNED
bug 715401 - Crash @nsCanvasRenderingContext2DAzure::ClearRect (Yahoo Toolbar) Bas Schouten is investigating
bug 671468 - Crash @nsSocketOutputStream::Write (Yandex) Over to Honza for STR
bug 711656 - Crash in TextStageManager::AddStagesForSubrect w/Intel GMA4500HD Benoit Jacob is investigating
bug 731720 - Hang logging into Evernote Alex Keybl is investigating
bug 656758 - Crash @nsSocketInputStream::Read (nsUrlClassifierDBService) Josh Aas is investigating
Followup Needed
Bug Status

Day 2 Testing

Ideas for Testing:

  • looking through unconfirmed bugs filed against Firefox 11 for possible regressions (http://bit.ly/wps2eJ)
  • regression tests on other platforms (WinXP, OSX 10.7, Ubuntu 64-bit, Windows 7 64-bit in that order)
  • exploratory testing against top-sites (Facebook, Youtube, webmail, banking, etc)
  • exploratory testing with top addons
  • default compatible add-ons (installing add-ons in a previous beta which are not compatible with 11 and checking that they still work after updating to Firefox 11 on releasetest)
  • exploratory testing for performance issues (GC/CC times, hangs, crashes, etc)

Blocklist Testing

When testing the following blocklist bugs, please record results in this spreadsheet and provide any specific details below.

bug 715744 - DLL block request BExternal.dll
  • Software
  • Steps to Perform:
    • Install Firefox 11.0b5
    • Install the Babylon Software
    • Start Firefox and verify the Babylon Toolbar is installed
    • Repeat with the try build
    • Verify the Babylon software has been blocklisted
  • Results
bug 665775 - Blocklist Extensions and dlls to prevent datamngr.dll crashes
  • Software
    • JZip (should include JZip Toolbar)
    • Bandoo (should include MediaBar/SearchQu Toolbar)
    • iMesh (should include iMesh Toolbar)
    • BearShare (should include BearShare Toolbar)
    • Try to track down older versions of the software as well
    • Try with different mixes (one, many, or all of the listed software)
  • Steps to Perform:
    • Install Firefox 11.0b5
    • Install some of the above software
    • Start Firefox and verify the add-ons are installed
    • Repeat with the [ try] build
    • Verify the add-ons are blocklisted
  • Results

Default to Compatible

See bug 729654

Automation

4.0b7-b12, 4.0, 4.0.1, 5.0b*, 5.0, 5.0.1, 6.0b*, 6.0, 7.0b*, 8.0b*, 9.0b* -> 11.0b5

Platform Releasetest: report Beta: report
Windows 2000 PASS - 2012-03-05 PASS - 2012-03-06
Windows XP PASS - 2012-03-05 PASS - 2012-03-06
Windows Vista PASS - 2012-03-05 PASS - 2012-03-06
Windows 7 32-bit PASS - 2012-03-05 PASS - 2012-03-06
Windows 7 64-bit PASS - 2012-03-05 PASS - 2012-03-06
Mac OS X 10.6 PASS - 2012-03-05 PASS - 2012-03-06
Mac OS 10.7 PASS - 2012-03-05 PASS - 2012-03-06
Ubuntu 11.10 32-bit PASS - 2012-03-05 PASS - 2012-03-06
Ubuntu 11.10 64-bit PASS - 2012-03-05 PASS - 2012-03-06


Manual (Releasetest)
  • one add-on with maxVer for each release background updates to 11.0b5 (you can re-use add-ons from this spreadsheet)
  • Results (Google Spreadsheet)
Addons Windows Mac Linux
Firefox 9.*

Cooliris, AniWeather, IE Tab Mix Plus, Evernote Web Clipper, Fire Shot, Download Flash and Video, Interface pt-BR, Awesome Screenshot, Foxy Tunes, Video Surf

PASS 2012-03-05 PASS 2012-03-05 PASS 2012-03-05
Firefox 8.*

English South African Language Pack, Romanian Language Pack, Afrikaans Language Pack, Esperanto Language Pack, Edit Cookies, Leo Search, Zulu Language Pack, Northern Soho Language Pack, Start Google Plus, Auto Shutdown NG

PASS 2012-03-05 PASS 2012-03-05 PASS 2012-03-05
Firefox 7.*

Banco do Brasil, Adicional de Seguranca, Fire DIff, Fire Tray, Fire Bomb, Sinafox, Zoom Fox, UrlBarExt, Metal + Glass, Fantapper

PASS 2012-03-05 PASS 2012-03-05 PASS 2012-03-05
Firefox 6.*

Screen Grab, Capture Fox, Maximum Adblock, Smart Video for Youtube, aFewClicksAway, Mozilla Labs Lab Kit, Image Fit Plus7Fox, Validate Me, MatchBox

PASS 2012-03-05 PASS 2012-03-05 PASS 2012-03-05
Firefox 5.*

kwi Click, Multirow Bookmarks Toolbar, Glue Book Recommendations, Google Sharing, Google Bar, ErrorZilla Plus, Facebook Blocker, Murdoch Alert, Cookie Safe, ESPN Cricinfo

PASS 2012-03-05 PASS 2012-03-05 PASS 2012-03-05
Firefox 4.*

Fast Dial, Price Gong, Irvine Context Menu, Greased Lightbox, SEO Toolbar, SEOpen, Clip to DEVONthink, ViewInFirefox, Native XP, Switch Hosts

PASS 2012-03-05 PASS 2012-03-05 PASS 2012-03-05
Manual (Beta)
  • Manual Updates
    • Install a previous version of Firefox
    • Install an add-on which is max-compatible with that version (ie. not compatible with Firefox 11.0)
    • Set update channel to beta
    • Manually check for and install an update
  • Repeat the above for Automatic Update after setting these preferences:
    • Set app.update.log = true
    • Set app.update.interval = 10
    • Set app.update.idletime = 0
    • Set app.update.download.backgroundInterval = 10
    • Restart and open Error Console
    • Once the update is completed downloading in the background, restart Firefox
  • EXPECTED:
    • You should not be prompted to enable/install any add-ons
    • You should successfully update to Firefox 11.0b5
    • Your addons should be installed and enabled in the Add-ons Manager
  • RESULTS:

Note - we will also run our full-suite DTC automation once we have RC updates (~ March 12)

Beta 6

Builds: HTTPS
Litmus: testrun, report

Checklist

  • [DONE] Functional automated tests, synced to Litmus
  • [DONE] Manual tests
  • [DONE] Bug fix verifications
  • [DONE] Updates on releasetest
  • [DONE] Peer sign-off
  • [DONE] Updates on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Result: report
Windows 2000 PASS - 2012-03-06
Windows XP PASS - 2012-03-06
Windows Vista PASS - 2012-03-06
Windows 7 32-bit PASS - 2012-03-06
Windows 7 64-bit PASS - 2012-03-06
Mac OS X 10.6 PASS - 2012-03-06
Mac OS X 10.7 PASS - 2012-03-06
Ubuntu 11.10 32-bit PASS - 2012-03-06
Ubuntu 11.10 64-bit PASS - 2012-03-06
Test Failures

Manual Tests

Smoketests

The following are the detailed results of the manual Smoketests tests.

Platform Assignee Result: report
Windows 2000 Simona PASS - 2012-03-06
Windows XP Paul PASS - 2012-03-06
Windows Vista Simona PASS - 2012-03-06
Windows 7 32-bit Mihaela PASS - 2012-03-06
Windows 7 64-bit Paul PASS - 2012-03-06
Mac OS X 10.6.8 VladG PASS - 2012-03-06
Mac OS X 10.7.3 VladG PASS - 2012-03-06
Ubuntu 11.10 32-bit Ioana PASS - 2012-03-07
Ubuntu 11.10 64-bit Virgil PASS - 2012-03-06

Focus Tests

The following are the detailed results of the focused testing and running the regression tests. Provide much detail about your testing: test cases, steps, bugs filed, URLs, add-on/plugin versions, platforms tested, etc

Platform Assignee Result: report
Windows 7 32-bit Mihaela PASS - 2012-03-06
Mac OS X 10.6.8 VladG PASS - 2012-03-07
Ubuntu 11.10 64-bit Virgil PASS - 2012-03-06
Windows XP Paul PASS - 2012-03-06

NOTE: Prioritize Windows 7, Mac OS X 10.6, and Ubuntu 32-bit -- do other platforms only as time permits

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

  • Any fix you very should have the status-firefox11 flag set to verified
  • If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
  • Softvision verifications spreadsheet
Queries
  • QA+: non-security bugs needing verification (9 / 10 VERIFIED)
  • QA+ SG: security bugs which need verification (15 / 36 VERIFIED)
  • status-firefox11:fixed: bugs needing triage for QA+ (49 / 49 TRIAGED)
  • UNCO: unconfirmed bugs filed during Firefox 11 (0 / 38 TRIAGED)

Update Tests

The following are details of update verifications

Note: bug 717214 disables new updates for 9.0b5 and earlier -- they update to 11.0b2 then get 11.0b2->11.0b4. This is a temporary stop-gap to ease burden on RelEng resources.

Platform Releasetest: report Beta: report
Windows 2000 PASS - 2012-03-06 PASS - 2012-03-07
Windows XP PASS - 2012-03-06 PASS - 2012-03-07
Windows Vista PASS - 2012-03-06 PASS - 2012-03-07
Windows 7 32-bit PASS - 2012-03-06 PASS - 2012-03-07
Windows 7 64-bit PASS - 2012-03-06 PASS - 2012-03-07
Mac OS X 10.6 PASS - 2012-03-06 PASS - 2012-03-07
Mac OS 10.7 PASS - 2012-03-06 PASS - 2012-03-07
Ubuntu 11.10 32-bit PASS - 2012-03-06 PASS - 2012-03-07
Ubuntu 11.10 64-bit PASS - 2012-03-06 PASS - 2012-03-07
Issues

Issues

New
Bug Status
QA Investigating (QA-WANTED)
Bug Status
bug 733709 - Lost all installed add-ons during a restart of Firefox Henrik is investigating
bug 701404 - Silverlight crashiness using Bing Video Vlad investigating possible regression in Silverlight
bug 716391 - Crash @je_free xul.dll@0xbb...f Marcia investigating possible malware
bug 723444 - Angry Birds is running jerky Paul investigating if 11.0b6 is better than 11.0b1
Engineering Investigating
Bug Status
bug 715401 - Crash @nsCanvasRenderingContext2DAzure::ClearRect (Yahoo Toolbar) testcase needed
bug 671468 - Crash @nsSocketOutputStream::Write (Yandex) Honza is investigating
bug 711656 - Crash in TextStageManager::AddStagesForSubrect w/Intel GMA4500HD Benoit Jacob is investigating
bug 731720 - Hang logging into Evernote Alex Keybl is investigating
bug 656758 - Crash @nsSocketInputStream::Read (nsUrlClassifierDBService) Josh Aas is investigating
bug 720679 - Crash @ mozilla::dom::workers::WorkerPrivate::CancelAllTimeouts Gregory Szorc is investigating
Followup Needed
Bug Status

Beta 7

Builds: HTTPS
Litmus: testrun, report

Checklist

  • {{ |}} Functional automated tests, synced to Litmus
  • {{ |}} Manual tests
  • {{ |}} Bug fix verifications
  • {{ |}} Updates on releasetest
  • {{ |}} Peer sign-off
  • {{ |}} Updates on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Result: report
Windows 2000 PASS - 2012-03-08
Windows XP PASS - 2012-03-08
Windows Vista PASS - 2012-03-08
Windows 7 32-bit PASS - 2012-03-08
Windows 7 64-bit PASS - 2012-03-08
Mac OS X 10.6 PASS - 2012-03-08
Mac OS X 10.7 PASS - 2012-03-08
Ubuntu 11.10 32-bit PASS - 2012-03-08
Ubuntu 11.10 64-bit PASS - 2012-03-08
Test Failures

Manual Tests

Smoketests

The following are the detailed results of the manual Smoketests tests.

Platform Assignee Result: report
Windows 2000 Simona PASS - 2012-03-09
Windows XP Ioana PASS - 2012-03-09
Windows Vista Simona PASS - 2012-03-09
Windows 7 32-bit Mihaela PASS - 2012-03-09
Windows 7 64-bit Mihaela PASS - 2012-03-09
Mac OS X 10.6.8 Vlad PASS - 2012-03-09
Mac OS X 10.7.3 Vlad PASS - 2012-03-09
Ubuntu 11.10 32-bit Ioana PASS - 2012-03-09
Ubuntu 11.10 64-bit Virgil PASS - 2012-03-09

Blocklist Tests

Bugs
  • bug 715401 – Firefox Crash @ nsCanvasRenderingContext2DAzure::ClearRect with Yahoo Toolbar
  • bug 711656 – Firefox 11.0a1 Crash in TextStageManager::AddStagesForSubrect with Intel GMA X4500HD, 4500MHD, and HD Graphics up to 8.15.10.2141
Summary

The current thinking is that the above two crashes are being caused by improper blocklisting of drivers. In other words, some old versions are *not* being blocklisted when indeed they should.

What to Test
  1. Install one of the old drivers
  2. In Firefox 10.0.2, verify blocklisted in about:support ("Direct2d Enabled" status)
  3. In Firefox 11.0b7, verify blocklisted in about:support ("Direct2d Enabled" status)
  4. Install Yahoo Toolbar
  5. Try to trigger a crash by playing around with websites using canvas

NOTE: Use these try-server builds only if there are no 11.0b7-candidate builds available

Results
Driver GPU Blocklisted? Crash?
Example
7.14.1.1134 VMware SVGA 3D (Microsoft Corporation - WDDM) Fx10: YES, Fx11: NO NO
Intel
8.15.10.1749 Blocked for your graphics card because of unresolved driver issues. Fx10: YES, Fx11: YES NO
8.15.10.1930 Blocked for your graphics card because of unresolved driver issues. Fx10: YES, Fx11: YES NO
ATI
NVidia
8.17.11.9539 0/1. Blocked for your graphics card because of unresolved driver issues. (NVidia GeForce 210) Fx10: YES, Fx11: YES NO
8.17.12.7533 1/1 Direct3D 10 Fx10: NO, Fx11: NO NO

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

  • Any fix you very should have the status-firefox11 flag set to verified
  • If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
  • Softvision verifications spreadsheet
Queries
  • QA+: non-security bugs needing verification (0 / 1 VERIFIED)
  • QA+ SG: security bugs which need verification (0 / 13 VERIFIED)
  • status-firefox11:fixed: bugs needing triage for QA+ (7 / 7 TRIAGED)
  • UNCO: unconfirmed bugs filed during Firefox 11 (0 / 39 TRIAGED)

Update Tests

The following are details of update verifications

Note: bug 717214 disables new updates for 9.0b6 and earlier -- they update to 11.0b2 then get 11.0b2->11.0b7. This is a temporary stop-gap to ease burden on RelEng resources.

Platform Releasetest: report Beta: report
Windows 2000 PASS - 2012-03-09 PASS - 2012-03-09
Windows XP PASS - 2012-03-09 PASS - 2012-03-09
Windows Vista PASS - 2012-03-09 PASS - 2012-03-09
Windows 7 32-bit PASS - 2012-03-09 PASS - 2012-03-09
Windows 7 64-bit PASS - 2012-03-09 PASS - 2012-03-09
Mac OS X 10.6 PASS - 2012-03-09 PASS - 2012-03-09
Mac OS 10.7 PASS - 2012-03-09 PASS - 2012-03-09
Ubuntu 11.10 32-bit PASS - 2012-03-09 PASS - 2012-03-09
Ubuntu 11.10 64-bit PASS - 2012-03-09 PASS - 2012-03-09
Issues
  • Some failures due to slow mirrors

Issues

New
Bug Status

Beta 8

RESPIN FOR bug 711656

Builds: HTTPS
Litmus: testrun, report

Checklist

  • {{ |}} Functional automated tests, synced to Litmus
  • {{ |}} Manual tests
  • {{ |}} Bug fix verifications
  • {{ |}} Updates on releasetest
  • {{ |}} Peer sign-off
  • {{ |}} Updates on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Result: report
Windows 2000 PASS - 2012-03-10
Windows XP PASS - 2012-03-10
Windows Vista PASS - 2012-03-10
Windows 7 32-bit PASS - 2012-03-10
Windows 7 64-bit PASS - 2012-03-10
Mac OS X 10.6 PASS - 2012-03-10
Mac OS X 10.7 PASS - 2012-03-10
Ubuntu 11.10 32-bit PASS - 2012-03-10
Ubuntu 11.10 64-bit PASS - 2012-03-10
Test Failures

Manual Tests

Smoketests

The following are the detailed results of the manual Smoketests tests.

Platform Assignee Result: report
Windows 2000 ashughes PASS - 2012-03-10
Windows XP ashughes PASS - 2012-03-10
Windows Vista ashughes PASS - 2012-03-10
Windows 7 32-bit jsmith PASS - 2012-03-11
Windows 7 64-bit jsmith PASS - 2012-03-11
Mac OS X 10.6.8 marcia PASS - 2012-03-10
Mac OS X 10.7.3 ashughes PASS - 2012-03-10
Ubuntu 11.10 32-bit ashughes PASS - 2012-03-10
Ubuntu 11.10 64-bit ashughes PASS - 2012-03-10

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

  • Any fix you very should have the status-firefox11 flag set to verified
  • If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
  • Softvision verifications spreadsheet
Queries
  • QA+: non-security bugs needing verification (0 / 1 VERIFIED)
  • QA+ SG: security bugs which need verification (0 / 13 VERIFIED)
  • status-firefox11:fixed: bugs needing triage for QA+ (7 / 7 TRIAGED)
  • UNCO: unconfirmed bugs filed during Firefox 11 (0 / 39 TRIAGED)

Update Tests

The following are details of update verifications

Note: bug 717214 disables new updates for 9.0b6 and earlier -- they update to 11.0b2 then get 11.0b2->11.0b7. This is a temporary stop-gap to ease burden on RelEng resources.

Platform Releasetest: report Beta: report
Windows 2000 PASS - 2012-03-11 PASS - 2012-03-11
Windows XP PASS - 2012-03-11 PASS - 2012-03-11
Windows Vista PASS - 2012-03-11 PASS - 2012-03-11
Windows 7 32-bit PASS - 2012-03-11 PASS - 2012-03-11
Windows 7 64-bit PASS - 2012-03-11 PASS - 2012-03-11
Mac OS X 10.6 PASS - 2012-03-11 PASS - 2012-03-11
Mac OS 10.7 PASS - 2012-03-11 PASS - 2012-03-11
Ubuntu 11.10 32-bit PASS - 2012-03-11 PASS - 2012-03-11
Ubuntu 11.10 64-bit PASS - 2012-03-11 PASS - 2012-03-11
Issues

Issues

Full Query
ID Summary Priority Status
734721 Full-screen HTML 5 Video produces short random freezes when watching nature by numbers in Win 7 64-bit -- RESOLVED
734723 Printing http://www.mozilla.org/en-US/firefox/fx/ is missing many parts of the page (color, images, etc) -- RESOLVED
734725 Uninstalling Firefox leaves behind some folders and files in program files location where it was installed -- RESOLVED

3 Total; 0 Open (0%); 3 Resolved (100%); 0 Verified (0%);


RC

SCRAPPED DUE TO bug 711656, SEE RC build 2

Builds: HTTPS
Litmus: testrun, report

Checklist

  • {{ |}} Functional automated tests, synced to Litmus
  • {{ |}} Manual tests
  • {{ |}} Bug fix verifications
  • {{ |}} Updates on releasetest
  • {{ |}} Peer sign-off
  • {{ |}} Updates on beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Result: report
Windows 2000 PASS - 2012-03-09
Windows XP PASS - 2012-03-09
Windows Vista PASS - 2012-03-09
Windows 7 32-bit PASS - 2012-03-09
Windows 7 64-bit PASS - 2012-03-09
Mac OS X 10.6 PASS - 2012-03-09
Mac OS X 10.7 PASS - 2012-03-09
Ubuntu 11.10 32-bit PASS - 2012-03-09
Ubuntu 11.10 64-bit PASS - 2012-03-09
Test Failures

Manual Tests

Smoketests

The following are the detailed results of the manual Smoketests tests.

Platform Assignee Result: report
Windows 2000 tester result - YYYY-MM-DD
Windows XP tester result - YYYY-MM-DD
Windows Vista tester result - YYYY-MM-DD
Windows 7 32-bit jsmith result - YYYY-MM-DD
Windows 7 64-bit jsmith result - YYYY-MM-DD
Mac OS X 10.6.8 marcia result - YYYY-MM-DD
Mac OS X 10.7.3 tester result - YYYY-MM-DD
Ubuntu 11.10 32-bit tester result - YYYY-MM-DD
Ubuntu 11.10 64-bit ashughes result - YYYY-MM-DD

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

  • Any fix you very should have the status-firefox11 flag set to verified
  • If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
  • Softvision verifications spreadsheet
Queries
  • QA+: non-security bugs needing verification (0 / 1 VERIFIED)
  • QA+ SG: security bugs which need verification (0 / 13 VERIFIED)
  • status-firefox11:fixed: bugs needing triage for QA+ (7 / 7 TRIAGED)
  • UNCO: unconfirmed bugs filed during Firefox 11 (0 / 39 TRIAGED)

Update Tests

The following are details of update verifications

Note: bug 717214 disables new updates for 9.0b6 and earlier -- they update to 11.0b2 then get 11.0b2->11.0b7. This is a temporary stop-gap to ease burden on RelEng resources.

Platform Releasetest: [ report] Beta: [ report]
Windows 2000 result - YYYY-MM-DD result - YYYY-MM-DD
Windows XP result - YYYY-MM-DD result - YYYY-MM-DD
Windows Vista result - YYYY-MM-DD result - YYYY-MM-DD
Windows 7 32-bit result - YYYY-MM-DD result - YYYY-MM-DD
Windows 7 64-bit result - YYYY-MM-DD result - YYYY-MM-DD
Mac OS X 10.6 result - YYYY-MM-DD result - YYYY-MM-DD
Mac OS 10.7 result - YYYY-MM-DD result - YYYY-MM-DD
Ubuntu 11.10 32-bit result - YYYY-MM-DD result - YYYY-MM-DD
Ubuntu 11.10 64-bit result - YYYY-MM-DD result - YYYY-MM-DD
Issues
  • Some failures due to slow mirrors

Issues

New
Bug Status
bug 734721 Full-screen HTML 5 Video produces short random freezes
Tracking
Bug Status
bug 715401 – Firefox Crash @ nsCanvasRenderingContext2DAzure::ClearRect with Yahoo Toolbar
bug 711656 – Firefox 11.0a1 Crash in TextStageManager::AddStagesForSubrect with Intel GMA X4500HD, 4500MHD, and HD Graphics up to 8.15.10.2141

RC2

RESPIN FOR bug 711656

Builds: HTTPS
Litmus: testrun, report

Checklist

  • [DONE] Functional automated tests, synced to Litmus
  • [DONE] Manual tests
  • [DONE] Bug fix verifications
  • [DONE] Updates on betatest
  • [DONE] Peer sign-off
  • [DONE] Updates on releasetest
  • [DONE] Updates on release
  • {{ |}} DTC testing (can only be run with release snippets)

Automated Tests

The following are the detailed results of the automated BFT test runs.

Platform Result: report
Windows 2000 PASS - 2012-03-12
Windows XP PASS - 2012-03-12
Windows Vista PASS - 2012-03-12
Windows 7 32-bit PASS - 2012-03-12
Windows 7 64-bit PASS - 2012-03-12
Mac OS X 10.6 PASS - 2012-03-12
Mac OS X 10.7 PASS - 2012-03-12
Ubuntu 11.10 32-bit PASS - 2012-03-12
Ubuntu 11.10 64-bit PASS - 2012-03-12
Test Failures

Manual Tests

The following are the detailed results of the manual Smoketests tests.

Platform Assignee Result: report
Windows 2000 Simona PASS - 2012-03-13
Windows XP Ioana PASS - 2012-03-13
Windows Vista Simona PASS - 2012-03-13
Windows 7 32-bit Mihaela PASS - 2012-03-13
Windows 7 64-bit Paul PASS - 2012-03-13
Mac OS X 10.6.8 Vlad PASS - 2012-03-13
Mac OS X 10.7.3 Vlad PASS - 2012-03-13
Ubuntu 11.10 32-bit Ioana PASS - 2012-03-13
Ubuntu 11.10 64-bit Virgil PASS - 2012-03-13

Focus Tests

The following are the detailed results of the focused testing and running the regression tests.

Platform Assignee Result: report
Windows 7 32-bit Mihaela PASS - 2012-03-13
Mac OS X 10.6.8 VladG PASS - 2012-03-13
Ubuntu 11.10 64-bit Virgil PASS - 2012-03-13

Bug Fix Verifications

The following are the detailed results of verifications of bug fixes landed in Beta.

  • Any fix you very should have the status-firefox11 flag set to verified
  • If the fix has been verified on all affected versions of Firefox, change [qa+] in the whiteboard to [qa!]
  • Softvision verifications spreadsheet
Queries
  • QA+: non-security bugs needing verification (0 / 1 VERIFIED)
  • QA+ SG: security bugs which need verification (0 / 13 VERIFIED)
  • status-firefox11:fixed: bugs needing triage for QA+ (7 / 7 TRIAGED)
  • UNCO: unconfirmed bugs filed during Firefox 11 (0 / 39 TRIAGED)

Update Tests

The following are details of update verifications

Platform Betatest: report Releasetest: report Release: report
Windows 2000 PASS - 2012-03-12 PASS - 2012-03-13 PASS - 2012-03-13
Windows XP PASS - 2012-03-12 PASS - 2012-03-13 PASS - 2012-03-13
Windows Vista PASS - 2012-03-12 PASS - 2012-03-13 PASS - 2012-03-13
Windows 7 32-bit PASS - 2012-03-12 PASS - 2012-03-13 PASS - 2012-03-13
Windows 7 64-bit PASS - 2012-03-12 PASS - 2012-03-13 PASS - 2012-03-13
Mac OS X 10.6 PASS - 2012-03-12 PASS - 2012-03-13 PASS - 2012-03-13
Mac OS 10.7 PASS - 2012-03-12 PASS - 2012-03-13 PASS - 2012-03-13
Ubuntu 11.10 32-bit PASS - 2012-03-12 PASS - 2012-03-13 PASS - 2012-03-13
Ubuntu 11.10 64-bit PASS - 2012-03-12 PASS - 2012-03-13 PASS - 2012-03-13
Issues

Default to Compatible Addons

Pre-update Endurance Testrun (Memory) reports
  • Mac: 199 - 295 MB
  • Win XP: 127 - 270 MB
  • Win 7: 132 - 248 MB
Post-update Endurance Testrun (Memory) reports
  • Mac: 210 - 219 MB
  • Win XP: 99 - 148 MB
  • Win 7: 125 - 157 MB
Add-on Incompatibilities reports
  • [Addon] Яндекс.Бар 4.2.2 (disabled, incompatible)
  • [Addon] Text Link 4.0.2011021601 (disabled, incompatible)
  • [Addon] United States English Spellchecker 5.0.1 (disabled, incompatible)
  • [Addon] PriceGong 2.2.0 (disabled, incompatible)
  • [Addon] IE Tab 1.5.20090525 (disabled, incompatible)
  • [Addon] British English Dictionary 1.19.1 (disabled, incompatible)
  • [Addon] Google Toolbar for Firefox 7.1.20110512W (disabled, incompatible)
  • [Theme] NASA Night Launch 0.6.20120129 (disabled, compatible)
  • [Theme] Default 10.0.2 (disabled, compatible)

Issues

New
Bug Status
bug 734837 - Broken history when navigating to and from cnn.com QA investigating -- might be server-side issue
bug 735243 - On HTML5 videos Esc key exits from both full screen and Highlighter NEW