Releases/Firefox 20/Test Plan

From MozillaWiki
Jump to: navigation, search

« Firefox 19 | Firefox 20 | Firefox 21 »

Contents

Firefox 20 Test Plan

Summary

Milestone Firefox 20
Status Currently in Beta -- going to Release on April 2nd, 2013
Lead Juan Becerra (irc: juanb)
2nd Anthony Hughes (irc: ashughes)
Builds [ftp] [http] [web]

The following is the test plan for Firefox 20 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 19. After the release this document will be used as an archive of what was done to validate this release.

If you have some free time, please pick a task below and get in touch with one of the leads.

Details

The following is a description of the tasks which need to be completed throughout this release.

  • 2012-11-19: {{|}} Nightly Merge
  • 2013-01-07: {{|}} Aurora Merge
  • 2013-02-08: {{|}} Pre-Beta feature sign-offs
  • 2013-02-21: {{|}} Beta 1
  • 2013-03-01: {{|}} Beta 2
  • 2013-03-08: {{|}} Beta 3
  • 2013-03-15: {{|}} Beta 4
  • 2013-03-22: {{|}} Beta 5
  • 2013-03-27: {{|}} Beta 6
  • 2013-04-01: {{|}} Release Candidate
  • 2013-04-02: {{|}} Final

Nightly

Merge

Functional Automation - result
Update Spotchecks - result

Features

See the Feature Release Tracking wiki for a general overview

Features

See the Feature Release Tracking wiki for a general overview

Feature Owner Status Tests
[Desktop] Panel-based Download Manager Simona Badau On Track
[DevTools] JavaScript Profiling Mihaela Velimiroviciu On Track
[DevTools] Tools In Windows Simona Badau On Track

Endurance Automation

The purpose of this section is to track any potential memory regressions via spikes reported in the dashboard.

What to Do
  1. Check the Endurance Charts
  2. If you see an anomalous event (ie. spike or dip), get the report for the specific event
  3. If the event happens twice in a row, file a bug under Mozmill Tests
  4. Notify Anthony Hughes of the issue
  5. Record the event in this wiki below
Current Status (Mihaela Velimiroviciu) - Last Updated -
ID Summary Priority Status
826684 Increased memory usage on Mac 10.8.2 since Dec 26th 2012 - Nightly branch -- RESOLVED

1 Total; 0 Open (0%); 1 Resolved (100%); 0 Verified (0%);


Functional Automation

The purpose of this section is to track any potential functional regressions via results reported in the dashboard.

What to Do
  1. Check the testrun reports
  2. If you see any failing tests, check that there are not any bugs on file for it already
  3. If not, file a bug under Mozmill Tests
  4. Notify Anthony Hughes of the issue
  5. Record the event in this wiki below

Bug Triage

The purpose of this section is to triage UNCONFIRMED incoming bugs.

What to Do
  1. Check the UNCONFIRMED bugs
  2. Test the bugs in the latest Firefox Nightly
  3. If you can't reproduce the bug, ask for the reporter to provide more information
  4. If you reproduce the bug, add your testing information and mark the bug NEW
  5. Notify Anthony Hughes of the NEW issue
Current Status (SoftVision) - Last Updated

[Result Spreadsheet]

Aurora Merge

Builds nightly
Changeset tip (pushlog)
Checklist

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform PASS: report
Windows XP PASS - 2013-01-11
Windows Vista PASS - 2013-01-11
Windows 7 32-bit PASS - 2013-01-11
Windows 7 64-bit PASS - 2013-01-11
Windows 8 32-bit PASS - 2013-01-11
Windows 8 64-bit PASS - 2013-01-11
Mac OS X 10.6 PASS - 2013-01-11
Mac OS X 10.7 PASS - 2013-01-11
Mac OS X 10.8 PASS - 2013-01-11
Ubuntu 12.04 32-bit PASS - 2013-01-11
Ubuntu 12.04 64-bit PASS - 2013-01-11
Test Failures (known bugs)

Manual Testing

Operating System Tester Plug-in Compatibility Web Compatibility OSX 10.5 EOL
Windows XP tester PASS - 2013-01-11 PASS - 2013-01-11 PASS - 2013-01-11
Windows 7 tester PASS - 2013-01-11 PASS - 2013-01-11 PASS - 2013-01-11
Windows 8 tester PASS - 2013-01-11 PASS - 2013-01-11 PASS - 2013-01-11
Mac OS X tester PASS - 2013-01-11 PASS - 2013-01-11 PASS - 2013-01-11
Ubuntu 12.04 tester PASS - 2013-01-11 PASS - 2013-01-11 PASS - 2013-01-11

Plug-in Compatibility

Install the following plug-ins and go to websites which use them, looking for rendering issues, functional issues, performance issues, and crashes.

Web Compatibility

Test the following websites for rendering issues, functional issues, performance issues, and crashes

  1. Overall Top 10
  2. Top by Country (pick a random site from three random countries)
  3. Top by Category (pick a random site from three random categories)
  4. Webmail clients like gmail, hotmail, and yahoo mail (pay particular attention to any built in web messenger software)
  5. Try Google's MapsGL demo
  6. Try YouTube's HTML5 demo

Feature Sign-offs

  • Log in to moztrap.mozilla.org
  • Select Run Tests
  • Select Product: Desktop Firefox
  • Select Versions: 20
  • Select Runs: Firefox 20 Aurora Merge Feature sign off
Feature Owner Win32 Mac Linux
[Desktop] Panel-based Download Manager Simona Badau PASS - 2013-01-15 PASS - 2013-01-15 PASS - 2013-01-15
[DevTools] JavaScript Profiling Mihaela Velimiroviciu PASS - 2013-01-10 (Win7x64) PASS - 2013-01-10 PASS - 2013-01-10
[DevTools] Tools In Windows Simona Badau PASS - 2013-01-11 PASS - 2013-01-11 PASS - 2013-01-11
Details

Report any detailed results or bugs here...

Update Testing

Spotcheck updates from a previous Aurora to Aurora 20.

Platform Auroratest Aurora
Windows 7 64-bit PASS - 2013-01-11 result - yyyy-mm-dd
Windows 8 64-bit PASS - 2013-01-11 result - yyyy-mm-dd
Mac OS X 10.7 PASS - 2013-01-11 result - yyyy-mm-dd
Ubuntu 12.04 64-bit PASS - 2013-01-11 result - yyyy-mm-dd
Issues

Bug Triage

  1. VERIFYME (no of bugs): verify the fix and do some exploratory testing around the issue
  2. FIXED (no of bugs): triage these bugs and test to ensure the fix is effective
  3. QAWANTED (no of bugs): triage these bugs and test to fulfill the QA request
  4. UNCONFIRMED (no of bugs): triage these bugs for potential new regressions
Results

Pre-beta Feature testing and Sign-off

Feature testing

Run the Firefox 20.0 Pre-beta Feature Sign off tests in MozTrap and report results below.

Feature Owner Win 7 64bit Win 8 32bit Mac OSX 10.8 Ubuntu 12.04
[Desktop] Panel-based Download Manager Simona Badau PASS - 2013-02-18 PASS - 2013-02-15 PASS - 2013-02-19 PASS - 2013-02-15
[DevTools] JavaScript Profiling Mihaela Velimiroviciu PASS - 2013-02-18 PASS - 2013-02-18 PASS - 2013-02-18 PASS - 2013-02-18
[DevTools] Tools In Windows Simona Badau PASS - 2013-02-14 PASS - 2013-02-14 PASS - 2013-02-14 PASS - 2013-02-14
[Desktop] Per-window Private Browsing Ioana Budnar PASS - 2013-02-19 PASS - 2013-02-19 PASS - 2013-02-15 PASS - 2013-02-19
[Desktop] Windows Plugin Hang UI Manuela Muntean PASS - 2013-02-14 PASS - 2013-02-14 N/A - 2013-02-14 N/A - 2013-02-14
Performance Fixes Bogdan Maris PASS - 2013-02-14 PASS - 2013-02-14 PASS - 2013-02-14 PASS - 2013-02-14

Notes:

  • Windows_Plugin_Hang_UI
    • Not applicable on Mac OSX and Ubuntu platforms
    • All testing was made with the latest Flash version: 11.6.602.168
    • Also tested on the following OSs:
      • Windows XP 32-bit - PASS - 2013-02-14
      • Windows Vista 32-bit - PASS - 2013-02-14
      • Windows 7 32-bit - PASS - 2013-02-15
      • Windows 8 64-bit - PASS - 2013-02-15
    • Concerns:
      • bug 834127 - should be fixed before the release of Firefox 20
  • Developer Tools Window
    • Also tested on:
    • Windows XP - PASS - 2013-02-14

Smoke tests

Run the smoketests in MozTrap and report results below.

Platform Assignee Result:
Windows XP tester RESULTS - yyyy-mm-dd
Windows 7 64-bit tester RESULTS - yyyy-mm-dd
Mac OS X 10.7.4 tester RESULTS - yyyy-mm-dd
Ubuntu 12.04 32-bit tester RESULTS - yyyy-mm-dd

New Issues

Bugzilla query error

Query options must be valid json


Known Issues

Bugzilla query error

Query options must be valid json


Beta Checklist

The following should be checked at least once during the 6-week Beta cycle across all platforms.

  • Add-on Compatibility: Top-10 AMO addons, top-10 third-party add-ons, toolbar add-ons (yandex, avg, norton, yahoo, bandoo, conduit, etc)
  • Plugin Compatibility: Flash (current & beta), Java (6 & 7), Silverlight 5, Realplayer, Quicktime, Shockwave
    • Explicit Testing:
  • Web Compatibility: search, webmail, banking, video, audio, gaming, news, cms (drupal, moodle, wordpress, etc)
  • Testday: at least one Beta testday to cover new features, bug triage, compatibility testing, and general feedback

The following checklist should be completed before signing off for Release.

Beta 1
  • {{ |}} Breakpad End-to-End functionality
  • {{ |}} Mac OSX 10.8 Signing
  • {{ |}} Add-on compatibility (top AMO, top third-party, toolbars, yandex, avg, norton, yahoo, bandoo, conduit, etc)
  • {{ |}} Exploratory testing and bug triage against the flagship feature
  • Platforms: Windows 7 32-bit, Windows 8 64-bit, Mac OSX 10.8, Ubuntu 12.04 32-bit
Beta 2
  • {{ |}} Breakpad End-to-End functionality
  • {{ |}} Stop accepting MD5 as a hash algorithm in signatures
  • {{ |}} Plug-in compatibility (Flash, Java, Silverlight, Realplayer, Quicktime, Shockwave, Unity [Mac])
  • {{ |}} Firefox Beta testday stressing flagship features and bug triage
  • {{ |}} Exploratory testing and bug triage against the flagship feature
  • Platforms: Windows 7 64-bit, Windows 8 32-bit, Mac OSX 10.7, Ubuntu 12.04 64-bit
Beta 3
  • {{ |}} Breakpad End-to-End functionality
  • {{ |}} Mac OSX 32-bit mode
  • {{ |}} Web compatibility (search, webmail, banking, video, audio, gaming, news, cms, etc.)
  • {{ |}} Exploratory testing and bug triage against the flagship feature
  • Platforms: Windows 7 32-bit, Windows Vista, Mac OSX 10.6, Ubuntu 12.04 32-bit
Beta 4
  • {{ |}} Breakpad End-to-End functionality
  • {{ |}} Verify update applies with root owned files
  • {{ |}} Add-on compatibility (top AMO, top third-party, toolbars, yandex, avg, norton, yahoo, bandoo, conduit, etc)
  • {{ |}} Exploratory testing and bug triage against the flagship feature
  • {{ |}} Firefox Beta testday stressing flagship features and bug triage
  • Platforms: Windows 7 64-bit, Windows XP, Mac OSX 10.8, Ubuntu 12.10 32-bit
Beta 5
  • {{ |}} Breakpad End-to-End functionality
  • {{ |}} What's New Page on Firefox update
  • {{ |}} Plug-in compatibility (Flash, Java, Silverlight, Realplayer, Quicktime, Shockwave, Unity [Mac])
  • {{ |}} Exploratory testing and bug triage against the flagship feature
  • Platforms: Windows 7 32-bit, Windows 8 64-bit, Mac OSX 10.7, Ubuntu 12.04 32-bit
Beta 6
  • {{ |}} Breakpad End-to-End functionality
  • {{ |}} Mac OSX 10.5 and Windows 2000 de-support (install and update)
  • {{ |}} Web compatibility (search, webmail, banking, video, audio, gaming, news, cms, etc.)
  • {{ |}} Exploratory testing and bug triage against the flagship feature
  • Platforms: Windows 7 64-bit, Windows 8 32-bit, Mac OSX 10.6, Ubuntu 12.04 64-bit, Mac OSX 10.5, Windows 2000


New Issues

Bugzilla query error

Query options must be valid json


Known Issues

Bugzilla query error

Query options must be valid json


Beta 1

Builds FTP, HTTP
ID 20130220104816
Changeset 016db7820856
Checklist
  • [DONE] Functional Automation
  • [DONE] Manual Testing
  • [DONE] Bug Triage
  • [DONE] Update Automation on betatest channel
  • [DONE] Update Automation on releasetest channel
  • [DONE] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform Functional Results Restart Results
Windows XP [DONE] 0 Fails [DONE] 0 Fails
Windows Vista [DONE] 0 Fails [DONE] 0 Fails
Windows 7 32-bit [DONE] 0 Fails [DONE] 0 Fails
Windows 7 64-bit [DONE] 0 Fails [DONE] 0 Fails
Windows 8 32-bit [DONE] 0 Fails [DONE] 0 Fails
Windows 8 64-bit [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.6 [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.7 [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.8 [DONE] 0 Fails [DONE] 0 Fails
Ubuntu 12.04 32-bit [DONE] 0 Fails [DONE] 0 Fails
Ubuntu 12.04 64-bit [DONE] 0 Fails [DONE] 0 Fails
Test Failures (known bugs)

Manual Testing

Report any bugs found during exploratory testing in Bugzilla and add them to the Issues Found section below.

Focus Area Windows 7 32bit Windows 8 64bit Mac OSX 10.8 Ubuntu 12.04 32bit
Regression Tests:
Run the Firefox 20.0b1 Regression Tests in MozTrap. Do some exploratory testing around the testcases to try to exercise some edge-cases.
[DONE]
Bogdan
0 Bugs
[DONE]
Alexandra
0 Bugs
[DONE]
Cornel, Simona
0 Bugs
[DONE]
Manuela
0 Bugs
Add-on Compatibility:
Test the top add-ons for stability issues and functional issues. Test featured add-ons from AMO and Third-party toolbars like Yahoo, Yandex, AVG, Amazon, Norton, etc.
[DONE]
Bogdan
0 Bugs
[DONE]
Alexandra
0 Bugs
[DONE]
Cornel
0 Bugs
[DONE]
Manuela
0 Bugs
[DevTools] Tools In Windows:
Do some exploratory testing and Bug Triage
[DONE]
Bogdan
0 Bugs
[DONE]
Alexandra
0 Bugs
[DONE]
Cornel
0 Bugs
[DONE]
Manuela
0 Bugs
feature 2" [ON TRACK]
tester
X Bugs
[ON TRACK]
tester
X Bugs
[ON TRACK]
tester
X Bugs
[ON TRACK]
tester
X Bugs
feature ..." [ON TRACK]
tester
X Bugs
[ON TRACK]
tester
X Bugs
[ON TRACK]
tester
X Bugs
[ON TRACK]
tester
X Bugs

Update Testing

Results of running the Mozmill update automation.
20.0b1 should get a partial update, <= 19.0b6 should get a complete update.

Platform Betatest (2013-02-22) Releasetest (2013-02-22) Beta (2013-02-22)
Windows XP [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails
Windows Vista [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails
Windows 7 32-bit [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails
Windows 7 64-bit [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails
Windows 8 32-bit [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails
Windows 8 64-bit [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails
Mac OS X 10.6 [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails
Mac OS X 10.7 [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails
Mac OS X 10.8 [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails
Ubuntu 12.04 32-bit [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails
Ubuntu 12.04 64-bit [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails

Bug Triage

QAWANTED

Bugs: Test to find steps to reproduce, regression range, or testcases Status
  • [DONE] bug 829568 Starting a private window from the Firefox icon pinned to the taskbar wipes your regular session (Ioana)

NORMAL FIXES

Bugs: Verify these fixes in the latest Beta candidate Status
  • [DONE] VERIFIED bug 816603 background-color does not print in certain site (Ioana)
  • [DONE] VERIFIED bug 797288 Pdf.js download behavior inconsistent (Virgil)
  • [DONE] VERIFIED bug 813851 Canvas gets confused about its path/transform state (Virgil)
  • [DONE] VERIFIED bug 837845 Multi-GB memory spike when using regular expressions (Virgil)
  • [DONE] DROPPED bug 824162 unable to focus into content in chat windows, iframe receives focus (Virgil)
  • [DONE] VERIFIED bug 819377 Hang on https://www.mozilla.org/en-US/contribute/ when playing video (Manuela)
  • [DONE] COMMENTED bug 824301 cnn.com - Unresponsive (after changeset ea373e534245) due to Optimizely not restoring setters (Manuela)
  • [DONE] VERIFIED bug 829435 Japanese characters appear completely garbled in PDF viewer (Manuela)
  • [DONE] VERIFIED bug 827035 Etherpad won't load in 20.0a1 (2013-01-05) (Manuela)
  • [DONE] VERIFIED bug 830278 printing selections in pdf.js don't work and can cause OOM errors in large PDFs (Simona)
  • [DONE] VERIFIED bug 815359 Bing Maps v7 Ajax Pushpin Incompatibility (Simona)
  • [DONE] VERIFIED bug 836875 Background is no longer painted in Print output/Print preview (Simona)
  • [DONE] VERIFIED bug 834075 pdf.js: The PDF url in the address bar becomes wrong in a certain situation (Alexandra)
  • [DONE] VERIFIED bug 827954 Firefox button overlaps the first tab in private windows (Bogdan)
  • [DONE] COMMENTED bug 825620 Regression: Flash is triggering the discrete GPU (Bogdan)
  • [DONE] VERIFIED bug 827979 add Growl support back (Cornel)
  • [DONE] VERIFIED bug 831780 scrollWidth, scrollHeight of DIV containing IMG reported as 40x40px until the image's actual size is known, regardless of CSS sizing of the IMG (Simona)
  • [DONE] VERIFIED bug 828184 startup crash in nsPrefBranch::GetIntPref with QIPCAP.dll (Websense Endpoint) (Ioana)
  • [DONE] VERIFIED bug 803568 Nightly crashed when see pdf file in fullscreen (presentation mode) or zooming (Ioana)

SECURITY FIXES

Bugs: Verify these fixes in the latest Beta candidate (requires privileged access) Status

TOPCRASH FIXES

Verify these signatures are no longer present on crashstats in the current Beta

  • [DONE] VERIFIED bug 806206 crash in JSObject::global (Ioana)
  • [DONE] VERIFIED bug 827217 webgl.can-lose-context-in-foreground=false crashes in desktop Firefox (Ioana)

UNCONFIRMED

Bugs: Triage unconfirmed bugs for potential incoming regressions. Status

Issues Found

Bugs: New issues found during testing the Beta candidate Status

Items in bold are BLOCKER bugs

Beta 2

Builds FTP, HTTP
ID 20130227063501
Changeset f1192bc4c66a (pushlog)
Checklist
  • [DONE] Functional Automation
  • [DONE] Manual Testing
  • [DONE] Bug Triage
  • [DONE] Update Automation on betatest channel
  • [DONE] Update Automation on releasetest channel
  • {{|}} Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform Functional Results Restart Results
Windows XP [DONE] 0 Fails [DONE] 0 Fails
Windows Vista [DONE] 0 Fails [DONE] 0 Fails
Windows 7 32-bit [DONE] 0 Fails [DONE] 0 Fails
Windows 7 64-bit [DONE] 0 Fails [DONE] 0 Fails
Windows 8 32-bit [DONE] 0 Fails [DONE] 0 Fails
Windows 8 64-bit [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.6 [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.7 [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.8 [DONE] 0 Fails [DONE] 0 Fails
Ubuntu 12.04 32-bit [DONE] 0 Fails [DONE] 0 Fails
Ubuntu 12.04 64-bit [DONE] 0 Fails [DONE] 0 Fails
Test Failures (known bugs)

Manual Testing

Report any bugs found during exploratory testing in Bugzilla and add them to the Issues Found section below.

Focus Area Windows 7 32bit Windows 8 64bit Mac OSX 10.8 Ubuntu 12.04 32bit
Regression Tests:
Run the Firefox 20.0b2 Regression Tests in MozTrap. Do some exploratory testing around the testcases to try to exercise some edge-cases.
[DONE]
Cornel
0 Bugs
[DONE]
Manuela
0 Bugs
[DONE]
Alexandra
0 Bugs
[DONE]
Simona
0 Bugs
Plug-in compatibility
Test Flash, Java, Silverlight, Realplayer, Quicktime, Shockwave, Unity [Mac] for functional and stability issues.
[DONE]
Cornel
1 Bugs
[DONE]
Manuela
1 Bugs
[DONE]
Alexandra
1 Bugs
[DONE]
Simona
0 Bugs
Per-window Private Browsing
Exploratory testing taking into account bugs 841359, 832325, and 840739.
[DONE]
Cornel
0 Bugs
[DONE]
Manuela
0 Bugs
[DONE]
Alexandra
0 Bugs
[DONE]
Simona
0 Bugs

Update Testing

Results of running the Mozmill update automation.
20.0b1 should get a partial update, <= 19.0b6 should get a complete update.

Platform Betatest (2013-02-27) Releasetest (2013-02-27) Beta (2013-02-27)
Windows XP [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails
Windows Vista [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails
Windows 7 32-bit [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails
Windows 7 64-bit [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails
Windows 8 32-bit [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails
Windows 8 64-bit [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails
Mac OS X 10.6 [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails
Mac OS X 10.7 [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails
Mac OS X 10.8 [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails
Ubuntu 12.04 32-bit [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails
Ubuntu 12.04 64-bit [DONE] : 0 Fails [DONE] : 0 Fails [DONE] : 0 Fails

Bug Triage

QAWANTED

Bugs: Test to find steps to reproduce, regression range, or testcases Status

NORMAL FIXES

Bugs: Verify these fixes in the latest Beta candidate Status
  • [DONE] VERIFIED bug 816498 Table backgrounds don't print, starting from FX17 (Bogdan)
  • [DONE] VERIFIED bug 829568 Starting a private window from the Firefox icon pinned to the taskbar wipes your regular session (Ioana)
  • [DONE] VERIFIED bug 821775 {inc} After horizontal window-resize in quirks mode, vertically-stretched nested flex container is no longer stretched (Paul)
  • [DONE] VERIFIED bug 829886 Can't select options in a select list whose parent is translated with a transform on mac (Paul)
  • [DONE] VERIFIED bug 841359 Avoid logging script errors coming from private windows in the global error console (Ioana)
  • [DONE] VERIFIED bug 842577 Remove from History is visible for a paused download (Mihaela)
  • [DONE] VERIFIED bug 843190 Camera/mics can't be shared in getUserMedia if the permissions UI is disabled (Mihaela)
  • [DONE] UPDATED bug 844007 Firefox 19 : cross-domain XmlHttpRequest.responseText in chinese (Mihaela)
    • Need further info
  • [DONE] VERIFIED bug 842650 Disable Profiler Preference in Firefox 20 (Mihaela)
  • [DONE] VERIFIED bug 820379 crash in mozilla::dom::CanvasRenderingContext2D::GetImageDataArray (Bogdan)
  • [DONE] VERIFIED bug 832325 Undo close tab no longer works in permanent Private Browsing mode (“Never remember history”) (Ioana)
  • [DONE] VERIFIED bug 840739 Flash links in private windows are converting normal windows into private windows. (Bogdan)
  • [DONE] VERIFIED bug 829921 click events not delivered with trivial CSS transform, margin, overflow: hidden (Ioana)
  • [DONE] COMMENTED bug 842589 pdf.js missing character in Firefox19 and 20.0a2 with HWA enabled (Manuela)
  • [DONE] VERIFIED bug 793972 Details button on crashreporter dialogue doesn't work (Manuela)

SECURITY FIXES

Bugs: Verify these fixes in the latest Beta candidate (requires privileged access) Status
  • [DONE] VERIFIED bug 803870 XSS using timed document navigations (Ioana)
  • [DONE] UPDATED bug 784730 "ASSERTION: wrapper already in new scope" with canvas, adoptNode (Ioana)
    • Couldn't reproduce on pre-fix builds -> cannot be verified reliably.

TOPCRASH FIXES

Verify these signatures are no longer present on crashstats in the current Beta

  • [DONE] VERIFIED bug 829909 crash in nsWindow::Enable @ UserCallWinProcCheckWow (Manuela)
  • [DONE] VERIFIED bug 828034 crash in mozilla::ipc::RPCChannel::EnteredCxxStack (Manuela)
  • [DONE] VERIFIED bug 806206 crash in JSObject::global (Ioana)
  • [DONE] VERIFIED bug 819888 crash in nsWindow::DealWithPopups (Paul)
  • [DONE] VERIFIED bug 798446 startup crash in nsGlobalWindow::nsGlobalWindow (Paul)
  • [DONE] VERIFIED bug 818962 crash in nsHttpConnectionMgr::ProcessPendingQForEntry (Paul)

UNCONFIRMED

Bugs: Triage unconfirmed bugs for potential incoming regressions. Status

Issues Found

Bugs: New issues found during testing the Beta candidate Status

Items in bold are BLOCKER bugs

  • NEW bug 846305 - Hang when loading 3D Games Website
  • NEW bug 846254 - Crash in js::gc::Mark
  • NEW bug 846334 - Panels are flickering when opened in Social API

Beta 3

Builds FTP, HTTP
ID 20130305164032
Changeset de200581d19b (pushlog)
Checklist
  • [DONE] Functional Automation
  • [DONE] Manual Testing
  • [DONE] Bug Triage
  • [DONE] Update Automation on betatest channel
  • [DROPPED] Update Automation on releasetest channel - skipping
  • {{|}} Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform Functional Results Restart Results
Windows XP [DONE] 0 Fails [DONE] 0 Fails
Windows Vista [DONE] 0 Fails [DONE] 0 Fails
Windows 7 32-bit [DONE] 0 Fails [DONE] 0 Fails
Windows 7 64-bit [DONE] 0 Fails [DONE] 0 Fails
Windows 8 32-bit [DONE] 0 Fails [DONE] 0 Fails
Windows 8 64-bit [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.6 [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.7 [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.8 [DONE] 0 Fails [DONE] 0 Fails
Ubuntu 12.04 32-bit [DONE] 0 Fails [DONE] 0 Fails
Ubuntu 12.04 64-bit [DONE] 0 Fails [DONE] 0 Fails
Test Failures (known bugs)

Manual Testing

Report any bugs found during exploratory testing in Bugzilla and add them to the Issues Found section below.

Focus Area Windows 7 32bit Windows Vista 32bit Mac OSX 10.6 Ubuntu 12.04 32bit
Regression Tests:
Run the Firefox 20.0b3 Regression Tests in MozTrap. Do some exploratory testing around the testcases to try to exercise some edge-cases. See Beta 3 checklist for focus areas.
[DONE]
Alexandra
1 Bugs
[DONE]
Cornel
0 Bugs
[DONE]
Bogdan
0 Bugs
[DONE]
Manuela
0 Bugs
Web compatibility (search, webmail, banking, video, audio, gaming, news, cms, etc.) [DONE]
Alexandra
0 Bugs
[DONE]
Cornel
0 Bugs
[DONE]
Bogdan
0 Bugs
[DONE]
Manuela
0 Bugs

Update Testing

Results of running the Mozmill update automation.
20.0b2 should get a partial update, <= 20.0b1 should get a complete update.

Platform Betatest (2013-03-06) Releasetest (2013-03-06) Beta (2013-03-06)
Windows XP [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails
Windows Vista [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails
Windows 7 32-bit [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails
Windows 7 64-bit [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails
Windows 8 32-bit [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails
Windows 8 64-bit [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails
Mac OS X 10.6 [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails
Mac OS X 10.7 [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails
Mac OS X 10.8 [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails
Ubuntu 12.04 32-bit [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails
Ubuntu 12.04 64-bit [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails

Bug Triage

QAWANTED

Bugs: Test to find steps to reproduce, regression range, or testcases Status

NORMAL FIXES

Bugs: Verify these fixes in the latest Beta candidate Status
  • [DONE] VERIFIED bug 845491 Unresponsive plugin/plug-in (hang dialog): Button width doesn't fit text size (Manuela)
  • [DONE] VERIFIED bug 845881 HTTP status 204 is considered a preflight failure in FX 20 beta 1 (Virgil)
  • [DONE] VERIFIED bug 846186 Tilt error flooding error console (TypeError: mouseLerp is undefined) (Virgil)
  • [DONE] VERIFIED bug 845743 Firefox 19.0 does not support BIG5-HKSCS encoding. Version 18.0 does (Virgil)
  • [DONE] VERIFIED bug 844671 Tabs sent through Sync should not open in private browsing windows (Virgil)
  • [DONE] VERIFIED bug 843971 The sharing dialog for getUserMedia always reports the last shared set of devices, not the devices actively in use in a tab(Mihaela)
  • [DONE] VERIFIED bug 843214 getUserMedia - Request video for video #1, then request video/audio in video #2 in same/different tab - video #2 shows no camera/mic stream (Mihaela)
  • [DONE] VERIFIED bug 738952 "Save as..." File menu entry or Ctrl+S produces unexpected results when having a PDF file opened within PDF Viewer (Mihaela)
  • [DONE] COMMENTED bug 842228 Bug 540537 regressed in bug 491947 (Virgil)
  • [DONE] COMMENTED bug 554780 Make plugins provider correctly handle plugins being added and removed through detection at runtime (Paul)
  • [DONE] VERIFIED bug 833823 Random, unpredictable behavior with multiple YouTube embeds (Paul)
  • [DONE] VERIFIED bug 843247 Hyperlinks from other applications only open in new window when using private browsing (instead of in a new tab) (Paul)

SECURITY FIXES

Bugs: Verify these fixes in the latest Beta candidate (requires privileged access) Status
  • [DONE] VERIFIED bug 840480 use-after-poison in nsIFrame::Properties() (Ioana)
  • [DONE] VERIFIED bug 722831 some grayscale png images display improperly when "gfx.color_management.enablev4" enabled (Ioana)
  • [DONE] VERIFIED bug 837714 Assertion failure: allocated(), at ../../gc/Heap.h:472 or Opt-Crash [@ js::GCMarker::drainMarkStack] (Ioana)

TOPCRASH FIXES

Verify these signatures are no longer present on crashstats in the current Beta

  • N/A

UNCONFIRMED

Bugs: Triage unconfirmed bugs for potential incoming regressions. Status

Issues Found

Bugs: New issues found during testing the Beta candidate Status

Items in bold are BLOCKER bugs

Known Issues

ID Summary Priority Status
791195 Session is not restored after Firefox crash -- RESOLVED

1 Total; 0 Open (0%); 1 Resolved (100%); 0 Verified (0%);


Beta 4

Builds FTP, HTTP
ID 20130307075451
Changeset 1bc54ffd7a04 (pushlog)
Checklist
  • [DONE] Functional Automation
  • [DONE] Manual Testing
  • [DONE] Bug Triage
  • [DONE] Update Automation on betatest channel
  • [DROPPED] Update Automation on releasetest channel - skipping
  • {{|}} Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform Functional Results Restart Results
Windows XP [DONE] 0 Fails [DONE] 0 Fails
Windows Vista [DONE] 0 Fails [DONE] 0 Fails
Windows 7 32-bit [DONE] 0 Fails [DONE] 0 Fails
Windows 7 64-bit [DONE] 0 Fails [DONE] 0 Fails
Windows 8 32-bit [DONE] 0 Fails [DONE] 0 Fails
Windows 8 64-bit [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.6 [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.7 [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.8 [DONE] 0 Fails [DONE] 0 Fails
Ubuntu 12.04 32-bit [DONE] 0 Fails [DONE] 0 Fails
Ubuntu 12.04 64-bit [DONE] 0 Fails [DONE] 0 Fails
Test Failures (known bugs)

Manual Testing

We will verify the chemspill bug 848644 and run spot checks across platforms spending some time using sites with built-in text editors. Please see the etherpad for guidelines.

Breakpad End-to-End functionality (testcase)
  • [DONE] PASS Windows 8 64-bit (tester)
  • [DONE] PASS Windows 7 64-bit (juanb)
  • [DONE] PASS Ubuntu 12.04 32-bit (juanb)
  • [DONE] RESULT Mac OSX 10.7 (juanb)
Smoke Testing (basic functionality and web compatibility, particularly websites with built in text editors)
  • [DONE] PASS Windows 8 64-bit (Manuela)
  • [DONE] PASS Windows 7 64-bit (juanb)
  • [DONE] PASS Ubuntu 12.04 32-bit (juanb)
  • [DONE] PASS Mac OSX 10.7 (Alexandra)

Update Testing

Results of running the Mozmill update automation.
20.0b3 should get a partial update, <= 20.0b2 should get a complete update.

Platform Betatest (2013-03-07) Releasetest (skip) Beta (2013-03-07)
Windows XP [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails
Windows Vista [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails
Windows 7 32-bit [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails
Windows 7 64-bit [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails
Windows 8 32-bit [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails
Windows 8 64-bit [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails
Mac OS X 10.6 [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails
Mac OS X 10.7 [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails
Mac OS X 10.8 [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails
Ubuntu 12.04 32-bit [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails
Ubuntu 12.04 64-bit [DONE] : 0 Fails [DROPPED] : 0 Fails [ON TRACK] : 0 Fails

Bug Triage

SECURITY FIXES

Bugs: Verify these fixes in the latest Beta candidate (requires privileged access) Status
bug 848644: Security fix VERIFIED

NORMAL FIXES

Bugs: Verify these fixes in the latest Beta candidate Status
  • [DONE] VERIFIED bug 829557 Firefox freezes when content is copied to clipboard with the Zero Clipboard library (based on Flash) (Paul)
  • [DONE] UPDATED bug 846617 text shaping with AAT Myanmar fonts broken in Firefox 19 (Ioana)
    • Can't reproduce locally.
  • [DONE] UPDATED bug 844461 charset=iso-8859-1 parameter causes blank white page flash (Ioana)
    • Need more info for a more reliable verification.
  • [DONE] VERIFIED bug 817019 CSS radial gradient is broken with d2d/azure enabled (Bogdan)
  • [DONE] VERIFIED bug 743252 Don't print the URL and other information when printing PDFs (Bogdan)
  • [DONE] VERIFIED bug 829829 The stub installer fails to download the file when the server redirects to an HTTPS server (Bogdan)
  • [DONE] COMMENTED bug 846334 Panels are flickering when opened in Social API (Alexandra)

SECURITY FIXES

Bugs: Verify these fixes in the latest Beta candidate (requires privileged access) Status
  • [DONE] VERIFIED bug 848644 (CVE-2013-0787) Use-after-free caused by us replacing the nsHTMLEditor's edit rules object while running scripts from the flush happening in nsEditor::IsPreformatted triggered by execCommand from web content (juanb)

TOPCRASH FIXES

Verify these signatures are no longer present on crashstats in the current Beta

  • N/A

UNCONFIRMED

Bugs: Triage unconfirmed bugs for potential incoming regressions. Status

Issues Found

Bugs: New issues found during testing the Beta candidate Status

Items in bold are BLOCKER bugs

  • NEW bug 849216 - More than one button is highlighted in Social API

Issues Found

Bugs: New issues found during testing the Beta candidate Status

Items in bold are BLOCKER bugs

Beta 5

Builds FTP, HTTP
ID 20130313170052
Changeset 8bc9657bc249 (pushlog)
Checklist
  • [DONE] Functional Automation
  • [DONE] Manual Testing
  • [DONE] Bug Triage
  • [DONE] Update Automation on betatest channel
  • [DROPPED] Update Automation on releasetest channel - skipping for this beta period
  • {{|}} Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform Functional Results Restart Results
Windows XP [DONE] 0 Fails [DONE] 0 Fails
Windows Vista [DONE] 0 Fails [DONE] 0 Fails
Windows 7 32-bit [DONE] 0 Fails [DONE] 0 Fails
Windows 7 64-bit [DONE] 0 Fails [DONE] 0 Fails
Windows 8 32-bit [DONE] 0 Fails [DONE] 0 Fails
Windows 8 64-bit [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.6 [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.7 [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.8 [DONE] 0 Fails [DONE] 0 Fails
Ubuntu 12.04 32-bit [DONE] 0 Fails [DONE] 0 Fails
Ubuntu 12.04 64-bit [DONE] 0 Fails [DONE] 0 Fails
Test Failures (known bugs)

Manual Testing

Report any bugs found during exploratory testing in Bugzilla and add them to the Issues Found section below.

Focus Area Windows XP Windows 7 64bit Mac OSX 10.8 Ubuntu 12.10 32bit
Regression Tests:
Run the Firefox 20.0b5 Regression Tests in MozTrap. Do some exploratory testing around the testcases to try to exercise some edge-cases.
[DONE]
Paul
0 Bugs
[DONE]
Simona
0 Bugs
[DONE]
Mihaela
0 Bugs
[DONE]
Virgil
0 Bugs
Add-on compatibility
(top AMO, top third-party, toolbars, yandex, avg, norton, yahoo, bandoo, conduit, etc)
[DONE]
Paul
0 Bugs
[DONE]
Simona
1 Bugs
[DONE]
Mihaela
0 Bugs
[DONE]
Virgil
0 Bugs
Per-window Private Browsing
Exploratory testing around: data leaks, permanent PB mode, plug-in compatibility, web compatibility, basic functionality.
[DONE]
Paul
0 Bugs
[DONE]
Simona
1 Bugs
[DONE]
Mihaela
0 Bugs
[DONE]
Virgil
0 Bugs

Update Testing

Results of running the Mozmill update automation.
20.0b4 should get a partial update, <= 20.0b3 should get a complete update.

Platform Betatest (2013-03-14) Releasetest (Skip) Beta (2013-03-14)
Windows XP [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails
Windows Vista [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails
Windows 7 32-bit [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails
Windows 7 64-bit [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails
Windows 8 32-bit [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails
Windows 8 64-bit [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails
Mac OS X 10.6 [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails
Mac OS X 10.7 [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails
Mac OS X 10.8 [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails
Ubuntu 12.04 32-bit [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails
Ubuntu 12.04 64-bit [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails

Bug Triage

QAWANTED

Bugs: Test to find steps to reproduce, regression range, or testcases Status

NORMAL FIXES

Bugs: Verify these fixes in the latest Beta candidate Status
  • [DONE] VERIFIED bug 842290 "Restart Aurora" pop up doesn't appear when selecting private browsing mode (Ioana)
  • [DONE] VERIFIED bug 844750 CSS background-images don't render properly (MarioMi)
  • [DONE] VERIFIED bug 841308Bug 832641 caused panels to flicker as they're animating open (MarioMi)
  • [DONE] VERIFIED bug 842553 Double clicking an in-progress download in the Library opens its containing folder (Simona)
  • [DONE] COMMENTED bug 787158 crash in nsHttpConnection::OnSocketWritable (Manuela)
  • [DONE] VERIFIED bug 840818 "ASSERTION: value should always be stored and non-empty when state set" with nested -moz-column, odd characters (Manuela)
  • [DONE] VERIFIED bug 806820 crash in SuppressDeletedPropertyHelper (Manuela)
  • [DONE] COMMENTED bug 844406 Greasemonkey script stops working correctly in Firefox 20. 'self-hosted' builtin functions destroy js-callstack in certain case. (Manuela)
  • [DONE] VERIFIED bug 846617 text shaping with AAT Myanmar fonts broken in Firefox 19 (Manuela)
  • [DONE] VERIFIED bug 822620 disable Notification Center support for Firefox 19 and 20, it isn't ready (Cornel)
  • [DONE] VERIFIED bug 843342 PDF doesn't print correctly on USPS.com with pdfjs (pdf.js does not support automatic print using Adobe JavaScript) (Mihaela)
  • [DONE] VERIFIED bug 849903 uplift patches related to JBIG2 (Mihaela)

SECURITY FIXES

Bugs: Verify these fixes in the latest Beta candidate (requires privileged access) Status

TOPCRASH FIXES

Verify these signatures are no longer present on crashstats in the current Beta

  • [DONE] VERIFIED bug 820707 crash in gfxGDIFont::Initialize @ gfxFontFamily::HasItalicFace (Ioana)
  • [DONE] VERIFIED bug 821230 crash in nsRefreshDriver::SetHighPrecisionTimersEnabled (Ioana)
  • [DONE] VERIFIED bug 830654 crash in js::ObjectImpl::getPrivate (Ioana)
  • [DONE] VERIFIED bug 837288 crash in nsBlockFrame::DestroyFrom @ nsIFrame::SetNextSibling (Ioana)
  • [DONE] VERIFIED bug 572011 Crash @ nsDiskCacheStreamIO::FlushBufferToFile (Ioana)

UNCONFIRMED

Bugs: Triage unconfirmed bugs for potential incoming regressions. Status

Issues Found

Bugs: New issues found during testing the Beta candidate Status
  • NEW bug 851031 The Downloads button's position is changed after uninstalling Yandex (Simona)
  • NEW bug 851105 Data leak due to Yandex autocomplete (Simona)

Items in bold are BLOCKER bugs

Beta 6

Builds FTP, HTTP
ID 20130320062118
Changeset 2b185ce3436a (pushlog)
Checklist
  • [DONE] Functional Automation
  • [DONE] Manual Testing
  • [DONE] Bug Triage
  • [DONE] Update Automation on betatest channel
  • [DONE] Update Automation on releasetest channel - skipping for this beta period
  • {{|}} Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform Functional Results Restart Results
Windows XP [DONE] x Fails [DONE] x Fails
Windows Vista [DONE] x Fails [DONE] x Fails
Windows 7 32-bit [DONE] x Fails [DONE] x Fails
Windows 7 64-bit [DONE] x Fails [DONE] x Fails
Windows 8 32-bit [DONE] x Fails [DONE] x Fails
Windows 8 64-bit [DONE] x Fails [DONE] x Fails
Mac OS X 10.6 [DONE] x Fails [DONE] x Fails
Mac OS X 10.7 [DONE] x Fails [DONE] x Fails
Mac OS X 10.8 [DONE] x Fails [DONE] x Fails
Ubuntu 12.04 32-bit [DONE] x Fails [DONE] x Fails
Ubuntu 12.04 64-bit [DONE] x Fails [DONE] x Fails
Test Failures (known bugs)

Manual Testing

Report any bugs found during exploratory testing in Bugzilla and add them to the Issues Found section below.

Focus Area Windows 7 32bit Windows 8 64bit Mac OSX 10.7 Ubuntu 12.04 32bit
Regression Tests:
Run the Firefox 20.0b6 Regression Tests in MozTrap. Do some exploratory testing around the testcases to try to exercise some edge-cases.
[DONE]
Simona
0 Bugs
[DONE]
Cornel
0 Bugs
[DONE]
Virgil
0 Bugs
[DONE]
Manuela
0 Bugs
Plug-in compatibility
Flash, Java, Silverlight, Realplayer, Quicktime, Shockwave, Unity [Mac]
[DONE]
Simona
1 Bugs
[DONE]
Cornel
0 Bugs
[DONE]
Virgil
0 Bugs
[DONE]
Manuela
0 Bugs
Panel-based Downloads Manager
Download in normal and private browsing (including permanent PB), download large files, multiple downloads, open downloads, open containing folder, context menus etc.
[DONE]
Simona
1 Bugs
[DONE]
Cornel
0 Bugs
[DONE]
Virgil
1 Bug
[DONE]
Manuela
0 Bugs
Permanent Private Browsing
Entering mode, exiting mode, opening links in new tabs and windows, pinned tabs, tab reordering, web compatibility (alexa top 10).
[DONE]
Simona
0 Bugs
[DONE]
Cornel
2 Bugs
[DONE]
Virgil
1 Bug
[DONE]
Manuela
0 Bugs

Update Testing

Results of running the Mozmill update automation.
20.0b5 should get a partial update, <= 20.0b4 should get a complete update.

Platform Betatest (2013-03-20) Releasetest Beta (2013-03-21)
Windows XP [DONE] : 0 Fails [DONE] : 0 Fails [ON TRACK] : x Fails
Windows Vista [DONE] : 0 Fails [DONE] : 0 Fails [ON TRACK] : x Fails
Windows 7 32-bit [DONE] : 0 Fails [DONE] : 0 Fails [ON TRACK] : x Fails
Windows 7 64-bit [DONE] : 0 Fails [DONE] : 0 Fails [ON TRACK] : x Fails
Windows 8 32-bit [DONE] : 0 Fails [DONE] : 0 Fails [ON TRACK] : x Fails
Windows 8 64-bit [DONE] : 0 Fails [DONE] : 0 Fails [ON TRACK] : x Fails
Mac OS X 10.6 [DONE] : 0 Fails [DONE] : 0 Fails [ON TRACK] : x Fails
Mac OS X 10.7 [DONE] : 0 Fails [DONE] : 0 Fails [ON TRACK] : x Fails
Mac OS X 10.8 [DONE] : 0 Fails [DONE] : 0 Fails [ON TRACK] : x Fails
Ubuntu 12.04 32-bit [DONE] : 0 Fails [DONE] : 0 Fails [ON TRACK] : x Fails
Ubuntu 12.04 64-bit [DONE] : 0 Fails [DONE] : 0 Fails [ON TRACK] : x Fails

Bug Triage

QAWANTED

Bugs: Test to find steps to reproduce, regression range, or testcases Status

NORMAL FIXES

Bugs: Verify these fixes in the latest Beta candidate Status
  • [DONE] VERIFIED bug 846334 Panels are flickering when opened in Social API (Alexandra)
  • [DONE] COMMENTED bug 844580 crash in js::ObjectImpl::nativeLookup (Alexandra)
  • [DONE] VERIFIED bug 813436 Tools > Facebook Messenger menu shown when Social API is disabled (Alexandra)
  • [DONE] VERIFIED bug 829870 Appear the non-expected color gap on window that are opened from private one. (Alexandra)
  • [DONE] VERIFIED bug 787158 crash in nsHttpConnection::OnSocketWritable (Manuela)
  • [DONE] VERIFIED bug 844406 Greasemonkey script stops working correctly in Firefox 20. 'self-hosted' builtin functions destroy js-callstack in certain case. (Manuela)
  • [DONE] VERIFIED bug 845592 New Private Window entry in File menu doesn't make sense in permanent private browsing (Ioana)
  • [DONE] VERIFIED bug 851473 Restart dialogues say opposite feature enable/disable things when toggling Private browsing with checkbox or drop-down (Always use private browsing mode / Never remember history) (Ioana)
  • [DONE] VERIFIED bug 851489 Private Browsing is not enabled after selecting "Always use private browsing mode" (Ioana)
  • [DONE] VERIFIED bug 817497 Generic extension icons do not display anymore in Add-ons Manager (Virgil)
  • [DONE] VERIFIED bug 817790 Stretched Facebook icon briefly shows over entire toolbar during window startup (Virgil)
  • [DONE] VERIFIED bug 740923 Lion Fullscreen: Black area when opening new windows (Virgil)
  • [DONE] VERIFIED bug 851495 text shaping with AAT Khmer fonts broken in Firefox 19 (Virgil)
  • [DONE] VERIFIED bug 828815 SVGRectElement["onxxx"] = Function does not work, No Tooltip on SVG graphic (StatCounter) (Bogdan)
  • [DONE] VERIFIED bug 812897 Context menu, awesome bar, and search bar popup content occasionally or persistently not rendered (Bogdan)
  • [DONE] VERIFIED bug 827938 Appear the non-expected line on browser toolbar on OSX when tab-on-bottom. (Bogdan)
  • [DONE] VERIFIED bug 832290 Proper styling for full-screen private windows on Mac (Bogdan)
  • [DONE] VERIFIED bug 829180 Externally opened urls may open in private browsing windows (Bogdan)
  • [DONE] VERIFIED bug 812453 Unexpected problem prompt when printing <canvas> (Bogdan)
  • [DONE] VERIFIED bug 845273 Firefox 20 (beta) declares "Corrupted Content Error" when loading website with Access-Control-Allow-Origin header (Paul)
  • [DONE] VERIFIED bug 822906 Crash [@ mozilla::css::OverflowChangedTracker::Flush] with table and transform (Paul)
  • [DONE] VERIFIED bug 822842 Crash [@ nsROCSSPrimitiveValue::Reset] after setting padding-left-value (!? (Paul)
  • [DONE] VERIFIED bug 822865 Crash [@ nsDisplayBackgroundImage::AppendBackgroundItemsToTop] with -moz-appearance, display:table, elementFromPoint (Paul)
  • [DONE] VERIFIED bug 827272 crash in nsSSLIOLayerHelpers::Cleanup (Paul)
  • [DONE] VERIFIED bug 827274 crash in `anonymous namespace::CTypesActivityCallback(JSContext*, js::CTypesActivityType) with wkhtmltopdf (Paul)

SECURITY FIXES

Bugs: Verify these fixes in the latest Beta candidate (requires privileged access) Status

TOPCRASH FIXES

Verify these signatures are no longer present on crashstats in the current Beta

  • N/A

UNCONFIRMED

Bugs: Triage unconfirmed bugs for potential incoming regressions. Status

Issues Found

Bugs: New issues found during testing the Beta candidate Status

Items in bold are BLOCKER bugs

New Issues

  • NEW bug 853422 File menu should be more consistent for private sessions (permanent & not) (Ioana)
  • NEW bug 853414 Themes installed in PB mode should not be visible in Normal Mode (Cornel)
  • NEW bug 853426 Themes installed in PB mode are only visible in normal mode (Cornel)
  • NEW bug 853435 Real Player breaks HTML5 videos (Simona)
  • NEW bug 853463 Download statusbar add-on continues downloading files from Normal Browsing to Private Broswing (Virgil)
  • NEW bug 853472 Downloads Panel jumps with Download Statusbar (Virgil)
  • NEW bug 853473 Clear List in private browsing (Simona)
  • NEW bug 853365 Panel messages displayed wrong in SocialAPI (Bogdan)

Known Issues

  • NEW bug 606336 Yahoo toolbar broken on trunk, possibly by a change backported to branches

Beta 7

Builds FTP, HTTP
ID 20130325214615
Changeset 06ddb0618063 (pushlog)
Checklist
  • [DONE] Functional Automation
  • [DONE] Manual Testing
  • [DONE] Bug Triage
  • [DROPPED] Update Automation on betatest channel - skipped in favor of releasetest
  • [DONE] Update Automation on releasetest channel - skipping for this beta period
  • [DONE] Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform Functional Results Restart Results
Windows XP [DONE] 0 Fails [DONE] 0 Fails
Windows Vista [DONE] 0 Fails [DONE] 0 Fails
Windows 7 32-bit [DONE] 0 Fails [DONE] 0 Fails
Windows 7 64-bit [DONE] 0 Fails [DONE] 0 Fails
Windows 8 32-bit [DONE] 0 Fails [DONE] 0 Fails
Windows 8 64-bit [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.6 [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.7 [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.8 [DONE] 0 Fails [DONE] 0 Fails
Ubuntu 12.04 32-bit [DONE] 0 Fails [DONE] 0 Fails
Ubuntu 12.04 64-bit [DONE] 0 Fails [DONE] 0 Fails
Test Failures (known bugs)

Manual Testing

Report any bugs found during exploratory testing in Bugzilla and add them to the Issues Found section below.

Focus Area Windows 7 64bit Windows 8 32bit Mac OSX 10.6 Ubuntu 12.04 64bit
Regression Tests:
Run the Firefox 20.0b6 Regression Tests in MozTrap. Do some exploratory testing around the testcases to try to exercise some edge-cases.
[DONE]
Cornel
0 Bugs
[DONE]
Simona
0 Bugs
[DONE]
Bogdan
0 Bugs
[DONE]
Virgil
0 Bugs
Web compatibility
Search, webmail, banking, video, audio, gaming, news, cms, etc.
[DONE]
Cornel
0 Bugs
[DONE]
Simona
0 Bugs
[DONE]
Bogdan
1 Bug
[DONE]
Virgil
0 Bugs
Private browsing
Data leaks (history, cache, autocomplete, cookies etc), plug-in compatibility (flash, silverlight, java), pdf.js, downloads, session restore etc.
[DONE]
Cornel
0 Bugs
[DONE]
Simona
0 Bugs
[DONE]
Bogdan
0 Bugs
[DONE]
Virgil
0 Bugs

Mac OSX 10.5 and Windows 2000 de-support (install and update)

  • [DONE] PASS Windows 2000 (Simona)
  • [DONE] PASS Mac OSX 10.5 (Simona)

Update Testing

Results of running the Mozmill update automation.
20.0b5 should get a partial update, <= 20.0b4 should get a complete update.

Platform Betatest (2013-03-26) Releasetest Beta (2013-03-27)
Windows XP [ON TRACK] : X Fails [DONE] : X Fails [DONE] : X Fails
Windows Vista [ON TRACK] : X Fails [DONE] : X Fails [DONE] : X Fails
Windows 7 32-bit [ON TRACK] : X Fails [DONE] : X Fails [DONE] : X Fails
Windows 7 64-bit [ON TRACK] : X Fails [DONE] : X Fails [DONE] : X Fails
Windows 8 32-bit [ON TRACK] : X Fails [DONE] : X Fails [DONE] : X Fails
Windows 8 64-bit [ON TRACK] : X Fails [DONE] : X Fails [DONE] : X Fails
Mac OS X 10.6 [ON TRACK] : X Fails [DONE] : X Fails [DONE] : X Fails
Mac OS X 10.7 [ON TRACK] : X Fails [DONE] : X Fails [DONE] : X Fails
Mac OS X 10.8 [ON TRACK] : X Fails [DONE] : X Fails [DONE] : X Fails
Ubuntu 12.04 32-bit [ON TRACK] : X Fails [DONE] : X Fails [DONE] : X Fails
Ubuntu 12.04 64-bit [ON TRACK] : X Fails [DONE] : X Fails [DONE] : X Fails

Bug Triage

QAWANTED

Bugs: Test to find steps to reproduce, regression range, or testcases Status

NORMAL FIXES

Bugs: Verify these fixes in the latest Beta candidate Status
  • [DONE] VERIFIED bug 853996 A menu command and the shortcut bring a different effect in permanent private browsing (Ioana)
  • [DONE] VERIFIED bug 827225 "Assertion failure: obj->isTypedArray()" with WebGL readPixels (Manuela)
  • [DONE] VERIFIED bug 829430 Assertion failure: "Don't call FixWeakMappingGrayBits during a GC" (Manuela)
  • [DONE] VERIFIED bug 840906 "Assertion failure: !aIID.Equals(...)" in nsDOMCSSRGBColor::QueryInterface (Manuela)
  • [DONE] VERIFIED bug 719180 Loading an XBL binding via chrome:// locks XPI file, prevents restartless extension from updating (Manuela)
  • [DONE] VERIFIED bug 838864 keyword.URL reset interferes poorly with some partner add-ons (Manuela)
  • [DONE] VERIFIED bug 853387 crash in nsNPAPIPluginInstance::HandleEvent @ libnpgeplugin with Google Earth Plugin (in-process in 32-bit mode) on Google Maps (Manuela)
  • [DONE] COMMENTED bug 828180 crash in nsIContent::IsHTML (Manuela)
  • [DONE] VERIFIED bug 820067 crash in nsDocShell::InternalLoad (Paul)
  • [DONE] VERIFIED bug 835978 crash in nsDisplayList::HitTest (Paul)
  • [DONE] VERIFIED bug 814828 canvas fillRect with op=destOut 64X slower in Firefox 17 with hardware acceleration disabled (Paul)

SECURITY FIXES

Bugs: Verify these fixes in the latest Beta candidate (requires privileged access) Status

TOPCRASH FIXES

Verify these signatures are no longer present on crashstats in the current Beta

  • N/A

UNCONFIRMED

Bugs: Triage unconfirmed bugs for potential incoming regressions. Status

Issues Found

Bugs: New issues found during testing the Beta candidate Status

Items in bold are BLOCKER bugs

New Issues

  • NEW bug 854923 Resize window icon is not visible
  • NEW bug 854864 crash in nsDocShell::InternalLoad
  • NEW bug 854926 Firefox prompts a warning when closing private windows with multiple tabs open

Known Issues

  • NEW bug 742394 Firefox Not Keeping The Default Browser Setting for Linux 64-bit (Virgil)

Release Candidate 1

Builds FTP, HTTP
ID 20130326150557
Changeset c90d44bfa96c (pushlog)
Checklist
  • [DONE] Functional Automation
  • [DONE] Manual Testing
  • [DONE] Bug Triage
  • [DONE] Update Automation on betatest channel
  • {{|}} Update Automation on releasetest channel
  • {{|}} Final sign-off sent and updates qualified on Beta channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform Functional Results Restart Results
Windows XP [DONE] 0 Fails [DONE] 0 Fails
Windows Vista [DONE] 0 Fails [DONE] 0 Fails
Windows 7 32-bit [DONE] 0 Fails [DONE] 0 Fails
Windows 7 64-bit [DONE] 0 Fails [DONE] 0 Fails
Windows 8 32-bit [DONE] 1 Failure [DONE] 0 Fails
Windows 8 64-bit [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.6 [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.7 [DONE] 0 Fails [DONE] 0 Fails
Mac OS X 10.8 [DONE] 0 Fails [DONE] 0 Fails
Ubuntu 12.04 32-bit [DONE] 0 Fails [DONE] 0 Fails
Ubuntu 12.04 64-bit [DONE] 0 Fails [DONE] 0 Fails
Test Failures (known bugs)

Manual Testing

Report any bugs found during exploratory testing in Bugzilla and add them to the Issues Found section below.

Focus Area Windows XP Windows 7 64bit Mac OSX 10.8 Ubuntu 12.10 32bit
Regression Tests:
Run the Firefox 20.0 RC Regression Tests in MozTrap. Do some exploratory testing around the testcases to try to exercise some edge-cases.
[DONE]
Manuela
0 Bugs
[DONE]
Cornel
1 Bug
[DONE]
Simona
1 Bugs
[DONE]
Virgil
0 Bugs

Update Testing

Results of running the Mozmill update automation.
20.0b5 should get a partial update, <= 20.0b4 should get a complete update.

Platform Betatest (2013-03-27) Releasetest Release (2013-04-02)
Windows XP [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails
Windows Vista [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails
Windows 7 32-bit [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails
Windows 7 64-bit [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails
Windows 8 32-bit [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails
Windows 8 64-bit [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails
Mac OS X 10.6 [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails
Mac OS X 10.7 [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails
Mac OS X 10.8 [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails
Ubuntu 12.04 32-bit [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails
Ubuntu 12.04 64-bit [DONE] : 0 Fails [ON TRACK] : 0 Fails [ON TRACK] : 0 Fails

Bug Triage

QAWANTED

Bugs: Test to find steps to reproduce, regression range, or testcases Status

NORMAL FIXES

Bugs: Verify these fixes in the latest release candidate Status
  • [DONE] VERIFIED bug 828180 crash in nsIContent::IsHTML (Manuela)
  • [DONE] VERIFIED bug 749394 Desktop theme changes for per-window Private Browsing (Windows and OS X) (Ioana)
  • [DONE] VERIFIED bug 827454 Private browsing theme changes for Linux (Ioana)
  • [DONE] VERIFIED bug 804572 Tilt can't display correctly on some pages. (Ioana)
  • [DONE] VERIFIED bug 833004 Can't start Tilt (Ioana)
  • [DONE] VERIFIED bug 826388 Allow single click on line numbers to add/remove breakpoints (Ioana)
  • [DONE] VERIFIED bug 815239 [toolbox] Using a tool keyboard shortcut twice closes the toolbox. It should select it instead. (Ioana)
  • [DONE] VERIFIED bug 815383 pressing command key over vnc results in "Assertion failure: !(flag & [aNativeEvent modifierFlags]), at .../widget/cocoa/TextInputHandler.mm:1676" (Bogdan)
  • [DONE] VERIFIED bug 802390 alt text is shown simultaneously with images (Bogdan)
  • [DONE] VERIFIED bug 802440 Some elements are invalidated when other elements are scrolled off the screen (Bogdan)
  • [DONE] VERIFIED bug 830066 Drag & drop links into toolkit downloads manager to download files fails (TypeError: aInitiatingDocument is undefined) (Bogdan)
  • [DONE] COMMENTED bug 819252 CMD + swipe left to open previous location in new tab is broken with Magic Mouse (Bogdan)
  • [DONE] VERIFIED bug 820373 crash in mozilla::dom::TextEncoderBinding::encode @ mozilla::dom::MaybeWrapValue (Alexandra)
  • [DONE] VERIFIED bug 831472 Crash [@ nsGlobalWindow::GetInnerScreenRect()] due to lack of null check (Alexandra)
  • [DONE] VERIFIED bug 843675 transform:scale(1) on MacOs don't work with <select> and mouse click (Alexandra)
  • [DONE] VERIFIED bug 808466 Changing selection sometimes doesn't repaint immediately (Alexandra)
  • [DONE] VERIFIED bug 813550 Location bar's placeholder in an RTL chrome should be aligned on the right (Paul)
  • [DONE] VERIFIED bug 818023 IonMonkey: function.caller does not always work (and in interpreter and JM) (Paul)
  • [DONE] VERIFIED bug 821152 SVG graphic clipped with mask applied (Paul)
  • [DONE] VERIFIED bug 806432 SVG filter region sometimes clipped when viewBox and opacity properties used (Paul)
  • [DONE] VERIFIED bug 832641 Menu items slow to paint/respond after peeking their sub-menu popups (Paul)

SECURITY FIXES

Bugs: Verify these fixes in the latest release candidate (requires privileged access) Status

TOPCRASH FIXES

Verify these signatures are no longer present on crashstats in the current RC

  • N/A

UNCONFIRMED

Bugs: Triage unconfirmed bugs for potential incoming regressions. Status

Issues Found

Bugs: New issues found during testing the Beta candidate Status

Items in bold are BLOCKER bugs

New Issues

  • NEW bug 855705 Selected text remains highlighted
  • NEW bug 855726 Copy actions are broken o HTML5 videos

Known Issues

  • bug 791195 Session is not restored after Firefox crash

20.0.1 (Windows Only)

Builds FTP, HTTP
ID 20130409194949
Changeset 346a2850042d (pushlog)
Checklist
  • [DONE] Functional Automation
  • [DONE] Manual Testing
  • [DONE] Bug Verifications
  • [DONE] Update Automation on betatest channel
  • [ON TRACK] Update Automation on releasetest channel
  • [ON TRACK] Final sign-off sent and updates qualified on the release channel

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform Functional Results [results] Remote [report] Addon [report]
Windows XP 0 Failures 0 Failures 0 Failures
Windows Vista 0 Failures 0 Failures 0 Failures
Windows 7 32-bit 0 Failures 0 Failures 0 Failures
Windows 7 64-bit 0 Failures 0 Failures 0 Failures
Windows 8 32-bit 0 Failures 0 Failures 0 Failures
Windows 8 64-bit 0 Failures 0 Failures 0 Failures
Test Failures (known bugs)

Manual Testing

Report any bugs found during exploratory testing in Bugzilla and add them to the Issues Found section below.

Focus Area Windows XP Windows 7 64bit
Regression Tests:
Run a basic spot check on the candidate builds. This is Windows only.
[DONE]
Simona
0 Bugs
[DONE]
Virgil
0 Bugs

Update Testing

Results of running the Mozmill update automation.
20.0 should get a partial update, <= 19.0.1 should get a complete update.

Platform Betatest (2013-04-10) Releasetest Release (2013-04-10)
Windows XP [DONE] : 0 Fails [ON TRACK] : x Fails [ON TRACK] : x Fails
Windows Vista [DONE] : 0 Fails [ON TRACK] : x Fails [ON TRACK] : x Fails
Windows 7 32-bit [DONE] : 0 Fails [ON TRACK] : x Fails [ON TRACK] : x Fails
Windows 7 64-bit [DONE] : 0 Fails [ON TRACK] : x Fails [ON TRACK] : x Fails
Windows 8 32-bit [DONE] : 0 Fails [ON TRACK] : x Fails [ON TRACK] : x Fails
Windows 8 64-bit [DONE] : 0 Fails [ON TRACK] : x Fails [ON TRACK] : x Fails

Bug Fix Verifications

  • [DONE] - bug 846848 – [OS.File] Handle UNC-formatted paths (Paul)
  • [DONE] - bug 857588 – Search engine list empty when the profile is on a network drive. (Paul)
  • [DONE] - bug 857672 – Address Bar not working (Paul)
  • [DONE] - bug 857291 – SPNEGO / MS KRB5 no longer working. Tries to use NTLM SSP instead. (contributor)

Issues Found

Bugs: New issues found during testing the Beta candidate Status

Items in bold are BLOCKER bugs