Releases/Firefox 21/Test Plan

From MozillaWiki
Jump to: navigation, search

« Firefox 20 | Firefox 21 | Firefox 22 »

Contents

Firefox 21 Test Plan

Summary

Milestone Firefox 21
Status Currently in Beta -- merging to Release on 2013-05-14
Lead Anthony Hughes (irc: ashughes)
2nd Juan Becerra (irc: juanb)
Builds 21.0a2

The following is the test plan for Firefox 21 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 21. After the release this document will be used as an archive of what was done to validate this release.

If you have some free time, please pick a task below and get in touch with one of the leads.

Get Involved

If you'd like to help make this version of Firefox better, please don't hesitate to contact us. We accept help from people of all backgrounds, including people with no formal QA training and experience.

If you have any of the following operating systems, we need your help:

  • Windows: XP, Vista, 7, 8
  • Mac OSX: 10.6, 10.7, 10.8
  • Linux: any modern distribution

Here are some ways you can help:

  • Install the latest build, use it as your day-to-day browser, and report any issues you encounter
  • Provide general feedback on a new feature
  • Select a fixed bug and test to verify that it is truly fixed
  • Select an unconfirmed bug and test to confirm that it is truly an issue
  • Select a regressionwindow-wanted bug and test to find the regression range

If you'd like to know more, please email Anthony Hughes.

Schedule

The following is a description of the tasks which need to be completed throughout this release.

  • [DONE] 2013-01-07: Nightly Merge
  • [DONE] 2013-02-18: Aurora Merge
  • [DONE] 2013-03-29: Pre-Beta feature sign-offs
  • [DONE] 2013-04-03: Beta 1
  • [DONE] 2013-04-11: Beta 2
  • [DONE] 2013-04-18: Beta 3
  • [DONE] 2013-04-25: Beta 4
  • 2013-05-03: Beta 5
  • 2013-05-08: Beta 6
  • 2013-05-13: Release Candidate
  • 2013-05-14: Final

Nightly

Features

See the Feature Release Tracking wiki for a general overview

Feature Owner Status Tests
H264/AAC/MP3 support for Windows 7/8 Cornel Ionce On Track. Landed in Firefox 21 but may be pref'd off at some point
3-state UI for DNT Alexandra Lucinet On Track. Landed: currently tracked in bug 765398
Mixed Content Blocker Virgil Dicu

Endurance Automation

The purpose of this section is to track any potential memory regressions via spikes reported in the dashboard.

What to Do
  1. Check the Endurance Charts
  2. If you see an anomalous event (ie. spike or dip), get the report for the specific event
  3. If the event happens twice in a row, file a bug under Mozmill Tests
  4. Notify Anthony Hughes of the issue
  5. Record the event in this wiki below
Current Status (Mihaela Velimiroviciu) - Last Updated - yyyy.mm.dd

Bugzilla query error

Query options must be valid json


Functional Automation

The purpose of this section is to track any potential functional regressions via results reported in the dashboard.

What to Do
  1. Check the testrun reports
  2. If you see any failing tests, check that there are not any bugs on file for it already
  3. If not, file a bug under Mozmill Tests
  4. Notify Anthony Hughes of the issue
  5. Record the event in this wiki below
Current Status - Last Updated - yyyy-mm-dd

Bug Triage

The purpose of this section is to triage UNCONFIRMED incoming bugs.

What to Do
  1. Check the UNCONFIRMED bugs
  2. Test the bugs in the latest Firefox Nightly
  3. If you can't reproduce the bug, ask for the reporter to provide more information
  4. If you reproduce the bug, add your testing information and mark the bug NEW
  5. Notify Anthony Hughes of the NEW issue
Current Status (SoftVision) - Last Updated

Result Spreadsheet

Aurora Merge

Builds nightlies
Changeset tip (pushlog)
Checklist
  • [DONE] Run automated Mozmill tests
  • [DONE] Qualify included features
  • [DONE] Verify important bug fixes
  • [DONE] Qualify updates on auroratest channel
  • [DONE] Send sign-off to release-drivers to enable updates
  • [DONE] Qualify updates on aurora channel

Bug Fix Verifications

Bug QA Contact Status
bug 841094 "extensions" directory disappeared after updating to Feb12th Nightly Anthony Hughes [DONE] VERIFIED FIXED
bug 842039 Updater removes command line option after update Anthony Hughes [DONE] VERIFIED FIXED
bug 841919 Turn off --enable-metro on Aurora after Firefox 21 uplift to Aurora Anthony Hughes [DONE] VERIFIED FIXED
bug 843933 Disable health report on Aurora 21 Anthony Hughes [DONE] VERIFIED DISABLED

Update Testing

Platform auroratest aurora
Windows 7 32-bit [DONE]
SOURCE: Aurora 20.0a2 20130123042017
TARGET: Aurora 21.0a2 20130222111254
Mac OS X 10.7 [DONE]
SOURCE: Aurora 19.0a2 20121202042013
TARGET: Aurora 21.0a2 20130222111254
Ubuntu 12.04 64-bit [DONE]
SOURCE: Aurora 20.0a2 20130201042018
TARGET: Aurora 21.0a2 20130222111254
Issues

AURORA Channel

Expected target build: Aurora 21.0a2 en-US 20130222111254

Windows 7 32-bit

  • [DONE] Aurora 20.0a2 en-US 20130123042017 -> Aurora 21.0a2 en-US 20130222111254
  • FAIL Aurora 20.0a2 it 20130123042017 -> Aurora 20.0a2 20130219042021 (see bug 844336)

Mac OSX 10.7

  • [DONE] Aurora 19.0a2 en-US 20121202042013 -> Aurora 21.0a2 en-US 20130222111254
  • [DONE] Aurora 19.0a2 fr 20121202042013 -> Aurora 21.0a2 fr 20130222111254

Ubuntu 12.04 32-bit

  • [DONE] Aurora 19.0a2 en-US 20130201042018 -> Aurora 21.0a2 en-US 20130222111254
  • [DONE] Aurora 19.0a2 es-ES 20130201042018 -> Aurora 21.0a2 es-ES 20130222111254

Feature Sign Off

Feature Owner Win32 Mac Linux
Mixed Content Blocker Virgil Dicu [DROPPED] Scoped for Fx22 [DROPPED] Scoped for Fx22 [DROPPED] Scoped for Fx22
H264/AAC/MP3 support for Windows 7/8 Cornel Ionce [DONE] Ready for Aurora N/A N/A
3-state UI for DNT Alexandra Lucinet [DONE] Ready for Aurora [DONE] Ready for Aurora [DONE] Ready for Aurora
JavaScript Profiling Mihaela Velimiroviciu [DONE] Ready for Aurora [DONE] Ready for Aurora [DONE] Ready for Aurora

Platforms: Windows 7 64bit, Windows 8 32bit, Ubuntu 12.04, Mac OSX 10.8.2.

l10n

Checklist
  • [DONE] Functional testing via mozmill-ci: win32 fr
  • [DONE] Update testing via mozmill-ci: win32 fr
  • [DONE] Manual spotcheck functional and updates for de, it, es-ES
  • [DONE] Download links verified [1]
  • [DONE] Verify fixed bug 844262, bug 844336

Beta 1

Builds FTP, HTTP
ID 20130401192816
Changeset 2e91ff229d84 (pushlog)
Checklist
  • [DONE] Automated Tests
  • [DONE] Feature Sign-off
  • [DONE] Regression Tests
  • [DONE] Bug Fix Verifications
  • [DONE] Update Tests (betatest)
  • [DONE] Update Tests (releasetest)
  • [DONE] Update Tests (beta)

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform Results [dashboard]
Windows XP [DONE] en-US & da
Windows Vista [DONE] en-US & it
Windows 7 32-bit [DONE] en-US & zh-TW
Windows 7 64-bit [DONE] en-US & ro
Windows 8 32-bit [DONE] en-US & es-ES
Windows 8 64-bit [DONE] en-US & pt-BR
Mac OSX 10.6 [DONE] en-US & de
Mac OSX 10.7 [DONE] en-US & fr
Mac OSX 10.8 [DONE] en-US & it
Ubuntu 12.04 32-bit [DONE] en-US & de
Ubuntu 12.04 64-bit [DONE] en-US & fr

Known test failures in bugzilla

Feature Sign Off

Regression Tests

As per the checklist we will regression test the following areas.

Test Windows 7 32-bit Windows 8 64-bit Mac OSX 10.8 Ubuntu 12.04 32-bit
Breakpad end-to-end functionality [DONE] Manuela [DONE] Alexandra [DONE] Virgil [DONE] Cornel
Mac OSX 10.8 code signing N/A N/A [DONE] Virgil N/A
HTML5 Audio/Video playback [DONE] Manuela [DONE] Alexandra [DONE] Virgil [DONE] Cornel
Feedly Add-on Compatibility [DONE] Manuela [DONE] Alexandra [DONE] Virgil [DONE] Cornel
Tab animation with D2D on/off (bug 838758) [DONE] Manuela [DONE] Alexandra N/A N/A

Bug Fix Verifications

The following bugs have been fixed and need confirmation:

  • [DONE] bug 717178 Printing stops after 73%, and browser hangs up (Paul Silaghi)
  • [DONE] bug 738952 "Save as..." File menu entry or Ctrl+S produces unexpected results when having a PDF file opened within PDF Viewer (Ioana Budnar)
  • [ON TRACK] bug 799523 [Retina] Crash on entering full screen mode in OS X (Matt Wobensmith) - Pending verification
  • [DONE] bug 803568 Nightly crashed when see pdf file in fullscreen (presentation mode) or zooming (Ioana Budnar)
  • [DONE] bug 835045 - Core Graphics stops rendering the canvas content (Ioana Budnar)
  • [DONE] bug 836044 Aurora stub installer doesn't seem to be working (Paul Silaghi)
  • [DONE] bug 837486 Reposition pinned tabs when resetting tab strip underflow (Anthony Hughes)
  • [DONE] bug 840195 Update fails if FF is installed in a non-default install path (Paul Silaghi)
  • [DONE] bug 841239 Pandora.com stops playing music due to jplayer error frequently (Stephen Donner)
  • [DONE] bug 843671 - Cannot enter comment in plugin crash UI (Anthony Hughes)
  • [DONE] bug 844466 - Tab bar doesn't render tab for link from a modal dialog until dialog is closed (Ioana Budnar)
  • [DONE] bug 844553 - Plugins disabled by Aurora 21.x (Paul Silaghi)
  • [DROPPED] bug 844561 - Firefox prompts when closing private window (Ioana Budnar) - Pending fix for bug 854926
  • [DONE] bug 844671 - Tabs sent through Sync should not open in private browsing windows (Virgil Dicu)
  • [DONE] bug 845486 webgl conformance crashes frequently on ubuntu VMs (Anthony Hughes)
  • [DONE] bug 845592 - New Private Window entry in File menu doesn't make sense in permanent private browsing (Ioana Budnar)
  • [DONE] bug 845837 - Drag and Drop features in Blackboard Learn 9.1 SP9 broken after updating to Firefox 19 (Bogdan Maris)
  • [DONE] bug 846186 - Tilt error flooding error console (TypeError: mouseLerp is undefined) (Ioana Budnar)
  • [DONE] bug 846334 - Panels are flickering when opened in Social API (Bogdan Maris)
  • [DONE] bug 846617 - Text shaping with AAT Myanmar fonts broken in Firefox 19 (Simona Marcu)
  • [DONE] bug 846843 Profile directory (containing PII) may be submitted in Firefox Health Report errors (Anthony Hughes)
  • [DONE] bug 847627 - The clear Downloads History at shutdown option should be updated (Simona Marcu)
  • [DONE] bug 848794 Can't re-run Firefox stub installer after cancelling Install (Virgil Dicu)
  • [DONE] bug 850077 - Tooltips do not work in the sidebar (Simona Marcu)
  • [DONE] bug 851436 - Pinned tabs sometimes scroll with normal tabs (Bogdan Maris)
  • [DONE] bug 851473 Restart dialogues say opposite feature enable/disable when toggling Private browsing (Ioana Budnar)
  • [DONE] bug 851495 Text shaping with AAT Khmer fonts broken in Firefox 19 (Virgil Dicu)
  • [DONE] bug 852530 - notificationBox is not visible on Linux (Bogdan Maris)
  • [DONE] bug 852569 Cannot uninstall provider when social is disabled (Ioana Budnar)
  • [DONE] bug 853387 - Crash in nsNPAPIPluginInstance::HandleEvent @ libnpgeplugin with Google Earth Plugin (Simona Marcu)

Update Tests

The following is the results of the Mozmill automated update tests.

Platform Betatest [dashboard] Releasetest [dashboard] Beta [dashboard]
Windows XP [DONE] 18.0b5 en-US & fr
[DONE] 19.0b5 en-US & ko
[DONE] 20.0b5 en-US & de
[DONE] 20.0b7 en-US & ru
[DONE] 18.0b5 en-US & fr
[DONE] 19.0b5 en-US & ko
[DONE] 20.0b5 en-US & de
[DONE] 20.0b7 en-US & ru
[DONE] 18.0b5 en-US & fr
[DONE] 19.0b5 en-US & ko
[DONE] 20.0b5 en-US & de
[DONE] 20.0b7 en-US & ru
Windows Vista [DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & pt-BR
[DONE] 20.0b4 en-US & it
[DONE] 20.0b7 en-US & es-ES
[DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & pt-BR
[DONE] 20.0b4 en-US & it
[DONE] 20.0b7 en-US & es-ES
[DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & pt-BR
[DONE] 20.0b4 en-US & it
[DONE] 20.0b7 en-US & es-ES
Windows 7 32-bit [DONE] 18.0b2 en-US & es-ES
[DONE] 19.0b2 en-US & zh-TW
[DONE] 20.0b2 en-US & fr
[DONE] 20.0b7 en-US & da
[DONE] 18.0b2 en-US & es-ES
[DONE] 19.0b2 en-US & zh-TW
[DONE] 20.0b2 en-US & fr
[DONE] 20.0b7 en-US & da
[DONE] 18.0b2 en-US & es-ES
[DONE] 19.0b2 en-US & zh-TW
[DONE] 20.0b2 en-US & fr
[DONE] 20.0b7 en-US & da
Windows 7 64-bit [DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & de
[DONE] 20.0b3 en-US & ko
[DONE] 20.0b7 en-US & ro
[DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & de
[DONE] 20.0b3 en-US & ko
[DONE] 20.0b7 en-US & ro
[DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & de
[DONE] 20.0b3 en-US & ko
[DONE] 20.0b7 en-US & ro
Windows 8 32-bit [DONE] 18.0b6 en-US & es-ES
[DONE] 19.0b6 en-US & zh-TW
[DONE] 20.0b6 en-US & ro
[DONE] 20.0b7 en-US & da
[DONE] 18.0b6 en-US & es-ES
[DONE] 19.0b6 en-US & zh-TW
[DONE] 20.0b6 en-US & ro
[DONE] 20.0b7 en-US & da
[DONE] 18.0b6 en-US & es-ES
[DONE] 19.0b6 en-US & zh-TW
[DONE] 20.0b6 en-US & ro
[DONE] 20.0b7 en-US & da
Windows 8 64-bit [DONE] 18.0b1 en-US & fr
[DONE] 19.0b1 en-US & it
[DONE] 20.0b1 en-US & es-ES
[DONE] 20.0b7 en-US & zh-TW
[DONE] 18.0b1 en-US & fr
[DONE] 19.0b1 en-US & it
[DONE] 20.0b1 en-US & es-ES
[DONE] 20.0b7 en-US & zh-TW
[DONE] 18.0b1 en-US & fr
[DONE] 19.0b1 en-US & it
[DONE] 20.0b1 en-US & es-ES
[DONE] 20.0b7 en-US & zh-TW
Mac OSX 10.6 [DONE] 18.0b1 en-US & de
[DONE] 19.0b1 en-US & ja-JP-mac
[DONE] 20.0b1 en-US & it
[DONE] 20.0b7 en-US & es-ES
[DONE] 18.0b1 en-US & de
[DONE] 19.0b1 en-US & ja-JP-mac
[DONE] 20.0b1 en-US & it
[DONE] 20.0b7 en-US & es-ES
[DONE] 18.0b1 en-US & de
[DONE] 19.0b1 en-US & ja-JP-mac
[DONE] 20.0b1 en-US & it
[DONE] 20.0b7 en-US & es-ES
Mac OSX 10.7 [DONE] 18.0b2 en-US & pt-BR
[DONE] 19.0b2 en-US & fr
[DONE] 20.0b2 en-US & ru
[DONE] 20.0b7 en-US & zh-TW
[DONE] 18.0b2 en-US & pt-BR
[DONE] 19.0b2 en-US & fr
[DONE] 20.0b2 en-US & ru
[DONE] 20.0b7 en-US & zh-TW
[DONE] 18.0b2 en-US & pt-BR
[DONE] 19.0b2 en-US & fr
[DONE] 20.0b2 en-US & ru
[DONE] 20.0b7 en-US & zh-TW
Mac OSX 10.8 [DONE] 18.0b3 en-US & it
[DONE] 19.0b3 en-US & ro
[DONE] 20.0b3 en-US & de
[DONE] 20.0b7 en-US & ko
[DONE] 18.0b3 en-US & it
[DONE] 19.0b3 en-US & ro
[DONE] 20.0b3 en-US & de
[DONE] 20.0b7 en-US & ko
[DONE] 18.0b3 en-US & it
[DONE] 19.0b3 en-US & ro
[DONE] 20.0b3 en-US & de
[DONE] 20.0b7 en-US & ko
Ubuntu 12.04 32-bit [DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & ko
[DONE] 20.0b4 en-US & zh-TW
[DONE] 20.0b7 en-US & ru
[DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & ko
[DONE] 20.0b4 en-US & zh-TW
[DONE] 20.0b7 en-US & ru
[DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & ko
[DONE] 20.0b4 en-US & zh-TW
[DONE] 20.0b7 en-US & ru
Ubuntu 12.04 64-bit [DONE] 18.0b5 en-US & es-ES
[DONE] 19.0b5 en-US & pt-BR
[DONE] 20.0b5 en-US & de
[DONE] 20.0b7 en-US & it
[DONE] 18.0b5 en-US & es-ES
[DONE] 19.0b5 en-US & pt-BR
[DONE] 20.0b5 en-US & de
[DONE] 20.0b7 en-US & it
[DONE] 18.0b5 en-US & es-ES
[DONE] 19.0b5 en-US & pt-BR
[DONE] 20.0b5 en-US & de
[DONE] 20.0b7 en-US & it
Bugs
  • VERIFIED FIXED bug 857380 21b1 update tests failing with NS_ERROR_UNEXPECTED in nsIFileOutputStream.init

New Issues

The following is a list of bugs found while testing this Beta. If you report a bug to bugzilla please list it below.

  • bug 857096 Java drop down list not displayed properly

Beta 2

Builds FTP, HTTP
ID 20130408165307
Changeset c4dfe07f855c (pushlog)
Checklist
  • [DONE] Automated Tests
  • [DONE] Regression Tests
  • [DONE] Bug Fix Verifications
  • [DONE] Update Tests (betatest)
  • [DONE] Update Tests (releasetest)
  • {{ |}} Update Tests (beta)

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform Results [dashboard]
Windows XP [DONE] en-US & da
Windows Vista [DONE] en-US & it
Windows 7 32-bit [DONE] en-US & zh-TW
Windows 7 64-bit [DONE] en-US & ro
Windows 8 32-bit [DONE] en-US & es-ES
Windows 8 64-bit [DONE] en-US & pt-BR
Mac OSX 10.6 [DONE] en-US & de
Mac OSX 10.7 [DONE] en-US & fr
Mac OSX 10.8 [DONE] en-US & it
Ubuntu 12.04 32-bit [DONE] en-US & de
Ubuntu 12.04 64-bit [DONE] en-US & fr

Known test failures in bugzilla

Regression Tests

As per the checklist we will regression test the following areas.

Regression testrun in Moztrap

Test Windows 7 64-bit Windows 8 32-bit Mac OSX 10.7 Ubuntu 12.04 32-bit
Breakpad end-to-end functionality (run test in Moztrap) [DONE] Simona [DONE] Bogdan [DONE] Manuela [DONE] Mihaela
MD5 as a hash algorithm in signatures (run test in Moztrap) [DONE] Simona [DONE] Bogdan [DONE] Manuela [DONE] Mihaela
Flash 11.7 Beta (download and test Flash websites) [DONE] Simona [DONE] Bogdan [DONE] Manuela [DROPPED] Mihaela
Exploratory testing involving UNC path names [DONE] Simona [DONE] Bogdan [DONE] Manuela [DROPPED] Mihaela (*)

(*) Cannot access Network from the Choose Folder dialog (tried on several machines and also with other SW - open file in Google Chrome)

Notes on UNC Testing
  • UNC testing can be done by using a profile located on a Windows share (use Choose Folder in the Profile Manager)
  • Also test with a Windows system configured to use Roaming Profiles if at all possible
  • Behaviours to test include basic navigation, bookmarks, history, session restore, search and search engine management, profile migration, cookies, add-ons, software update, etc
  • Include other types of network sharing (smb, afp, USB drive) to run profiles from

Bug Fix Verifications

The following bugs have been fixed and need confirmation:

  • [ON TRACK] bug 799523 [Retina] Crash on entering full screen mode in OS X (Matt Wobensmith)
    • Pending NEEDINFO? from Jonathan Kew
  • [MISSED] bug 857199 Please start offering the stub installer for English/Win32 on Beta (Cornel)
    • Issue is not yet fixed
  • [DONE] bug 846848 [OS.File] Handle UNC-formatted paths (Paul)
  • [DONE] bug 857672 Address Bar not working (Paul)
  • [DONE] bug 857588 Search engine list empty when the profile is on a network drive (Paul)
  • [DONE] bug 858704 provider manifests (ashughes)
  • [DONE] bug 858305 Need change to abouthealth.css to work around lack of flexbox support (Alexandra)
  • [DONE] bug 827979 add Growl support back (Cornel)
  • [DONE] bug 828815 SVGRectElement["onxxx"] = Function does not work, No Tooltip on SVG graphic (StatCounter) (Cornel)
  • [DONE] bug 830066 Drag & drop links into toolkit downloads manager to download files fails (TypeError: aInitiatingDocument is undefined) (Cornel)
  • [DONE] bug 827954 Firefox button overlaps the first tab in private windows (Cornel)
  • [DONE] bug 827460Copy/paste of "About Firefox" window's build version text no longer works when the window is closed (Alexandra)
  • [DONE] bug 834075 pdf.js: The PDF url in the address bar becomes wrong in a certain situation. (Alexandra)
  • [DONE] bug 831472 Crash [@ nsGlobalWindow::GetInnerScreenRect()] due to lack of null check (Alexandra)
  • [DONE] bug 829870 Appear the non-expected color gap on window that are opened from private one. (Alexandra)
  • [DONE] bug 842290 "Restart Aurora" pop up doesn't appear when selecting private browsing mode (Alexandra)
  • [DONE] bug 857291 SPNEGO / MS KRB5 no longer working. Tries to use NTLM SSP instead (reporter)
  • [DONE] bug 844406 Greasemonkey script stops working correctly in Firefox 20. 'self-hosted' builtin functions destroy js-callstack in certain case. (Manuela)

Update Tests

The following is the results of the Mozmill automated update tests.

Platform Betatest [dashboard] Releasetest [dashboard] Beta [dashboard]
Windows XP [DONE] 18.0b5 en-US & fr
[DONE] 19.0b5 en-US & ko
[DONE] 20.0b5 en-US & de
[DONE] 21.0b1 en-US & ru
[DONE] 18.0b5 en-US & fr
[DONE] 19.0b5 en-US & ko
[DONE] 20.0b5 en-US & de
[DONE] 21.0b1 en-US & ru
[DONE] 18.0b5 en-US & fr
[DONE] 19.0b5 en-US & ko
[DONE] 20.0b5 en-US & de
[DONE] 21.0b1 en-US & ru
Windows Vista [DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & pt-BR
[DONE] 20.0b4 en-US & it
[DONE] 21.0b1 en-US & es-ES
[DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & pt-BR
[DONE] 20.0b4 en-US & it
[DONE] 21.0b1 en-US & es-ES
[DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & pt-BR
[DONE] 20.0b4 en-US & it
[DONE] 21.0b1 en-US & es-ES
Windows 7 32-bit [DONE] 18.0b2 en-US & es-ES
[DONE] 19.0b2 en-US & zh-TW
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b1 en-US & da
[DONE] 18.0b2 en-US & es-ES
[DONE] 19.0b2 en-US & zh-TW
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b1 en-US & da
[DONE] 18.0b2 en-US & es-ES
[DONE] 19.0b2 en-US & zh-TW
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b1 en-US & da
Windows 7 64-bit [DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & de
[DONE] 20.0b3 en-US & ko
[DONE] 21.0b1 en-US & ro
[DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & de
[DONE] 20.0b3 en-US & ko
[DONE] 21.0b1 en-US & ro
[DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & de
[DONE] 20.0b3 en-US & ko
[DONE] 21.0b1 en-US & ro
Windows 8 32-bit [DONE] 18.0b6 en-US & es-ES
[DONE] 19.0b6 en-US & zh-TW
[DONE] 20.0b6 en-US & ro
[DONE] 21.0b1 en-US & da
[DONE] 18.0b6 en-US & es-ES
[DONE] 19.0b6 en-US & zh-TW
[DONE] 20.0b6 en-US & ro
[DONE] 21.0b1 en-US & da
[DONE] 18.0b6 en-US & es-ES
[DONE] 19.0b6 en-US & zh-TW
[DONE] 20.0b6 en-US & ro
[DONE] 21.0b1 en-US & da
Windows 8 64-bit [DONE] 18.0b1 en-US & fr
[DONE] 19.0b1 en-US & it
[DONE] 20.0b1 en-US & es-ES
[DONE] 21.0b1 en-US & zh-TW
[DONE] 18.0b1 en-US & fr
[DONE] 19.0b1 en-US & it
[DONE] 20.0b1 en-US & es-ES
[DONE] 21.0b1 en-US & zh-TW
[DONE] 18.0b1 en-US & fr
[DONE] 19.0b1 en-US & it
[DONE] 20.0b1 en-US & es-ES
[DONE] 21.0b1 en-US & zh-TW
Mac OSX 10.6 [DONE] 18.0b1 en-US & de
[DONE] 19.0b1 en-US & ja-JP-mac
[DONE] 20.0b1 en-US & it
[DONE] 21.0b1 en-US & es-ES
[DONE] 18.0b1 en-US & de
[DONE] 19.0b1 en-US & ja-JP-mac
[DONE] 20.0b1 en-US & it
[DONE] 21.0b1 en-US & es-ES
[DONE] 18.0b1 en-US & de
[DONE] 19.0b1 en-US & ja-JP-mac
[DONE] 20.0b1 en-US & it
[DONE] 21.0b1 en-US & es-ES
Mac OSX 10.7 [DONE] 18.0b2 en-US & pt-BR
[DONE] 19.0b2 en-US & fr
[DONE] 20.0b2 en-US & ru
[DONE] 21.0b1 en-US & zh-TW
[DONE] 18.0b2 en-US & pt-BR
[DONE] 19.0b2 en-US & fr
[DONE] 20.0b2 en-US & ru
[DONE] 21.0b1 en-US & zh-TW
[DONE] 18.0b2 en-US & pt-BR
[DONE] 19.0b2 en-US & fr
[DONE] 20.0b2 en-US & ru
[DONE] 21.0b1 en-US & zh-TW
Mac OSX 10.8 [DONE] 18.0b3 en-US & it
[DONE] 19.0b3 en-US & ro
[DONE] 20.0b3 en-US & de
[DONE] 21.0b1 en-US & ko
[DONE] 18.0b3 en-US & it
[DONE] 19.0b3 en-US & ro
[DONE] 20.0b3 en-US & de
[DONE] 21.0b1 en-US & ko
[DONE] 18.0b3 en-US & it
[DONE] 19.0b3 en-US & ro
[DONE] 20.0b3 en-US & de
[DONE] 21.0b1 en-US & ko
Ubuntu 12.04 32-bit [DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & ko
[DONE] 20.0b4 en-US & zh-TW
[DONE] 21.0b1 en-US & ru
[DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & ko
[DONE] 20.0b4 en-US & zh-TW
[DONE] 21.0b1 en-US & ru
[DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & ko
[DONE] 20.0b4 en-US & zh-TW
[DONE] 21.0b1 en-US & ru
Ubuntu 12.04 64-bit [DONE] 18.0b5 en-US & es-ES
[DONE] 19.0b5 en-US & pt-BR
[DONE] 20.0b5 en-US & de
[DONE] 21.0b1 en-US & it
[DONE] 18.0b5 en-US & es-ES
[DONE] 19.0b5 en-US & pt-BR
[DONE] 20.0b5 en-US & de
[DONE] 21.0b1 en-US & it
[DONE] 18.0b5 en-US & es-ES
[DONE] 19.0b5 en-US & pt-BR
[DONE] 20.0b5 en-US & de
[DONE] 21.0b1 en-US & it
Bugs
  • [FIXED] bug 859977 Firefox 21.0b2 Windows 7 fallback update tests failing with "channelprefs.js not found"
    • Mitigated by manually spotchecking Windows 7 fall-back updates

New Issues

The following is a list of bugs found while testing this Beta. If you report a bug to bugzilla please list it below.

  • bug 859977 Firefox 21.0b2 Windows 7 fallback update tests failing with "channelprefs.js not found"

Beta 3

Builds FTP, HTTP
ID 20130416200523
Changeset e845a10035f2 (pushlog)
Checklist
  • [DONE] Automated Tests
  • [DONE] Regression Tests
  • [DONE] Bug Fix Verifications
  • [DONE] Update Tests (betatest)
  • [DONE] Update Tests (releasetest)
  • [DONE] Update Tests (beta)

Automated Testing

The following are the detailed results of the automated Mozmill test runs. These tests are run against the en-US and one random locale candidate per platform.

Platform Results [dashboard]
Windows XP [DONE] en-US & da
Windows Vista [DONE] en-US & ro
Windows 7 32-bit [DONE] en-US & pt-BR
Windows 7 64-bit [DONE] en-US & ru
Windows 8 32-bit [DONE] en-US & zh-TW
Windows 8 64-bit [DONE] en-US & ko
Mac OSX 10.6 [DONE] en-US & de
Mac OSX 10.7 [DONE] en-US & ja-JP-mac
Mac OSX 10.8 [DONE] en-US & it
Ubuntu 12.04 32-bit [DONE] en-US & fr
Ubuntu 12.04 64-bit [DONE] en-US & es-ES

Known test failures in bugzilla

Regression Tests

As per the checklist we will regression test the following areas.

Regression testrun in Moztrap

Test Windows 7 32-bit Windows Vista Mac OSX 10.6 Ubuntu 12.04 32-bit
Breakpad end-to-end functionality (run test in Moztrap) [DONE] Manuela [DONE] Simona [DONE] Cornel [DONE] Virgil
Mac OSX 32-bit mode (run test in Moztrap) N/A Manuela N/A Simona [DONE] Cornel N/A Virgil
Web compatibility (search, webmail, video/audio, maps, cms, etc) [DONE] Manuela [DONE] Simona [DONE] Cornel [DONE] Virgil
Exploratory testing around Private Browsing and Session Restore [DONE] Manuela [DONE] Simona [DONE] Cornel [DONE] Virgil
Exploratory testing around Mozmill failures (en-US & l10n) Windows XP Windows 8 32-bit Mac OSX 10.7 Ubuntu 12.04 32-bit
Search and search selection [DONE] Simona [DONE] Manuela [DONE] Cornel [DONE] Virgil
Safe browsing warning pages [DONE] Simona [DONE] Manuela [DONE] Cornel [DONE] Virgil
Bookmarks, including default bookmarks [DONE] Simona [DONE] Manuela [DONE] Cornel [DONE] Virgil

Bug Fix Verifications

The following bugs have been fixed and need confirmation:

Unassigned:

  • n/a


[BLOCKED]

  • bug 840881 Wrong menu rendering with layout.css.devPixelsPerPx non-default (HI-DPI displays) - Blocked by bug 861270


[ON TRACK]

  • bug 799523 [Retina] Crash on entering full screen mode in OS X (Matt Wobensmith)
  • bug 857199 Please start offering the stub installer for English/Win32 on Beta (Paul Silaghi)


[DONE]

  • bug 797288 Pdf.js download behavior inconsistent (Ioana Budnar)
  • bug 822933 WebM playback regression (Paul Silaghi)
  • bug 825035 Blocklisted click-to-play notification only fades in/out once per window per session (Paul Silaghi)
  • bug 825620 Crash Reporter's locale different from the build's one (Alexandra Lucinet)
  • bug 833823 Random, unpredictable behavior with multiple YouTube embeds (Paul Silaghi)
  • bug 837845 Multi-GB memory spike when using regular expressions (Bogdan Maris)
  • bug 839031 crash in mozilla::wmf::MFCreateSourceReaderFromByteStream @ mfflac with FLAC 1.2.0 (Bogdan Maris)
  • bug 840739 Flash links in private windows are converting normal windows into private windows. (Bogdan Maris)
  • bug 842577 Remove from History is visible for a paused download (Ioana Budnar)
  • bug 843342 PDF doesn't print correctly on USPS.com with pdfjs (Alexandra Lucinet)
  • bug 845681 Closing all windows and quitting Firefox afterwards leads to about:sessionrestore to be shown (Paul Silaghi)
  • bug 852295 LW Themes do not display in Permanent Private Browsing mode post Per-Window PB (Ioana Budnar)
  • bug 854175 permanent PB mode adds "(Private Browsing)" to window title (Ioana Budnar)
  • bug 854926 Firefox prompts a warning when closing private windows with multiple tabs open (Ioana Budnar)
  • bug 856784 crash in gfxUserFontSet::UserFontCache::ForgetFont (Ioana Budnar)
  • bug 819377 Hang on https://www.mozilla.org/en-US/contribute/ when playing video (Bogdan Maris)
  • bug 844031 Fail to detect the correct version of Intel graphics driver on Windows (Alexandra Lucinet)
  • bug 839515 Tiny fonts in the remote connection screen (Ioana Budnar)
  • bug 861188 Firefox 20 makes svg-edit place opened SVG images in lower-right corner instead of center (Paul Silaghi)

Update Tests

The following is the results of the Mozmill automated update tests.

Platform Betatest [dashboard] Releasetest [dashboard] Beta [dashboard]
Windows XP [DONE] 17.0b6 en-US & fr
[DONE] 18.0b1 en-US & ko
[DONE] 19.0b2 en-US & dr
[DONE] 20.0b2 en-US & ru
[DONE] 17.0b6 en-US & fr
[DONE] 18.0b1 en-US & ko
[DONE] 19.0b2 en-US & dr
[DONE] 20.0b2 en-US & ru
[DONE] 17.0b6 en-US & fr
[DONE] 18.0b1 en-US & ko
[DONE] 19.0b2 en-US & dr
[DONE] 20.0b2 en-US & ru
Windows Vista [DONE] 17.0b5 en-US & de
[DONE] 18.0b6 en-US & pt-BR
[DONE] 19.0b1 en-US & it
[DONE] 21.0b2 en-US & es-ES
[DONE] 17.0b5 en-US & de
[DONE] 18.0b6 en-US & pt-BR
[DONE] 19.0b1 en-US & it
[DONE] 21.0b2 en-US & es-ES
[DONE] 17.0b5 en-US & de
[DONE] 18.0b6 en-US & pt-BR
[DONE] 19.0b1 en-US & it
[DONE] 21.0b2 en-US & es-ES
Windows 7 32-bit [DONE] 17.0b3 en-US es-ES
[DONE] 18.0b4 en-US zh-TW
[DONE] 19.0b5 en-US fr
[DONE] 21.0b2 en-US da
[DONE] 17.0b3 en-US es-ES
[DONE] 18.0b4 en-US zh-TW
[DONE] 19.0b5 en-US fr
[DONE] 21.0b2 en-US da
[DONE] 17.0b3 en-US es-ES
[DONE] 18.0b4 en-US zh-TW
[DONE] 19.0b5 en-US fr
[DONE] 21.0b2 en-US da
Windows 7 64-bit [DONE] 17.0b4 en-US es-ES
[DONE] 18.0b5 en-US de
[DONE] 19.0b6 en-US ko
[DONE] 21.0b2 en-US ro
[DONE] 17.0b4 en-US es-ES
[DONE] 18.0b5 en-US de
[DONE] 19.0b6 en-US ko
[DONE] 21.0b2 en-US ro
[DONE] 17.0b4 en-US es-ES
[DONE] 18.0b5 en-US de
[DONE] 19.0b6 en-US ko
[DONE] 21.0b2 en-US ro
Windows 8 32-bit [DONE] 17.0b1 en-US es-ES
[DONE] 18.0b2 en-US zh-TW
[DONE] 19.0b3 en-US ro
[DONE] 21.0b2 en-US da
[DONE] 17.0b1 en-US es-ES
[DONE] 18.0b2 en-US zh-TW
[DONE] 19.0b3 en-US ro
[DONE] 21.0b2 en-US da
[DONE] 17.0b1 en-US es-ES
[DONE] 18.0b2 en-US zh-TW
[DONE] 19.0b3 en-US ro
[DONE] 21.0b2 en-US da
Windows 8 64-bit [DONE] 17.0b2 en-US fr
[DONE] 18.0b3 en-US it
[DONE] 19.0b4 en-US es-ES
[DONE] 21.0b2 en-US zh-TW
[DONE] 17.0b2 en-US fr
[DONE] 18.0b3 en-US it
[DONE] 19.0b4 en-US es-ES
[DONE] 21.0b2 en-US zh-TW
[DONE] 17.0b2 en-US fr
[DONE] 18.0b3 en-US it
[DONE] 19.0b4 en-US es-ES
[DONE] 21.0b2 en-US zh-TW
Mac OSX 10.6 [DONE] 17.0b1 en-US de
[DONE] 18.0b2 en-US ja-JP-mac
[DONE] 19.0b3 en-US it
[DONE] 21.0b2 en-US es-ES
[DONE] 17.0b1 en-US de
[DONE] 18.0b2 en-US ja-JP-mac
[DONE] 19.0b3 en-US it
[DONE] 21.0b2 en-US es-ES
[DONE] 17.0b1 en-US de
[DONE] 18.0b2 en-US ja-JP-mac
[DONE] 19.0b3 en-US it
[DONE] 21.0b2 en-US es-ES
Mac OSX 10.7 [DONE] 17.0b2 en-US pt-BR
[DONE] 18.0b3 en-US fr
[DONE] 19.0b4 en-US ru
[DONE] 21.0b2 en-US zh-TW
[DONE] 17.0b2 en-US pt-BR
[DONE] 18.0b3 en-US fr
[DONE] 19.0b4 en-US ru
[DONE] 21.0b2 en-US zh-TW
[DONE] 17.0b2 en-US pt-BR
[DONE] 18.0b3 en-US fr
[DONE] 19.0b4 en-US ru
[DONE] 21.0b2 en-US zh-TW
Mac OSX 10.8 [DONE] 17.0b3 en-US it
[DONE] 18.0b4 en-US ro
[DONE] 19.0b5 en-US de
[DONE] 21.0b2 en-US ko
[DONE] 17.0b3 en-US it
[DONE] 18.0b4 en-US ro
[DONE] 19.0b5 en-US de
[DONE] 21.0b2 en-US ko
[DONE] 17.0b3 en-US it
[DONE] 18.0b4 en-US ro
[DONE] 19.0b5 en-US de
[DONE] 21.0b2 en-US ko
Ubuntu 12.04 32-bit [DONE] 17.0b5 en-US de
[DONE] 18.0b6 en-US ko
[DONE] 19.0b1 en-US zh-TW
[DONE] 21.0b2 en-US ru
[DONE] 17.0b5 en-US de
[DONE] 18.0b6 en-US ko
[DONE] 19.0b1 en-US zh-TW
[DONE] 21.0b2 en-US ru
[DONE] 17.0b5 en-US de
[DONE] 18.0b6 en-US ko
[DONE] 19.0b1 en-US zh-TW
[DONE] 21.0b2 en-US ru
Ubuntu 12.04 64-bit [DONE] 17.0b6 en-US es-ES
[DONE] 18.0b1 en-US pt-BR
[DONE] 19.0b2 en-US de
[DONE] 21.0b2 en-US it
[DONE] 17.0b6 en-US es-ES
[DONE] 18.0b1 en-US pt-BR
[DONE] 19.0b2 en-US de
[DONE] 21.0b2 en-US it
[DONE] 17.0b6 en-US es-ES
[DONE] 18.0b1 en-US pt-BR
[DONE] 19.0b2 en-US de
[DONE] 21.0b2 en-US it

New Issues

The following is a list of bugs found while testing this Beta. If you report a bug to bugzilla please list it below.

  • bug 863183 The remote debugger "Connecting..." string and animation should be larger
  • bug 863238 Not localized mozilla pages opened from the Firefox options
  • bug 863256 Report Web Forgery warning is not always displayed

Beta 4

Builds FTP, HTTP
ID 20130423212553
Changeset changeset (pushlog)
Checklist
  • [DONE] Automated Tests
  • [DONE] Regression Tests
  • [DONE] Bug Fix Verifications
  • [DONE] Update Tests (betatest)
  • [DONE] Update Tests (releasetest)
  • [DONE] Update Tests (beta)

Automated Testing

The following are the detailed results of the automated Mozmill test runs. These tests are run against the en-US and one random locale candidate per platform.

Platform Results [dashboard]
Windows XP [DONE] en-US & pl & ro
Windows Vista [DONE] en-US & pl & ko
Windows 7 32-bit [DONE] en-US & pl & da
Windows 7 64-bit [DONE] en-US & pl & it
Windows 8 32-bit [DONE] en-US & pl & ru
Windows 8 64-bit [DONE] en-US & pl & ja-JP
Mac OSX 10.6 [DONE] en-US & pl & de
Mac OSX 10.7 [DONE] en-US & pl & es-ES
Mac OSX 10.8 [DONE] en-US & pl & fr
Ubuntu 12.04 32-bit [DONE] en-US & pl & pt-BR
Ubuntu 12.04 64-bit [DONE] en-US & pl & zh-TW

Known test failures in bugzilla

Regression Tests

As per the checklist we will regression test the following areas.

Regression testrun in Moztrap

Test Windows 7 64-bit Windows XP Mac OSX 10.8 Ubuntu 12.10 32-bit
Breakpad end-to-end functionality (run test in Moztrap) [DONE] Paul [DONE] Bogdan [DONE] Virgil [DONE] Manuela
Verify update applies with root owned files (use betatest channel) [DONE] Paul [DONE] Bogdan [DONE] Virgil [DONE] Manuela
Add-on compatibility (top third-party and addons.mozilla.org hosted) [DONE] Paul [DONE] Bogdan [DONE] Virgil [DONE] Manuela
  • Web compatibility exploratory testing around drop-down select menus and autocomplete forms etherpad

Bug Fix Verifications

The following bugs have been fixed and need confirmation.

Bug Assignee Status
bug 862606 [summary hidden] Matt Wobensmith [ON TRACK]
bug 861606 Add some assertions and guards for dir=auto (see also bug 861607, bug 861610) Anthony Hughes [DROPPED]
bug 861455 Daily collection when upload is disabled does not collect pull only providers Anthony Hughes [ON TRACK]
bug 861270 menu behavior is unstable depending on value of devPixelsPerPx (see also bug 840881) Cornel Ionce [DONE]
bug 860180 U+30D8 causes XML error in UTF-16BE Anthony Hughes [DROPPED]
bug 859172 Firefox now trying to mistakenly decode non-UTF8 links as UTF-8 in UTF-8 webpages Alexandra Lucinet [DONE]
bug 858459 drop-down select menu closes without selecting in Firefox 20.0 Ioana Budnar [DONE]
bug 857817 [NBV] Consumes all client resources when load is slightly increased Alexandra Lucinet [DONE]
bug 857199 Please start offering the stub installer for English/Win32 on Beta Paul Silaghi [ON TRACK] Pending web-dev fixes
bug 855024 High-level collection APIs aren't safe when run in parallel Ioana Budnar [DROPPED]
bug 853291 Add Polish (pl) to gecko localizations, central, mozilla-b2g18, mozilla-b2g18_v1_0_1 Anthony Hughes [DONE]
bug 852315 [summary hidden] Matt Wobensmith [ON TRACK]
bug 851630 Can't open popup notifications with the keyboard (focus gets stuck at the identity icon) Cornel Ionce [DONE]
bug 848237 [summary hidden] Matt Wobensmith [DROPPED]
bug 845743 Firefox 19.0 does not support BIG5-HKSCS encoding. Version 18.0 does Simona Badau [DONE]
bug 845555 Context menu item to switch to hide statistics instead of offering to show statistics again Cornel Ionce [DONE]
bug 838864 keyword.URL reset interferes poorly with some partner add-ons Manuela Muntean [DONE]
bug 838239 [hidpi] wrong devicePixelRatio in browser history Matt Wobensmith [DONE]
bug 826349 video unmutes itself after seeking Cornel Ionce [DONE]
bug 825545 crash in js_NewStringCopyN Simona Badau [DONE]
bug 816498 Table backgrounds don't print, starting from FX17 Simona Badau [DONE]
bug 799523 [Retina] Crash on entering full screen mode in OS X Matt Wobensmith [DONE]
Totals
  • 64% [DONE]
  • 18% [DROPPED]
  • 18% [ON TRACK]

Update Tests

The following is the results of the Mozmill automated update tests.

Platform Betatest [dashboard] Releasetest [dashboard] Beta [dashboard]
Windows XP [DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b3 en-US & ru
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b3 en-US & ru
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b3 en-US & ru
Windows Vista [DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & pt-BR
[DONE] 21.0b1 en-US & es-ES
[DONE] 21.0b3 en-US & ru
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & pt-BR
[DONE] 21.0b1 en-US & es-ES
[DONE] 21.0b3 en-US & ru
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & pt-BR
[DONE] 21.0b1 en-US & es-ES
[DONE] 21.0b3 en-US & ru
Windows 7 32-bit [DONE] 19.0b2 en-US & es-ES
[DONE] 20.0b2 en-US & zh-TW
[DONE] 21.0b1 en-US & fr
[DONE] 21.0b3 en-US & da
[DONE] 19.0b2 en-US & es-ES
[DONE] 20.0b2 en-US & zh-TW
[DONE] 21.0b1 en-US & fr
[DONE] 21.0b3 en-US & da
[DONE] 19.0b2 en-US & es-ES
[DONE] 20.0b2 en-US & zh-TW
[DONE] 21.0b1 en-US & fr
[DONE] 21.0b3 en-US & da
Windows 7 64-bit [DONE] 19.0b3 en-US & es-ES
[DONE] 20.0b3 en-US & de
[DONE] 21.0b2 en-US & ko
[DONE] 21.0b3 en-US & ro
[DONE] 19.0b3 en-US & es-ES
[DONE] 20.0b3 en-US & de
[DONE] 21.0b2 en-US & ko
[DONE] 21.0b3 en-US & ro
[DONE] 19.0b3 en-US & es-ES
[DONE] 20.0b3 en-US & de
[DONE] 21.0b2 en-US & ko
[DONE] 21.0b3 en-US & ro
Windows 8 32-bit [DONE] 19.0b6 en-US & es-ES
[DONE] 20.0b6 en-US & zh-TW
[DONE] 21.0b1 en-US ro
[DONE] 21.0b3 en-US da
[DONE] 19.0b6 en-US & es-ES
[DONE] 20.0b6 en-US & zh-TW
[DONE] 21.0b1 en-US ro
[DONE] 21.0b3 en-US da
[DONE] 19.0b6 en-US & es-ES
[DONE] 20.0b6 en-US & zh-TW
[DONE] 21.0b1 en-US ro
[DONE] 21.0b3 en-US da
Windows 8 64-bit [DONE] 19.0b1 en-US & fr
[DONE] 20.0b1 en-US & it
[DONE] 21.0b2 en-US & es-ES
[DONE] 21.0b3 en-US & zh-TW
[DONE] 19.0b1 en-US & fr
[DONE] 20.0b1 en-US & it
[DONE] 21.0b2 en-US & es-ES
[DONE] 21.0b3 en-US & zh-TW
[DONE] 19.0b1 en-US & fr
[DONE] 20.0b1 en-US & it
[DONE] 21.0b2 en-US & es-ES
[DONE] 21.0b3 en-US & zh-TW
Mac OSX 10.6 [DONE] 19.0b1 en-US & de
[DONE] 20.0b1 en-US & ja-JP-mac
[DONE] 21.0b1 en-US & it
[DONE] 21.0b3 en-US & es-ES
[DONE] 19.0b1 en-US & de
[DONE] 20.0b1 en-US & ja-JP-mac
[DONE] 21.0b1 en-US & it
[DONE] 21.0b3 en-US & es-ES
[DONE] 19.0b1 en-US & de
[DONE] 20.0b1 en-US & ja-JP-mac
[DONE] 21.0b1 en-US & it
[DONE] 21.0b3 en-US & es-ES
Mac OSX 10.7 [DONE] 19.0b2 en-US & pt-BR
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b2 en-US & ru
[DONE] 21.0b3 en-US & zh-TW
[DONE] 19.0b2 en-US & pt-BR
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b2 en-US & ru
[DONE] 21.0b3 en-US & zh-TW
[DONE] 19.0b2 en-US & pt-BR
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b2 en-US & ru
[DONE] 21.0b3 en-US & zh-TW
Mac OSX 10.8 [DONE] 19.0b3 en-US & it
[DONE] 20.0b3 en-US & ro
[DONE] 21.0b2 en-US & de
[DONE] 21.0b3 en-US & ko
[DONE] 19.0b3 en-US & it
[DONE] 20.0b3 en-US & ro
[DONE] 21.0b2 en-US & de
[DONE] 21.0b3 en-US & ko
[DONE] 19.0b3 en-US & it
[DONE] 20.0b3 en-US & ro
[DONE] 21.0b2 en-US & de
[DONE] 21.0b3 en-US & ko
Ubuntu 12.04 32-bit [DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & ko
[DONE] 21.0b1 en-US & zh-TW
[DONE] 21.0b3 en-US & ru
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & ko
[DONE] 21.0b1 en-US & zh-TW
[DONE] 21.0b3 en-US & ru
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & ko
[DONE] 21.0b1 en-US & zh-TW
[DONE] 21.0b3 en-US & ru
Ubuntu 12.04 64-bit [DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b2 en-US & de
[DONE] 21.0b3 en-US & it
[DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b2 en-US & de
[DONE] 21.0b3 en-US & it
[DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b2 en-US & de
[DONE] 21.0b3 en-US & it

New Issues

The following is a list of bugs found while testing this Beta. If you report a bug to bugzilla please list it below.

  • bug 865267 Duplicate folders in bookmarks menu when one folder is dragged and dropped intro another
  • bug 865672 Number of bookmarks doesn't update in about:healthreport

Beta 5

ADVISORY: THIS IS EFFECTIVELY A RESPIN OF 21.0b4 TO ADDRESS A POTENTIAL STABILITY CONCERN

Builds FTP, HTTP
ID 20130425162858
Changeset changeset (pushlog)
Checklist
  • [DONE] Automated Tests
  • [DONE] Update Tests (betatest)
  • [DONE] Update Tests (releasetest)
  • [ON TRACK] Update Tests (beta)
  • Manual testing reduced to basic dogfooding since this is built from the same changeset as 21.0b4

Automated Testing

The following are the detailed results of the automated Mozmill test runs. These tests are run against the en-US and one random locale candidate per platform.

Platform Results [dashboard]
Windows XP [DONE] en-US & ro
Windows Vista [DONE] en-US & ko
Windows 7 32-bit [DONE] en-US & da
Windows 7 64-bit [DONE] en-US & it
Windows 8 32-bit [DONE] en-US & ru
Windows 8 64-bit [DONE] en-US & ja
Mac OSX 10.6 [DONE] en-US & de
Mac OSX 10.7 [DONE] en-US & es-ES
Mac OSX 10.8 [DONE] en-US & fr
Ubuntu 12.04 32-bit [DONE] en-US & pt-BR
Ubuntu 12.04 64-bit [DONE] en-US & zh-TW

Known test failures in bugzilla

Update Tests

The following is the results of the Mozmill automated update tests.

Platform Betatest [dashboard] Releasetest [dashboard] Beta [dashboard]
Windows XP [DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru
Windows Vista [DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & pt-BR
[DONE] 21.0b1 en-US & es-ES
[DONE] 21.0b4 en-US & ru
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & pt-BR
[DONE] 21.0b1 en-US & es-ES
[DONE] 21.0b4 en-US & ru
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru
Windows 7 32-bit [DONE] 19.0b2 en-US & es-ES
[DONE] 20.0b2 en-US & zh-TW
[DONE] 21.0b3 en-US & fr
[DONE] 21.0b4 en-US & da
[DONE] 19.0b2 en-US & es-ES
[DONE] 20.0b2 en-US & zh-TW
[DONE] 21.0b3 en-US & fr
[DONE] 21.0b4 en-US & da
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru
Windows 7 64-bit [DONE] 19.0b3 en-US & es-ES
[DONE] 20.0b3 en-US & de
[DONE] 21.0b2 en-US & ko
[DONE] 21.0b4 en-US & ro
[DONE] 19.0b3 en-US & es-ES
[DONE] 20.0b3 en-US & de
[DONE] 21.0b2 en-US & ko
[DONE] 21.0b4 en-US & ro
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru
Windows 8 32-bit [DONE] 19.0b6 en-US & es-ES
[DONE] 20.0b6 en-US & zh-TW
[DONE] 21.0b1 en-US ro
[DONE] 21.0b4 en-US da
[DONE] 19.0b6 en-US & es-ES
[DONE] 20.0b6 en-US & zh-TW
[DONE] 21.0b1 en-US ro
[DONE] 21.0b4 en-US da
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru
Windows 8 64-bit [DONE] 19.0b1 en-US & fr
[DONE] 20.0b1 en-US & it
[DONE] 21.0b3 en-US & es-ES
[DONE] 21.0b4 en-US & zh-TW
[DONE] 19.0b1 en-US & fr
[DONE] 20.0b1 en-US & it
[DONE] 21.0b3 en-US & es-ES
[DONE] 21.0b4 en-US & zh-TW
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru
Mac OSX 10.6 [DONE] 19.0b1 en-US & de
[DONE] 20.0b1 en-US & ja-JP-mac
[DONE] 21.0b1 en-US & it
[DONE] 21.0b4 en-US & es-ES
[DONE] 19.0b1 en-US & de
[DONE] 20.0b1 en-US & ja-JP-mac
[DONE] 21.0b1 en-US & it
[DONE] 21.0b4 en-US & es-ES
[DONE] 19.0b5 en-US & fr*
[DONE] 20.0b5 en-US & ko*
[DONE] 21.0b2 en-US & de*
[DONE] 21.0b4 en-US & ru*
Mac OSX 10.7 [DONE] 19.0b2 en-US & pt-BR
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b2 en-US & ru
[DONE] 21.0b4 en-US & zh-TW
[DONE] 19.0b2 en-US & pt-BR
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b2 en-US & ru
[DONE] 21.0b4 en-US & zh-TW
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru
Mac OSX 10.8 [DONE] 19.0b3 en-US & it
[DONE] 20.0b3 en-US & ro
[DONE] 21.0b3 en-US & de
[DONE] 21.0b4 en-US & ko
[DONE] 19.0b3 en-US & it
[DONE] 20.0b3 en-US & ro
[DONE] 21.0b3 en-US & de
[DONE] 21.0b4 en-US & ko
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru
Ubuntu 12.04 32-bit [DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & ko
[DONE] 21.0b1 en-US & zh-TW
[DONE] 21.0b4 en-US & ru
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & ko
[DONE] 21.0b1 en-US & zh-TW
[DONE] 21.0b4 en-US & ru
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru
Ubuntu 12.04 64-bit [DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & it
[DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & it
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru

1. Mac OSX 10.6 results were tested manually due to a Mozmill Infrastructure issue: bug 867005

Beta 6

Builds FTP, HTTP
ID 20130430204233
Changeset changeset (pushlog)
Checklist
  • [DONE] Automated Tests
  • [DONE] Regression Tests
  • [DONE] Bug Fix Verifications
  • [DONE] Update Tests (betatest)
  • [DONE] Update Tests (releasetest)
  • [DONE] Update Tests (beta)

Stability

Beta candidate crashes per 100 ADIs on ship day

  • 21.0b3: 4.2 (2013-04-18)
  • 21.0b5: 4.5 (2013-04-29)
  • 21.0b6: 1.5 (2013-05-02)

Automated Testing

The following are the detailed results of the automated Mozmill test runs. These tests are run against the en-US and one random locale candidate per platform.

Platform Results [dashboard]
Windows XP [DONE] en-US & ro
Windows Vista [DONE] en-US & ko
Windows 7 32-bit [DONE] en-US & da
Windows 7 64-bit [DONE] en-US & it
Windows 8 32-bit [DONE] en-US & ru
Windows 8 64-bit [DONE] en-US & ja-JP
Mac OSX 10.6 [DONE] en-US & de
Mac OSX 10.7 [DONE] en-US & es-ES
Mac OSX 10.8 [DONE] en-US & fr
Ubuntu 12.04 32-bit [DONE] en-US & pt-BR
Ubuntu 12.04 64-bit [DONE] en-US & zh-TW

Known test failures in bugzilla

Regression Tests

As per the checklist we will regression test the following areas.

Breakpad

Goal: Ensure that crash reports are submitted with the current build (use this Moztrap test as a rough guideline).

Windows XP Windows 8 64-bit Mac OSX 10.7 Ubuntu 13.04 32-bit
[DONE] Paul [DONE] Alexandra [DONE] Virgil [DONE] Mihaela
Issues:
  • No issues found

Plug-in Compatibility

Goal: Ensure plug-ins and plug-in content remain compatibile and stable in the current build.

Suggested Plugins:

  • Flash 11.7, 11.6, 11.2, and 10.3
  • Java 7, 6
  • Apple QuickTime
  • Microsoft Silverlight 5
  • Microsoft Windows Media Player
  • Flip4Mac
  • Unity 3D
  • Skype Click-to-Call
  • RealPlayer


Plug-in Windows XP Windows 8 64-bit Mac OSX 10.7 Ubuntu 13.04 32-bit
Flash Player 10.3 [DONE] Paul [DONE] Alexandra [DONE] Virgil N/A Mihaela
Flash Player 11.2 [DONE] Paul [DONE] Alexandra [DONE] Virgil [DONE] Mihaela
Flash Player 11.6 [DONE] Paul [DONE] Alexandra [DONE] Virgil N/A Mihaela
Flash Player 11.7 [DONE] Paul [DONE] Alexandra [DONE] Virgil N/A Mihaela
Java 6 [DONE] Paul [DONE] Alexandra N/A Virgil [DONE] Mihaela
Java 7 [DONE] Paul [DONE] Alexandra [DONE] Virgil [DONE] Mihaela
Apple QuickTime [DONE] Paul [DONE] Alexandra [DONE] Virgil [DONE] Mihaela
Microsoft Silverlight 5 [DONE] Paul [DONE] Alexandra [DONE] Virgil [DONE] Mihaela
Microsoft Windows Media Player [DONE] Paul N/A Alexandra [DONE] [using flip4mac] Virgil [DONE] Mihaela
Unity 3D [DONE] Paul [DONE] Alexandra [DONE] Virgil N/A Mihaela
Skype Click-to-Call [DONE] Paul N/A Alexandra N/A Virgil N/A Mihaela
RealPlayer [DONE] Paul [DONE] Alexandra N/A Virgil N/A Mihaela
Issues:
  • [known] bug 857096 - Java dropdown list not displayed properly
  • [known] bug 767673 - windows media player plugin shows blank content
  • [known] bug 692446 - Firefox prompts to install windows media player plugin even though it is already installed
  • [new] bug 868005 - Java applet steals focus from the location bar

Bug Fix Verifications

The following bugs have been fixed and need confirmation.

Bug Assignee Status
bug 867005 Mac OSX 10.6 nodes failing on release1.qa.mtv1.mozilla.com Anthony Hughes [DONE]
bug 862606 [summary hidden] Matt Wobensmith [DONE]
bug 861455 Daily collection when upload is disabled does not collect pull only providers Anthony Hughes [DONE]
bug 861112 Flash plugin incorrectly positioned when using non-standard layout.css.devPixelsPerPx value Cornel Ionce [DONE]
bug 860897 stub installer localization Cornel Ionce [DONE]
bug 859099 crash in nsObjectLoadingContent::DoStopPlugin (also verify bug 863792) Simona Marcu, Paul Silaghi [DROPPED] see comment #57
bug 857199 Please start offering the stub installer for English/Win32 on Beta Paul Silaghi [ON TRACK]
bug 851378 Move tearing down plugin prototype from NPRuntime to content Ioana Budnar [DROPPED] see comment #26.
bug 847272 "ASSERTION: caching a font associated with no family yet" Cornel Ionce [DONE]
bug 838105 crash in gfxUserFontSet::UserFontCache::Entry::KeyEquals Anthony Hughes [DROPPED] Comment #13
bug 810726 Infinite loop in nsBlockInFlowLineIterator::FindValidLine() with specific layout Cornel Ionce [DROPPED] Comment #84
bug 831780 scrollWidth/Height of DIV containing IMG reported as 40x40px regardless of CSS sizing Ioana Budnar [DONE]
bug 743252 Don't print the URL and other information when printing PDFs Ioana Budnar [DONE]
Totals
  • 62% [DONE]
  • 31% [DROPPED]
  • 7% [ON TRACK]

Update Tests

The following is the results of the Mozmill automated update tests.

Platform Betatest [dashboard] Releasetest [dashboard] Beta [dashboard]
Windows XP [DONE] 18.0b5 en-US & it
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b4 en-US & de
[DONE] 21.0b5 en-US & ru
[DONE] 18.0b5 en-US & it
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b4 en-US & de
[DONE] 21.0b5 en-US & ru
[DONE] 18.0b5 en-US & it
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b4 en-US & de
[DONE] 21.0b5 en-US & ru
Windows Vista [DONE] 18.0b4 en-US & fr
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & pt-BR
[DONE] 21.0b3 en-US & it
[DONE] 21.0b5 en-US & es-ES
[DONE] 18.0b4 en-US & fr
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & pt-BR
[DONE] 21.0b3 en-US & it
[DONE] 21.0b5 en-US & es-ES
[DONE] 18.0b4 en-US & fr
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & pt-BR
[DONE] 21.0b3 en-US & it
[DONE] 21.0b5 en-US & es-ES
Windows 7 32-bit [DONE] 18.0b2 en-US & ru
[DONE] 19.0b2 en-US & es-ES
[DONE] 20.0b2 en-US & zh-TW
[DONE] 21.0b1 en-US & fr
[DONE] 21.0b5 en-US & da
[DONE] 18.0b2 en-US & ru
[DONE] 19.0b2 en-US & es-ES
[DONE] 20.0b2 en-US & zh-TW
[DONE] 21.0b1 en-US & fr
[DONE] 21.0b5 en-US & da
[DONE] 18.0b2 en-US & ru
[DONE] 19.0b2 en-US & es-ES
[DONE] 20.0b2 en-US & zh-TW
[DONE] 21.0b1 en-US & fr
[DONE] 21.0b5 en-US & da
Windows 7 64-bit [DONE] 18.0b3 en-US & pt-BR
[DONE] 19.0b3 en-US & es-ES
[DONE] 20.0b3 en-US & de
[DONE] 21.0b2 en-US & ko
[DONE] 21.0b5 en-US & ro
[DONE] 18.0b3 en-US & pt-BR
[DONE] 19.0b3 en-US & es-ES
[DONE] 20.0b3 en-US & de
[DONE] 21.0b2 en-US & ko
[DONE] 21.0b5 en-US & ro
[DONE] 18.0b3 en-US & pt-BR
[DONE] 19.0b3 en-US & es-ES
[DONE] 20.0b3 en-US & de
[DONE] 21.0b2 en-US & ko
[DONE] 21.0b5 en-US & ro
Windows 8 32-bit [DONE] 18.0b6 en-US & pl
[DONE] 19.0b6 en-US & es-ES
[DONE] 20.0b6 en-US & zh-TW
[DONE] 21.0b3 en-US & ro
[DONE] 21.0b5 en-US & da
[DONE] 18.0b6 en-US & pl
[DONE] 19.0b6 en-US & es-ES
[DONE] 20.0b6 en-US & zh-TW
[DONE] 21.0b3 en-US & ro
[DONE] 21.0b5 en-US & da
[DONE] 18.0b6 en-US & pl
[DONE] 19.0b6 en-US & es-ES
[DONE] 20.0b6 en-US & zh-TW
[DONE] 21.0b3 en-US & ro
[DONE] 21.0b5 en-US & da
Windows 8 64-bit [DONE] 18.0b1 en-US & ko
[DONE] 19.0b1 en-US & fr
[DONE] 20.0b1 en-US & it
[DONE] 21.0b4 en-US & es-ES
[DONE] 21.0b5 en-US & zh-TW
[DONE] 18.0b1 en-US & ko
[DONE] 19.0b1 en-US & fr
[DONE] 20.0b1 en-US & it
[DONE] 21.0b4 en-US & es-ES
[DONE] 21.0b5 en-US & zh-TW
[DONE] 18.0b1 en-US & ko
[DONE] 19.0b1 en-US & fr
[DONE] 20.0b1 en-US & it
[DONE] 21.0b4 en-US & es-ES
[DONE] 21.0b5 en-US & zh-TW
Mac OSX 10.6 [DONE] 18.0b1 en-US & da
[DONE] 19.0b1 en-US & de
[DONE] 20.0b1 en-US & ja-JP-mac
[DONE] 21.0b1 en-US & it
[DONE] 21.0b5 en-US & es-ES
[DONE] 18.0b1 en-US & da
[DONE] 19.0b1 en-US & de
[DONE] 20.0b1 en-US & ja-JP-mac
[DONE] 21.0b1 en-US & it
[DONE] 21.0b5 en-US & es-ES
[DONE] 18.0b1 en-US & da
[DONE] 19.0b1 en-US & de
[DONE] 20.0b1 en-US & ja-JP-mac
[DONE] 21.0b1 en-US & it
[DONE] 21.0b5 en-US & es-ES
Mac OSX 10.7 [DONE] 18.0b2 en-US & ja-JP-mac
[DONE] 19.0b2 en-US & pt-BR
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b2 en-US & ru
[DONE] 21.0b5 en-US & zh-TW
[DONE] 18.0b2 en-US & ja-JP-mac
[DONE] 19.0b2 en-US & pt-BR
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b2 en-US & ru
[DONE] 21.0b5 en-US & zh-TW
[DONE] 18.0b2 en-US & ja-JP-mac
[DONE] 19.0b2 en-US & pt-BR
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b2 en-US & ru
[DONE] 21.0b5 en-US & zh-TW
Mac OSX 10.8 [DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & it
[DONE] 20.0b3 en-US & ro
[DONE] 21.0b4 en-US & de
[DONE] 21.0b5 en-US & ko
[DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & it
[DONE] 20.0b3 en-US & ro
[DONE] 21.0b4 en-US & de
[DONE] 21.0b5 en-US & ko
[DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & it
[DONE] 20.0b3 en-US & ro
[DONE] 21.0b4 en-US & de
[DONE] 21.0b5 en-US & ko
Ubuntu 12.04 32-bit [DONE] 18.0b4 en-US & fr
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & ko
[DONE] 21.0b1 en-US & zh-TW
[DONE] 21.0b5 en-US & ru
[DONE] 18.0b4 en-US & fr
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & ko
[DONE] 21.0b1 en-US & zh-TW
[DONE] 21.0b5 en-US & ru
[DONE] 18.0b4 en-US & fr
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & ko
[DONE] 21.0b1 en-US & zh-TW
[DONE] 21.0b5 en-US & ru
Ubuntu 12.04 64-bit [DONE] 18.0b5 en-US & da
[DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b2 en-US & de
[DONE] 21.0b5 en-US & it
[DONE] 18.0b5 en-US & da
[DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b2 en-US & de
[DONE] 21.0b5 en-US & it
[DONE] 18.0b5 en-US & da
[DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b2 en-US & de
[DONE] 21.0b5 en-US & it

New Issues

The following is a list of bugs found while testing this Beta. If you report a bug to bugzilla please list it below.

ID Summary Priority Status
868005 Java applet steals focus from the location bar P3 RESOLVED

1 Total; 0 Open (0%); 1 Resolved (100%); 0 Verified (0%);


Beta 7

Builds FTP, HTTP
ID 20130506154904
Changeset changeset (pushlog)
Checklist
  • [DONE] Automated Tests
  • [DONE] Regression Tests
  • [DONE] Bug Fix Verifications
  • [DONE] Update Tests (betatest)
  • [DONE] Update Tests (releasetest)
  • [DONE] Stability checkpoint
  • [DONE] Update Tests (beta)


Automated Testing

The following are the detailed results of the automated Mozmill test runs. These tests are run against the en-US and one random locale candidate per platform.

Platform Results [dashboard]
Windows XP [DONE] en-US & ro
Windows Vista [DONE] en-US & ko
Windows 7 32-bit [DONE] en-US & da
Windows 7 64-bit [DONE] en-US & it
Windows 8 32-bit [DONE] en-US & ru
Windows 8 64-bit [DONE] en-US & ja
Mac OSX 10.6 [DONE] en-US & de
Mac OSX 10.7 [DONE] en-US & es-ES
Mac OSX 10.8 [DONE] en-US & fr
Ubuntu 12.04 32-bit [DONE] en-US & pt-BR
Ubuntu 12.04 64-bit [DONE] en-US & zh-TW

Known test failures in bugzilla

Regression Tests

As per the checklist we will regression test the following areas.

Breakpad

Goal: Ensure that crash reports are submitted with the current build (use this Moztrap test as a rough guideline).

Build Windows 7 64-bit Mac OSX 10.8 Ubuntu 12.04 64-bit
Firefox 21.0b7#1 en-US [DONE] Manuela [DONE] Bogdan [DONE] Virgil
Firefox 21.0b7#1 de [DONE] Manuela [DONE] Bogdan [DONE] Virgil
Firefox 21.0b7#1 fr [DONE] Manuela [DONE] Bogdan [DONE] Virgil
Firefox 21.0b7#1 it [DONE] Manuela [DONE] Bogdan [DONE] Virgil
Issues:
  • No issues found

Platform De-support

Goal: Ensure that platforms which are no longer supported do not receive software updates to this Beta.

Test Platform Tester Result
Firefox 7 Beta Mac OSX 10.5 Simona Updated to Firefox 11 beta 5
Firefox 8 Beta Windows 2000 Simona Updated to Firefox 11 beta 5
Firefox 9 Beta Mac OSX 10.5 Simona Updated to Firefox 11 beta 5
Firefox 10 Beta Windows 2000 Simona Firefox is up to date
Issues:
  • bug 869397 Firefox Update for the unsupported OSs is inconsistent

GPU Stability Dogfooding

What to Test:

  • P1 Platforms: Windows 8, Windows 7
  • P2 Platforms: Mac OSX 10.8, Mac OSX 10.7, Ubuntu 12.04
  • P3 Platforms: Windows XP, Windows Vista
  • GPUs: Any AMD Radeon (particularly 6xxx/7xxx series), Any NVidia (particularly GTX 5xx/6xx series)
  • Scenarios:
    • Video playback
    • Web gaming
    • Maps
    • Excessive tabbed browsing
    • PDF viewer
    • Web mail including instant messaging and video calling (Skype, Yahoo, GMail, etc)
    • Plug-in driven web interfaces and menus
    • Restart and Session restore
GPU Platform Tester Results
AMD Radeon 6310 Windows 7 Marc Schifer [DONE]
AMD Radeon HD 7290 Windows 8 64-bit Anthony Hughes [DONE]
NVIDIA GeForce GT 610 Windows 7 Manuela Muntean [DONE]
NVIDIA GeForce GT 610 Windows 8 Manuela Muntean [DONE]
AMD Radeon HD 6450 Ubuntu 12.04 Cornel Ionce [DONE]
AMD Radeon HD 6630 Mac OSX 10.7 Virgil Dicu [DONE]
NVIDIA GeForce 9400 Mac OSX 10.7 Bogdan Maris [DONE]
AMD Radeon HD 6630 Mac OSX 10.8 Virgil Dicu [DONE]
NVIDIA GeForce 9400 Mac OSX 10.8 Bogdan Maris [DONE]
AMD Radeon HD 6450 Windows Vista Cornel Ionce [DONE]
  • Notes
    • NVIDIA GeForce GT 610 - Windows 7 64bit - Skype Click to Call doesn't work with Firefox or Chrome, but works with IE

Bug Fix Verifications

The following bugs have been fixed and need confirmation.

Bug Assignee Status
bug 867737 Update FHR reporting URL Simona Badau [DONE]
bug 857199 Please start offering the stub installer for English/Win32 on Beta Paul Silaghi [DONE]
bug 813550 Location bar's placeholder in an RTL chrome should be aligned on the right Paul Silaghi [DONE]
bug 818023 IonMonkey: function.caller does not always work (and in interpreter and JM) Paul Silaghi [DONE]
bug 863338 beta 21 branch still uses a beta version of nspr 4.9.6, please pick up final release tag Paul Silaghi [DONE]
bug 813569 Long string (ONE_MOMENT) not fully displayed in localized stub installer Paul Silaghi [DONE]
bug 829909 crash in nsWindow::Enable @ UserCallWinProcCheckWow Alexandra Lucinet [DONE]
bug 799172 "Back" after "Options" just returns to the first dialog Alexandra Lucinet [DONE]
Totals

Update Tests

The following is the results of the Mozmill automated update tests.

Platform Betatest [dashboard] Releasetest [dashboard] Beta [dashboard]
Windows XP [DONE] 18.0b5 en-US & it
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b3 en-US & de
[DONE] 21.0b6 en-US & ru
[DONE] 18.0b5 en-US & it
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b3 en-US & de
[DONE] 21.0b6 en-US & ru
[DONE] 18.0b5 en-US & it
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b3 en-US & de
[DONE] 21.0b6 en-US & ru
Windows Vista [DONE] 18.0b4 en-US & fr
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & pt-BR
[DONE] 21.0b2 en-US & it
[DONE] 21.0b6 en-US & es-ES
[DONE] 18.0b4 en-US & fr
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & pt-BR
[DONE] 21.0b2 en-US & it
[DONE] 21.0b6 en-US & es-ES
[DONE] 18.0b4 en-US & fr
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & pt-BR
[DONE] 21.0b2 en-US & it
[DONE] 21.0b6 en-US & es-ES
Windows 7 32-bit [DONE] 18.0b2 en-US & ru
[DONE] 19.0b2 en-US & es-ES
[DONE] 20.0b2 en-US & zh-TW
[DONE] 21.0b5 en-US & fr
[DONE] 21.0b6 en-US & da
[DONE] 18.0b2 en-US & ru
[DONE] 19.0b2 en-US & es-ES
[DONE] 20.0b2 en-US & zh-TW
[DONE] 21.0b5 en-US & fr
[DONE] 21.0b6 en-US & da
[DONE] 18.0b2 en-US & ru
[DONE] 19.0b2 en-US & es-ES
[DONE] 20.0b2 en-US & zh-TW
[DONE] 21.0b5 en-US & fr
[DONE] 21.0b6 en-US & da
Windows 7 64-bit [DONE] 18.0b3 en-US & pt-BR
[DONE] 19.0b3 en-US & es-ES
[DONE] 20.0b3 en-US & de
[DONE] 21.0b1 en-US & ko
[DONE] 21.0b6 en-US & ro
[DONE] 18.0b3 en-US & pt-BR
[DONE] 19.0b3 en-US & es-ES
[DONE] 20.0b3 en-US & de
[DONE] 21.0b1 en-US & ko
[DONE] 21.0b6 en-US & ro
[DONE] 18.0b3 en-US & pt-BR
[DONE] 19.0b3 en-US & es-ES
[DONE] 20.0b3 en-US & de
[DONE] 21.0b1 en-US & ko
[DONE] 21.0b6 en-US & ro
Windows 8 32-bit [DONE] 18.0b5 en-US & pl
[DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b3 en-US & de
[DONE] 21.0b6 en-US & it
[DONE] 18.0b5 en-US & pl
[DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b3 en-US & de
[DONE] 21.0b6 en-US & it
[DONE] 18.0b5 en-US & pl
[DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b3 en-US & de
[DONE] 21.0b6 en-US & it
Windows 8 64-bit [DONE] 18.0b6 en-US & ko
[DONE] 19.0b6 en-US & fr
[DONE] 20.0b6 en-US & it
[DONE] 21.0b3 en-US & es-ES
[DONE] 21.0b6 en-US & zh-TW
[DONE] 18.0b6 en-US & ko
[DONE] 19.0b6 en-US & fr
[DONE] 20.0b6 en-US & it
[DONE] 21.0b3 en-US & es-ES
[DONE] 21.0b6 en-US & zh-TW
[DONE] 18.0b6 en-US & ko
[DONE] 19.0b6 en-US & fr
[DONE] 20.0b6 en-US & it
[DONE] 21.0b3 en-US & es-ES
[DONE] 21.0b6 en-US & zh-TW
Mac OSX 10.6 [DONE] 18.0b1 en-US & da
[DONE] 19.0b1 en-US & de
[DONE] 20.0b1 en-US & ja-JP-mac
[DONE] 21.0b1 en-US & it
[DONE] 21.0b6 en-US & es-ES
[DONE] 18.0b1 en-US & da
[DONE] 19.0b1 en-US & de
[DONE] 20.0b1 en-US & ja-JP-mac
[DONE] 21.0b1 en-US & it
[DONE] 21.0b6 en-US & es-ES
[DONE] 18.0b1 en-US & da
[DONE] 19.0b1 en-US & de
[DONE] 20.0b1 en-US & ja-JP-mac
[DONE] 21.0b1 en-US & it
[DONE] 21.0b6 en-US & es-ES
Mac OSX 10.7 [DONE] 18.0b2 en-US & ja-JP-mac
[DONE] 19.0b2 en-US & pt-BR
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b2 en-US & ru
[DONE] 21.0b6 en-US & zh-TW
[DONE] 18.0b2 en-US & ja-JP-mac
[DONE] 19.0b2 en-US & pt-BR
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b2 en-US & ru
[DONE] 21.0b6 en-US & zh-TW
[DONE] 18.0b2 en-US & ja-JP-mac
[DONE] 19.0b2 en-US & pt-BR
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b2 en-US & ru
[DONE] 21.0b6 en-US & zh-TW
Mac OSX 10.8 [DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & it
[DONE] 20.0b3 en-US & ro
[DONE] 21.0b5 en-US & de
[DONE] 21.0b6 en-US & ko
[DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & it
[DONE] 20.0b3 en-US & ro
[DONE] 21.0b5 en-US & de
[DONE] 21.0b6 en-US & ko
[DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & it
[DONE] 20.0b3 en-US & ro
[DONE] 21.0b5 en-US & de
[DONE] 21.0b6 en-US & ko
Ubuntu 12.04 32-bit [DONE] 18.0b4 en-US & fr
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & ko
[DONE] 21.0b1 en-US & zh-TW
[DONE] 21.0b6 en-US & ru
[DONE] 18.0b4 en-US & fr
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & ko
[DONE] 21.0b1 en-US & zh-TW
[DONE] 21.0b6 en-US & ru
[DONE] 18.0b4 en-US & fr
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & ko
[DONE] 21.0b1 en-US & zh-TW
[DONE] 21.0b6 en-US & ru
Ubuntu 12.04 64-bit [DONE] 18.0b5 en-US & da
[DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b2 en-US & de
[DONE] 21.0b6 en-US & it
[DONE] 18.0b5 en-US & da
[DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b2 en-US & de
[DONE] 21.0b6 en-US & it
[DONE] 18.0b5 en-US & da
[DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b2 en-US & de
[DONE] 21.0b6 en-US & it

New Issues

The following is a list of bugs found while testing this Beta. If you report a bug to bugzilla please list it below.

ID Summary Priority Status
869397 Firefox Update for the unsupported OSs is inconsistent P3 RESOLVED
869456 The Time Open stat is modified after crashing Firefox P4 NEW

2 Total; 1 Open (50%); 1 Resolved (50%); 0 Verified (0%);


Release Candidate 1

Builds FTP, HTTP
ID 20130507015204
Changeset changeset (pushlog)
Checklist
  • [DONE] Automated Tests
  • [ON TRACK] Regression Tests
  • [ON TRACK] Bug Fix Verifications
  • [DONE] Update Tests (betatest)
  • [ON TRACK] Update Tests (releasetest)
  • [ON TRACK] Stability checkpoint
  • [ON TRACK] Update Tests (release)

Automated Testing

The following are the detailed results of the automated Mozmill test runs. These tests are run against the en-US and one random locale candidate per platform.

Platform Results [dashboard]
Windows XP [DONE] en-US & ro
Windows Vista [DONE] en-US & ko
Windows 7 32-bit [DONE] en-US & da
Windows 7 64-bit [DONE] en-US & it
Windows 8 32-bit [DONE] en-US & ru
Windows 8 64-bit [DONE] en-US & ja
Mac OSX 10.6 [DONE] en-US & de
Mac OSX 10.7 [DONE] en-US & es-ES
Mac OSX 10.8 [DONE] en-US & fr
Ubuntu 12.04 32-bit [DONE] en-US & pt-BR
Ubuntu 12.04 64-bit [DONE] en-US & zh-TW

Known test failures in bugzilla

  • /testSecurity/testSubmitUnencryptedInfoWarning.js::testSubmitUnencryptedInfoWarning
  • /testSecurity/testDVCertificate.js::testLarryBlue
  • /testPreferences/testRemoveAllCookies.js::testRemoveAllCookies
  • /testSecurity/testSafeBrowsingWarningPages.js::testWarningPages
  • /testSearch/testSearchSelection.js::testSearchSelectionViaContextMenu

Regression Tests

As per the checklist we will regression test the following areas.

Breakpad

Goal: Ensure that crash reports are submitted with the current build (use this Moztrap test as a rough guideline).

Build Windows XP Windows 7 64-bit Mac OSX 10.8 Ubuntu 12.04 64-bit
Firefox 21.0 en-US [DONE] Simona Marcu [DONE] Alexandra Lucinet [DONE] Cornel Ionce [DONE] Manuela Muntean
Firefox 21.0 de [DONE] Simona Marcu [DONE] Alexandra Lucinet [DONE] Cornel Ionce [DONE] Manuela Muntean
Firefox 21.0 fr [DONE] Simona Marcu [DONE] Alexandra Lucinet [DONE] Cornel Ionce [DONE] Virgil Dicu
Firefox 21.0 it [DONE] Simona Marcu [DONE] Alexandra Lucinet [DONE] Cornel Ionce [DONE] Virgil Dicu
Issues:
  • No issues found

Partner Repack Updates

Goal: Ensure that updates are working on betatest (use this Moztrap test as a rough guideline).

Build Windows XP Windows 7 64-bit Mac OSX 10.8 Ubuntu 12.04 64-bit
Firefox with Bing.com [DONE] Simona Marcu [DONE] Alexandra Lucinet [DONE] Cornel Ionce N/A Virgil Dicu
Firefox with MSN.com [DONE] Simona Marcu [DONE] Alexandra Lucinet [DONE] Cornel Ionce N/A Virgil Dicu
Firefox with Yahoo.com [DONE] Simona Marcu [DONE] Alexandra Lucinet [DONE] Cornel Ionce N/A Virgil Dicu
Firefox with custom distribution [DONE] Simona Marcu [DONE] Alexandra Lucinet [DONE] Cornel Ionce [DONE] Virgil Dicu
Issues:
  • The Update is done to Firefox 21 (the build ID of the updated version is the right one: 20130507015204), but after checking the About Firefox dialog the same "Update Failed" message is displayed
    • After restarting Firefox and checking the About Firefox dialog there is an attempt to download a new update ("Checking for updates..." and "Downloading Update - 0 bytes of 25.1 MB" messages are displayed), that attempt fails and the message "Update failed. Download the latest version" is displayed.
  • On Ubuntu - there are no repack Firefox builds available for Bing.com, MSN.com and Yahoo.com.

Remote Profiles

Goal: Ensure that Firefox works with remote profiles (use this Moztrap test as a rough guideline).

Build Windows XP Windows Vista Windows 7 32-bit Windows 8 64-bit
Firefox 21.0 en-US [DONE] Simona Marcu [DONE] Simona Marcu [DONE] Alexandra Lucinet [DONE] Alexandra Lucinet
Firefox 21.0 de [DONE] Simona Marcu [DONE] Simona Marcu [DONE] Alexandra Lucinet [DONE] Alexandra Lucinet
Firefox 20.0.1->21.0 en-US [DONE] Simona Marcu [DONE] Simona Marcu [DONE] Alexandra Lucinet [DONE] Alexandra Lucinet
Firefox 19.0->21.0 fr [DONE] Simona Marcu [DONE] Simona Marcu [DONE] Alexandra Lucinet [DONE] Alexandra Lucinet
Issues:
  • The Update is done to Firefox 21 (the build ID of the updated version is the right one: 20130507015204), but after checking the About Firefox dialog the same "Update Failed" message is displayed
    • After restarting Firefox and checking the About Firefox dialog there is an attempt to download a new update ("Checking for updates..." and "Downloading Update - 0 bytes of 25.1 MB" messages are displayed), that attempt fails and the message "Update failed. Download the latest version" is displayed.

GPU Stability Dogfooding

What to Test:

  • P1 Platforms: Windows 8, Windows 7
  • P2 Platforms: Mac OSX 10.8, Mac OSX 10.7, Ubuntu 12.04
  • P3 Platforms: Windows XP, Windows Vista
  • GPUs: Any AMD Radeon (particularly 6xxx/7xxx series), Any NVidia (particularly GTX 5xx/6xx series)
  • Scenarios:
    • Video playback
    • Web gaming
    • Maps
    • Excessive tabbed browsing
    • PDF viewer
    • Web mail including instant messaging and video calling (Skype, Yahoo, GMail, etc)
    • Plug-in driven web interfaces and menus
    • Restart and Session restore
GPU Platform Tester Results
AMD Radeon HD 6450 Windows 7 Bogdan Maris [DONE]
NVIDIA GeForce GT 610 Windows 7 Manuela Muntean [DONE]
AMD Radeon HD 7700 Windows 8 Mihaela Velimiroviciu [DONE]
NVIDIA GeForce GT 610 Windows 8 Manuela Muntean [DONE]
AMD Radeon HD 6450 Ubuntu 12.04 Bogdan Maris [DONE]
NVIDIA GeForce GT 610 Ubuntu 12.10 Manuela Muntean [DONE]
AMD Radeon HD 6630 Mac OSX 10.7 Paul Silaghi [DONE]
NVIDIA GeForce 9400 Mac OSX 10.7 Cornel Ionce [DONE]
AMD Radeon HD 6630 Mac OSX 10.8 Paul Silaghi [DONE]
NVIDIA GeForce 9400 Mac OSX 10.8 Cornel Ionce [DONE]
AMD Radeon HD 7700 Windows XP Mihaela Velimiroviciu [DONE]
NVIDIA GeForce GT 610 Windows XP Manuela Muntean [DONE]
AMD Radeon HD 6450 Windows Vista Bogdan Maris [DONE]
NVIDIA GeForce GT 610 Windows Vista Manuela Muntean [DONE]

Bug Fix Verifications

The following bugs have been fixed and need confirmation.

Bug Assignee Status
bug 698296 crash in js::SweepBackgroundThings @ js::gc::Arena::finalize Ioana Budnar [DROPPED]
bug 864776 Remove mn, sw during beta-release migration for Firefox 21 Paul Silaghi [DONE]
bug 828180 crash in nsIContent::IsHTML Alexandra Lucinet NEEDINFO: Scoobidiver
bug 840881 Wrong menu rendering with layout.css.devPixelsPerPx non-default (makes some menus unusable) Paul Silaghi [DONE]
bug 806206 crash in JSObject::global Ioana Budnar [DONE]
bug 826388 Allow single click on line numbers to add/remove breakpoints Ioana Budnar [DONE]
bug 829921 click events not delivered with trivial CSS transform, margin, overflow: hidden Ioana Budnar [DONE]
bug 833004 Can't start Tilt Ioana Budnar [DONE]
bug 843971 The sharing dialog for getUserMedia always reports the last shared set of devices, not the devices actively in use in a tab Mihaela Velimiroviciu [DONE]
Totals

Update Tests

The following is the results of the Mozmill automated update tests.

Platform Betatest [dashboard] Releasetest dashboard Release dashboard
Windows XP [DONE] 16.0.1 en-US & it
[DONE] 17.0 en-US & fr
[DONE] 18.0.1 en-US & ko
[DONE] 19.0 en-US & de
[DONE] 20.0.1 en-US & ru
Windows Vista [DONE] 16.0.2 en-US & fr
[DONE] 17.0.1 en-US & de
[DONE] 18.0 en-US & pt-BR
[DONE] 19.0.1 en-US & it
[DONE] 20.0 en-US & es-ES
Windows 7 32-bit [DONE] 16.0.2 en-US & ru
[DONE] 17.0.1 en-US & es-ES
[DONE] 18.0.1 en-US & zh-TW
[DONE] 19.0 en-US & fr
[DONE] 20.0 en-US & da
Windows 7 64-bit [DONE] 16.0.1 en-US & it
[DONE] 17.0 en-US & fr
[DONE] 18.0.1 en-US & ko
[DONE] 19.0 en-US & de
[DONE] 20.0.1 en-US & ru
Windows 8 32-bit [DONE] 16.0 en-US & pl
[DONE] 17.0.1 en-US & es-ES
[DONE] 18.0.2 en-US & zh-TW
[DONE] 19.0.2 en-US & ro
[DONE] 20.0 en-US & da
Windows 8 64-bit [DONE] 16.0.1 en-US & ko
[DONE] 17.0 en-US & fr
[DONE] 18.0 en-US & it
[DONE] 19.0.1 en-US & es-ES
[DONE] 20.0.1 en-US & zh-TW
Mac OSX 10.6 [DONE] 16.0 en-US & da
[DONE] 17.0 en-US & de
[DONE] 18.0 en-US & ja-JP-mac
[DONE] 19.0.2 en-US & it
[DONE] 20.0 en-US & es-ES
Mac OSX 10.7 [DONE] 16.0.2 en-US & ja-JP-mac
[DONE] 17.0.1 en-US & pt-BR
[DONE] 18.0.1 en-US & fr
[DONE] 19.0.1 en-US & ru
[DONE] 20.0 en-US & zh-TW
Mac OSX 10.8 [DONE] 16.0.1 en-US & es-ES
[DONE] 17.0 en-US & it
[DONE] 18.0.2 en-US & ro
[DONE] 19.0 en-US & de
[DONE] 20.0.1 en-US & ko
Ubuntu 12.04 32-bit [DONE] 16.0 en-US & fr
[DONE] 17.0.1 en-US & de
[DONE] 18.0 en-US & ko
[DONE] 19.0.2 en-US & zh-TW
[DONE] 20.0 en-US & ru
Ubuntu 12.04 64-bit [DONE] 16.0.2 en-US & da
[DONE] 17.0 en-US & es-ES
[DONE] 18.0.1 en-US & pt-BR
[DONE] 19.0 en-US & de
[DONE] 20.0.1 en-US & it

Include manual spotcheck of partner repack updates

New Issues

The following is a list of bugs found while testing this Beta. If you report a bug to bugzilla please list it below.

ID Summary Priority Status
870355 Breakpoints in the Debugger aren't always set right -- RESOLVED

1 Total; 0 Open (0%); 1 Resolved (100%); 0 Verified (0%);


Release Candidate 2

BUILD DROPPED, SEE Release Candidate 3 BELOW
Builds FTP, HTTP
ID 20130507220413
Changeset changeset (pushlog)
Checklist
  • [DONE] Automated Tests
  • [DONE] Regression Tests
  • [DONE] Bug Fix Verifications
  • [DONE] Update Tests (betatest)
  • [DROPPED] Update Tests (releasetest)
  • [DROPPED] Stability checkpoint
  • [DROPPED] Update Tests (release)

Automated Testing

The following are the detailed results of the automated Mozmill test runs. These tests are run against the en-US and one random locale candidate per platform.

Platform Results [dashboard]
Windows XP
Windows Vista
Windows 7 32-bit
Windows 7 64-bit
Windows 8 32-bit
Windows 8 64-bit
Mac OSX 10.6
Mac OSX 10.7
Mac OSX 10.8
Ubuntu 12.04 32-bit
Ubuntu 12.04 64-bit

Known test failures in bugzilla

Regression Tests

As per the checklist we will regression test the following areas.

Breakpad

Goal: Ensure that crash reports are submitted with the current build (use this Moztrap test as a rough guideline).

Build Windows XP Windows 7 64-bit Mac OSX 10.8 Ubuntu 12.04 64-bit
Firefox 21.0 en-US [DONE] Simona Marcu [DONE] Alexandra Lucinet [DONE] Cornel Ionce [DONE] Virgil Dicu
Firefox 21.0 de [DONE] Simona Marcu [DONE] Alexandra Lucinet [DONE] Cornel Ionce [DONE] Virgil Dicu
Firefox 21.0 fr [DONE] Simona Marcu [DONE] Alexandra Lucinet [DONE] Cornel Ionce [DONE] Virgil Dicu
Firefox 21.0 it [DONE] Simona Marcu [DONE] Alexandra Lucinet [DONE] Cornel Ionce [DONE] Virgil Dicu
Issues:
  • bug 870363 Broken layout on it builds for the about:healthreport page

GPU Stability Dogfooding

What to Test:

  • P1 Platforms: Windows 8, Windows 7
  • P2 Platforms: Mac OSX 10.8, Mac OSX 10.7, Ubuntu 12.04
  • P3 Platforms: Windows XP, Windows Vista
  • GPUs: Any AMD Radeon (particularly 6xxx/7xxx series), Any NVidia (particularly GTX 5xx/6xx series)
  • Scenarios:
    • Video playback
    • Web gaming
    • Maps
    • Excessive tabbed browsing
    • PDF viewer
    • Web mail including instant messaging and video calling (Skype, Yahoo, GMail, etc)
    • Plug-in driven web interfaces and menus
    • Restart and Session restore
GPU Platform Tester Results
AMD Radeon HD 6450 Windows 7 Bogdan Maris [DONE]
NVIDIA GeForce GT 610 Windows 7 Manuela Muntean [DONE]
AMD Radeon HD 7700 Windows 8 Mihaela Velimiroviciu [DONE]
NVIDIA GeForce GT 610 Windows 8 Manuela Muntean [DONE]
AMD Radeon HD 6450 Ubuntu 12.04 Bogdan Maris [DONE]
NVIDIA GeForce GT 610 Ubuntu 12.10 Manuela Muntean [DONE]
AMD Radeon HD 6630 Mac OSX 10.7 Paul Silaghi [DONE]
NVIDIA GeForce 9400 Mac OSX 10.7 Cornel Ionce [DONE]
AMD Radeon HD 6630 Mac OSX 10.8 Paul Silaghi [DONE]
NVIDIA GeForce 9400 Mac OSX 10.8 Cornel Ionce [DONE]
AMD Radeon HD 7700 Windows XP Mihaela Velimiroviciu [DONE]
NVIDIA GeForce GT 610 Windows XP Manuela Muntean [DONE]
AMD Radeon HD 6450 Windows Vista Bogdan Maris [DONE]
NVIDIA GeForce GT 610 Windows Vista Manuela Muntean [DONE]

Bug Fix Verifications

The following bugs have been fixed and need confirmation.

Bug Assignee Status
bug 828180 crash in nsIContent::IsHTML Alexandra Lucinet [ON TRACK]
Totals

Update Tests

The following is the results of the Mozmill automated update tests.

Platform Betatest dashboard Releasetest dashboard Release dashboard
Windows XP
Windows Vista
Windows 7 32-bit
Windows 7 64-bit
Windows 8 32-bit
Windows 8 64-bit
Mac OSX 10.6
Mac OSX 10.7
Mac OSX 10.8
Ubuntu 12.04 32-bit
Ubuntu 12.04 64-bit

Include manual spotcheck of partner repack updates

New Issues

The following is a list of bugs found while testing this Beta. If you report a bug to bugzilla please list it below.

ID Summary Priority Status
870363 Broken layout on it builds for the about:healthreport page -- RESOLVED

1 Total; 0 Open (0%); 1 Resolved (100%); 0 Verified (0%);


Release Candidate 3

Builds FTP, HTTP
ID 20130511120803
Changeset changeset (pushlog)
Checklist
  • [DONE] Automated Tests
  • [DONE] Regression Tests
  • [DONE] Bug Fix Verifications
  • [DONE] Update Tests (betatest)
  • [DONE] Update Tests (releasetest)
  • [DONE] Stability checkpoint
  • [DONE] Update Tests (release)

Automated Testing

The following are the detailed results of the automated Mozmill test runs. These tests are run against the en-US and one random locale candidate per platform.

Platform Results [dashboard]
Windows XP [DONE] en-US & ro
Windows Vista [DONE] en-US & ko
Windows 7 32-bit [DONE] en-US & da
Windows 7 64-bit [DONE] en-US & it
Windows 8 32-bit [DONE] en-US & ru
Windows 8 64-bit [DONE] en-US & es-ES
Mac OSX 10.6 [DONE] en-US & de
Mac OSX 10.7 [DONE] en-US & es-ES
Mac OSX 10.8 [DONE] en-US & fr
Ubuntu 12.04 32-bit [DONE] en-US & pt-BR
Ubuntu 12.04 64-bit [DONE] en-US & fi
Issues
  • [blocker] bug 871209 Scraper is failing to see 21.0build3 builds (currently unassigned)

Known test failures in bugzilla

Regression Tests

As per the checklist we will regression test the following areas.

Breakpad

Goal: Ensure that crash reports are submitted with the current build (use this Moztrap test as a rough guideline).

Build Windows XP Windows 7 64-bit Mac OSX 10.8 Ubuntu 12.04 64-bit
Firefox 21.0 en-US [DONE] Raymond Etornam [DONE] Raymond Etornam [DONE] Raymond Etornam [DONE] Ioana Budnar
Firefox 21.0 de [DONE] Raymond Etornam [DONE] Raymond Etornam [DONE] Raymond Etornam [DONE] Ioana Budnar
Firefox 21.0 fr [DONE] Raymond Etornam [DONE] Raymond Etornam [DONE] Raymond Etornam [DONE] Ioana Budnar
Firefox 21.0 it [DONE] Raymond Etornam [DONE] Raymond Etornam [DONE] Raymond Etornam [DONE] Ioana Budnar
Issues:

GPU Stability Dogfooding

What to Test:

  • P1 Platforms: Windows 8, Windows 7
  • P2 Platforms: Mac OSX 10.8, Mac OSX 10.7, Ubuntu 12.04
  • P3 Platforms: Windows XP, Windows Vista
  • GPUs: Any AMD Radeon (particularly 6xxx/7xxx series), Any NVidia (particularly GTX 5xx/6xx series)
  • Scenarios:
    • Video playback
    • Web gaming
    • Maps
    • Excessive tabbed browsing
    • PDF viewer
    • Web mail including instant messaging and video calling (Skype, Yahoo, GMail, etc)
    • Plug-in driven web interfaces and menus
    • Restart and Session restore
GPU Platform Tester Results
AMD Radeon HD 6310 Windows 7 Juan Becerra [DONE]
AMD Radeon 7290 Windows 8 Anthony Hughes [DONE]
NVIDIA GeForce GT 610 Windows 7 Manuela Muntean [DONE]
NVIDIA GeForce GT 610 Windows 8 Manuela Muntean [DONE]
AMD Radeon HD 7700 Ubuntu 12.10 Mihaela Velimiroviciu [DONE]
NVIDIA GeForce GT 610 Ubuntu 12.10 Manuela Muntean [DONE]
AMD Radeon HD 6630 Mac OSX 10.7 Paul Silaghi [DONE]
NVIDIA GeForce 9400 Mac OSX 10.7 Cornel Ionce [DONE]
AMD Radeon HD 6630 Mac OSX 10.8 Paul Silaghi [DONE]
NVIDIA GeForce 9400 Mac OSX 10.8 Cornel Ionce [DONE]
AMD Radeon HD 7700 Windows XP Mihaela Velimiroviciu [DONE]
NVIDIA GeForce GT 610 Windows XP Manuela Muntean [DONE]
AMD Radeon HD 7700 Windows Vista Cornel Ionce [DONE]
NVIDIA GeForce GT 610 Windows Vista Manuela Muntean [DONE]

Partner Repack Updates

Goal: Ensure that updates are working on betatest (use this Moztrap test as a rough guideline).

Be sure to check:

  • custom branding information should appear in the about dialog before and after an update
  • custom branding information should appear in the about dialog after starting with a new profile
  • if the build comes with any customizations (default search engine, toolbar, bookmarks, etc) these should remain intact after update
  • dogfood the build after for a few minutes to check that core browser functionality remains intact
  • Note: This requires updates on betatest channel
Build Windows XP Windows 7 64-bit Mac OSX 10.8 Ubuntu 12.04 32-bit
AOL [DONE] Firefox 20.0 en-US [DONE] Firefox 20.0 en-US [DONE] Firefox 20.0 en-US [DONE] Firefox 20.0 en-US
Bing [DONE] Firefox 20.0.1 en-US [DONE] Firefox 20.0.1 en-US [DONE] Firefox 20.0.1 en-US n/a
EBay [DONE] Firefox 19.0.2 en-US [DONE] Firefox 19.0.2 de [DONE] Firefox 19.0.2 en-US [DONE] Firefox 19.0.2 fr
Google [DONE] Firefox 19.0.1 en-US [DONE] Firefox 19.0.1 es-ES [a] n/a n/a
Mail.RU [DONE] Firefox 19.0 en-US [DONE] Firefox 19.0 ru [DONE] Firefox 19.0 en-US [DONE] Firefox 19.0 ru
MSN [DONE] Firefox 18.0.2 en-US [DONE] Firefox 18.0.2 en-US [DONE] Firefox 18.0.2 en-US n/a
Twitter [DONE] Firefox 18.0.1 en-US [DONE] Firefox 18.0.1 en-US [DONE] Firefox 18.0.1 en-US [DONE] Firefox 18.0.1 en-US
Yahoo [DONE] Firefox 18.0 en-US [DONE] Firefox 18.0 en-US [DONE] Firefox 18.0 en-US [DONE] Firefox 18.0 en-US
Yahoo China [DONE] Firefox 18.0 zh-TW [DONE] Firefox 18.0 zh-TW [DONE] Firefox 18.0 zh-TW [DONE] Firefox 18.0 zh-TW
Firefox en-US custom distribution [DONE] Firefox 19.0.2 en-US [DONE] Firefox 20.0.1 en-US [DONE] Firefox 18.0 en-US [DONE] Firefox 17.0 en-US
Firefox l10n custom distribution [DONE] Firefox 19.0.1 de [DONE] Firefox 20.0 zh-CN [DONE] Firefox 18.0.1 ja-JP [DONE] Firefox 17.0.1 es-ES
Issues:
ID Summary Priority Status
871261 Firefox default theme "Enable" button missing for Partner Repack builds -- RESOLVED
871274 Apps are not added to/removed from toolbar with Yahoo customized Firefox build -- NEW

2 Total; 1 Open (50%); 1 Resolved (50%); 0 Verified (0%);

Notes:

Bug Fix Verifications

The following bugs have been fixed and need confirmation.

Bug Assignee
[DONE] bug 842334 Ensure distribution directory is still at the top level Anthony Hughes, Tomcat/Hong

Update Tests

The following is the results of the Mozmill automated update tests.

Platform Betatest [dashboard] Releasetest [dashboard] Release [dashboard]
Windows XP [DONE] 0 fails [DONE] 0 fails [DONE] 0 fails
Windows Vista [DONE] 0 fails [DONE] 0 fails [DONE] 0 fails
Windows 7 32-bit [DONE] 0 fails [DONE] 0 fails [DONE] 0 fails
Windows 7 64-bit [DONE] 0 fails [DONE] 0 fails [DONE] 0 fails
Windows 8 32-bit [DONE] 0 fails [DONE] 0 fails [DONE] 0 fails
Windows 8 64-bit [DONE] 0 fails [DONE] 0 fails [DONE] 0 fails
Mac OSX 10.6 [DONE] 0 fails [DONE] 0 fails [DONE] 0 fails
Mac OSX 10.7 [DONE] 0 fails [DONE] 0 fails [DONE] 0 fails
Mac OSX 10.8 [DONE] 0 fails [DONE] 0 fails [DONE] 0 fails
Ubuntu 12.04 32-bit [DONE] 0 fails [DONE] 0 fails [DONE] 0 fails
Ubuntu 12.04 64-bit [DONE] 0 fails [DONE] 0 fails [DONE] 0 fails
Tested Versions
  • Firefox 16.0 - 20.0.1
  • de da es-ES fi fr it ja-JP-mac ko pl pt-BR ro ru zh-TW

New Issues

The following is a list of bugs found while testing this Beta. If you report a bug to bugzilla please list it below.

ID Summary Priority Status
871249 crash with many open tabs (due to pdf.js?) -- RESOLVED
871260 YouTube video stops after loading game http://wickedgoodgames.com/flash27/pacificthunder.html -- RESOLVED
871261 Firefox default theme "Enable" button missing for Partner Repack builds -- RESOLVED
871274 Apps are not added to/removed from toolbar with Yahoo customized Firefox build -- NEW
871275 The time open stat is modified after updating to the latest version P4 RESOLVED

5 Total; 1 Open (20%); 4 Resolved (80%); 0 Verified (0%);


Release Candidate 4

Builds FTP, HTTP
ID 20130512144307
Changeset changeset (pushlog)
Checklist
  • [DONE] Automated Tests
  • [DONE] Regression Tests
  • [DONE] Bug Fix Verifications
  • [DONE] Update Tests (betatest)
  • [DROPPED] Update Tests (releasetest)
  • [DROPPED] Stability checkpoint
  • [DROPPED] Update Tests (release)
  • NOTE: Not pushed live since build3 was deemed viable

Automated Testing

The following are the detailed results of the automated Mozmill test runs. These tests are run against the en-US and one random locale candidate per platform.

Platform Results [dashboard]
Windows XP [DONE] en-US & ro
Windows Vista [DONE] en-US & ko
Windows 7 32-bit [DONE] en-US & da
Windows 7 64-bit [DONE] en-US & it
Windows 8 32-bit [DONE] en-US & ru
Windows 8 64-bit [DONE] en-US & es-ES
Mac OSX 10.6 [DONE] en-US & de
Mac OSX 10.7 [DONE] en-US & es-ES
Mac OSX 10.8 [DONE] en-US & fr
Ubuntu 12.04 32-bit [DONE] en-US & pt-BR
Ubuntu 12.04 64-bit [DONE] en-US & fi

Known test failures in bugzilla

Regression Tests

As per the checklist we will regression test the following areas.

Breakpad

Goal: Ensure that crash reports are submitted with the current build (use this Moztrap test as a rough guideline).

Build Windows XP Windows 7 64-bit Mac OSX 10.8 Ubuntu 12.04 64-bit
Firefox 21.0 en-US [DONE] Simona Marcu [DONE] Alexandra Lucinet [DONE] Cornel Ionce [DONE] Bogdan Maris
Firefox 21.0 de [DONE] Simona Marcu [DONE] Alexandra Lucinet [DONE] Cornel Ionce [DONE] Bogdan Maris
Firefox 21.0 fr [DONE] Simona Marcu [DONE] Alexandra Lucinet [DONE] Cornel Ionce [DONE] Bogdan Maris
Firefox 21.0 it [DONE] Simona Marcu [DONE] Alexandra Lucinet [DONE] Cornel Ionce [DONE] Bogdan Maris
Issues:

GPU Stability Dogfooding

What to Test:

  • P1 Platforms: Windows 8, Windows 7
  • P2 Platforms: Mac OSX 10.8, Mac OSX 10.7, Ubuntu 12.04
  • P3 Platforms: Windows XP, Windows Vista
  • GPUs: Any AMD Radeon (particularly 6xxx/7xxx series), Any NVidia (particularly GTX 5xx/6xx series)
  • Scenarios:
    • Video playback
    • Web gaming
    • Maps
    • Excessive tabbed browsing
    • PDF viewer
    • Web mail including instant messaging and video calling (Skype, Yahoo, GMail, etc)
    • Plug-in driven web interfaces and menus
    • Restart and Session restore
GPU Platform Tester Results
AMD Radeon HD 6310 Windows 7 Juan Becerra [DONE]
AMD Radeon 7290 Windows 8 Anthony Hughes [DONE]
NVIDIA GeForce GT 610 Windows 7 Manuela Muntean [DONE]
NVIDIA GeForce GT 610 Windows 8 Manuela Muntean [DONE]
AMD Radeon HD 7700 Ubuntu 12.10 Mihaela Velimiroviciu [DONE]
NVIDIA GeForce GT 610 Ubuntu 12.10 Manuela Muntean [DONE]
AMD Radeon HD 6630 Mac OSX 10.7 Virgil Dicu [DONE]
NVIDIA GeForce 9400 Mac OSX 10.7 Cornel Ionce [DONE]
AMD Radeon HD 6630 Mac OSX 10.8 Virgil Dicu [DONE]
NVIDIA GeForce 9400 Mac OSX 10.8 Cornel Ionce [DONE]
AMD Radeon HD 7700 Windows XP Mihaela Velimiroviciu [DONE]
NVIDIA GeForce GT 610 Windows XP Manuela Muntean [DONE]
AMD Radeon HD 6450 Windows Vista Bogdan Maris [DONE]
NVIDIA GeForce GT 610 Windows Vista Manuela Muntean [DONE]

Partner Repack Updates

Goal: Ensure that updates are working on betatest (use this Moztrap test as a rough guideline).

Be sure to check:

  • custom branding information should appear in the about dialog before and after an update
  • custom branding information should appear in the about dialog after starting with a new profile
  • if the build comes with any customizations (default search engine, toolbar, bookmarks, etc) these should remain intact after update
  • dogfood the build after for a few minutes to check that core browser functionality remains intact
  • Note: This requires updates on betatest channel
Build Windows XP Windows 7 64-bit Mac OSX 10.8 Ubuntu 12.04 32-bit
AOL [DONE] Firefox 20.0 en-US [DONE] Firefox 20.0 en-US [DONE] Firefox 20.0 en-US [DONE] Firefox 20.0 en-US
Bing [DONE] Firefox 20.0.1 en-US [DONE] Firefox 20.0.1 en-US [DONE] Firefox 20.0.1 en-US n/a
EBay [DONE] Firefox 19.0.2 en-US [DONE] Firefox 19.0.2 de [DONE] Firefox 19.0.2 en-US [DONE] Firefox 19.0.2 fr
Google [DONE] Firefox 19.0.1 en-US [DONE] Firefox 19.0.1 es-ES [a] n/a n/a
Mail.RU [DONE] Firefox 19.0 en-US [DONE] Firefox 19.0 ru [DONE] Firefox 19.0 en-US [DONE] Firefox 19.0 ru
MSN [DONE] Firefox 18.0.2 en-US [DONE] Firefox 18.0.2 en-US [DONE] Firefox 18.0.2 en-US n/a
Twitter [DONE] Firefox 18.0.1 en-US [DONE] Firefox 18.0.1 en-US [DONE] Firefox 18.0.1 en-US [DONE] Firefox 18.0.1 en-US
Yahoo [DONE] Firefox 18.0 en-US [DONE] Firefox 18.0 en-US [DONE] Firefox 18.0 en-US [DONE] Firefox 18.0 en-US
Yahoo China [DONE] Firefox 18.0 zh-TW [DONE] Firefox 18.0 zh-TW [DONE] Firefox 18.0 zh-TW [DONE] Firefox 18.0 zh-TW
Firefox en-US custom distribution [DONE] Firefox 19.0.2 en-US [DONE] Firefox 20.0.1 en-US [DONE] Firefox 18.0 en-US [DONE] Firefox 17.0 en-US
Firefox l10n custom distribution [DONE] Firefox 19.0.1 de [DONE] Firefox 20.0 zh-CN [DONE] Firefox 18.0.1 ja-JP [DONE] Firefox 17.0.1 es-ES
Issues:
Notes:

Bug Fix Verifications

The following bugs have been fixed and need confirmation.

Bug Assignee Status
bug 842334 Ensure distribution directory is still at the top level Anthony, Tomcat, Hong VERIFIED FIXED

Update Tests

The following is the results of the Mozmill automated update tests.

Platform Betatest [dashboard] Releasetest dashboard Release dashboard
Windows XP [DONE] 16.0.1 en-US & it
[DONE] 17.0 en-US & fr
[DONE] 18.0.1 en-US & ko
[DONE] 19.0 en-US & de
[DONE] 20.0.1 en-US & ru
16.0.1 en-US & it
17.0 en-US & fr
18.0.1 en-US & ko
19.0 en-US & de
20.0.1 en-US & ru
16.0.1 en-US & it
17.0 en-US & fr
18.0.1 en-US & ko
19.0 en-US & de
20.0.1 en-US & ru
Windows Vista [DONE] 16.0.2 en-US & fr
[DONE] 17.0.1 en-US & de
[DONE] 18.0 en-US & pt-BR
[DONE] 19.0.1 en-US & it
[DONE] 20.0 en-US & es-ES
16.0.2 en-US & fr
17.0.1 en-US & de
18.0 en-US & pt-BR
19.0.1 en-US & it
20.0 en-US & es-ES
16.0.2 en-US & fr
17.0.1 en-US & de
18.0 en-US & pt-BR
19.0.1 en-US & it
20.0 en-US & es-ES
Windows 7 32-bit [DONE] 16.0.2 en-US & ru
[DONE] 17.0.1 en-US & es-ES
[DONE] 18.0.1 en-US & zh-TW
[DONE] 19.0 en-US & fr
[DONE] 20.0 en-US & da
16.0.2 en-US & ru
17.0.1 en-US & es-ES
18.0.1 en-US & zh-TW
19.0 en-US & fr
20.0 en-US & da
16.0.2 en-US & ru
17.0.1 en-US & es-ES
18.0.1 en-US & zh-TW
19.0 en-US & fr
20.0 en-US & da
Windows 7 64-bit [DONE] 16.0 en-US & pt-BR
[DONE] 17.0 en-US & es-ES
[DONE] 18.0.2 en-US & de
[DONE] 19.0.2 en-US & ko
[DONE] 20.0.1 en-US & ro
16.0 en-US & pt-BR
17.0 en-US & es-ES
18.0.2 en-US & de
19.0.2 en-US & ko
20.0.1 en-US & ro
16.0 en-US & pt-BR
17.0 en-US & es-ES
18.0.2 en-US & de
19.0.2 en-US & ko
20.0.1 en-US & ro
Windows 8 32-bit [DONE] 16.0 en-US & pl
[DONE] 17.0.1 en-US & es-ES
[DONE] 18.0.2 en-US & zh-TW
[DONE] 19.0.2 en-US & ro
[DONE] 20.0 en-US & da
16.0 en-US & pl
17.0.1 en-US & es-ES
18.0.2 en-US & zh-TW
19.0.2 en-US & ro
20.0 en-US & da
16.0 en-US & pl
17.0.1 en-US & es-ES
18.0.2 en-US & zh-TW
19.0.2 en-US & ro
20.0 en-US & da
Windows 8 64-bit [DONE] 16.0.1 en-US & ko
[DONE] 17.0 en-US & fr
[DONE] 18.0 en-US & it
[DONE] 19.0.1 en-US & es-ES
[DONE] 20.0.1 en-US & zh-TW
16.0.1 en-US & ko
17.0 en-US & fr
18.0 en-US & it
19.0.1 en-US & es-ES
20.0.1 en-US & zh-TW
16.0.1 en-US & ko
17.0 en-US & fr
18.0 en-US & it
19.0.1 en-US & es-ES
20.0.1 en-US & zh-TW
Mac OSX 10.6 [DONE] 16.0 en-US & da
[DONE] 17.0 en-US & de
[DONE] 18.0 en-US & ja-JP-mac
[DONE] 19.0.2 en-US & it
[DONE] 20.0 en-US & es-ES
16.0 en-US & da
17.0 en-US & de
18.0 en-US & ja-JP-mac
19.0.2 en-US & it
20.0 en-US & es-ES
16.0 en-US & da
17.0 en-US & de
18.0 en-US & ja-JP-mac
19.0.2 en-US & it
20.0 en-US & es-ES
Mac OSX 10.7 [DONE] 16.0.2 en-US & ja-JP-mac
[DONE] 17.0.1 en-US & pt-BR
[DONE] 18.0.1 en-US & fr
[DONE] 19.0.1 en-US & ru
[DONE] 20.0 en-US & zh-TW
16.0.2 en-US & ja-JP-mac
17.0.1 en-US & pt-BR
18.0.1 en-US & fr
19.0.1 en-US & ru
20.0 en-US & zh-TW
16.0.2 en-US & ja-JP-mac
17.0.1 en-US & pt-BR
18.0.1 en-US & fr
19.0.1 en-US & ru
20.0 en-US & zh-TW
Mac OSX 10.8 [DONE] 16.0.1 en-US & es-ES
[DONE] 17.0 en-US & it
[DONE] 18.0.2 en-US & ro
[DONE] 19.0 en-US & de
[DONE] 20.0.1 en-US & ko
16.0.1 en-US & es-ES
17.0 en-US & it
18.0.2 en-US & ro
19.0 en-US & de
20.0.1 en-US & ko
16.0.1 en-US & es-ES
17.0 en-US & it
18.0.2 en-US & ro
19.0 en-US & de
20.0.1 en-US & ko
Ubuntu 12.04 32-bit [DONE] 16.0 en-US & fr
[DONE] 17.0.1 en-US & de
[DONE] 18.0 en-US & ko
[DONE] 19.0.2 en-US & zh-TW
[DONE] 20.0 en-US & ru
16.0 en-US & fr
17.0.1 en-US & de
18.0 en-US & ko
19.0.2 en-US & zh-TW
20.0 en-US & ru
16.0 en-US & fr
17.0.1 en-US & de
18.0 en-US & ko
19.0.2 en-US & zh-TW
20.0 en-US & ru
Ubuntu 12.04 64-bit [DONE] 16.0.2 en-US & da
[DONE] 17.0 en-US & es-ES
[DONE] 18.0.1 en-US & pt-BR
[DONE] 19.0 en-US & de
[DONE] 20.0.1 en-US & it
16.0.2 en-US & da
17.0 en-US & es-ES
18.0.1 en-US & pt-BR
19.0 en-US & de
20.0.1 en-US & it
16.0.2 en-US & da
17.0 en-US & es-ES
18.0.1 en-US & pt-BR
19.0 en-US & de
20.0.1 en-US & it

New Issues

The following is a list of bugs found while testing this Beta. If you report a bug to bugzilla please list it below.

ID Summary Priority Status
871519 Ctrl + Enter doesn't work well when the string in the URL bar contains a dot -- RESOLVED
871520 Mozilla Crash Reporter dialog title for italian build is in english -- RESOLVED
871538 Some extensions that are installed with repack builds are not kept if Firefox is opened with older profiles -- NEW

3 Total; 1 Open (33.33%); 2 Resolved (66.67%); 0 Verified (0%);


Known Issues

ID Summary Priority Status
692446 Firefox prompts to install windows media player plugin even though it is already installed -- RESOLVED
767673 windows media player plugin shows blank content P3 RESOLVED
791195 Session is not restored after Firefox crash -- RESOLVED
793126 crash with error message: "Failed to create temporary texture in system memory. Error code: 2147942414", while navigating street view on Google MapsGL P3 ASSIGNED
834256 Firefox crashes OOM while viewing PDF when many PDF documents are opened P3 NEW
842290 "Restart Aurora" pop up doesn't appear when selecting private browsing mode -- VERIFIED
842491 Source page remains active when Private Browsing is closed -- NEW
842577 Remove from History is visible for a paused download -- RESOLVED
843087 Private and non private downloads shown in the same Library window -- NEW
843540 [Windows] Shield icon missing in Mixed content blocker doorhanger -- VERIFIED
843559 [Ubuntu] Profiler keyboard shortcut (CTRL+SHIFT+Y) opens Library -- VERIFIED
857096 Java dropdown list not displayed properly P3 RESOLVED
863183 The remote debugger "Connecting..." string and animation should be larger -- NEW
863238 Not localized mozilla pages opened from the Firefox options -- RESOLVED
863256 Report Web Forgery warning is not always displayed -- NEW
868005 Java applet steals focus from the location bar P3 RESOLVED

16 Total; 6 Open (37.5%); 7 Resolved (43.75%); 3 Verified (18.75%);


QAWANTED Bugs

The following bugs require help investigating steps to reproduce, regression ranges, workarounds, or other conditions (like hardware or software). Select a bug from the bugzilla query and do some testing to provide the information being requested. Be sure to add your bug to the list below.

Bug Assignee Status
bug 865701 crash in nsFrameManager::ReResolveStyleContext with AMD Radeon 6310/6320 Juan Becerra [ON TRACK]
bug 864478 Unable to upload receipt to our Health and Wellness provider Juan Becerra [DONE]
bug 863714 crash in nsCSSScanner::Next mainly with AMD Radeon HD 6310 Juan Becerra [DROPPED]
bug 860052 Java plugins get caught in nested event loop, browser ceases repainting Ioana Budnar [DONE]
bug 859099 crash in nsObjectLoadingContent::DoStopPlugin Paul Silaghi [DONE]
bug 858655 crash in js::ScopeIter::settle Anthony Hughes [DONE]
bug 852915 [Win7] crash in mozilla::MediaDecoderReader::DecodeToFirstAudioData Anthony Hughes [DROPPED]
bug 851788 crash in JSScript::clearTraps Anthony Hughes [DROPPED]
bug 851630 Can't open popup notifications with the keyboard (focus gets stuck at the identity icon) Anthony Hughes [DONE]
bug 850571 crash in nsView::`.* deleting destructor' Juan Becerra [DROPPED]
bug 848663 [Win8] Can’t update from FF19.0 to FF19.0.1 w/custom browser Anthony Hughes [DONE]
bug 844553 Plugins disabled by Aurora 21.x Paul Silaghi [DONE]
bug 842473 Crash in mozilla::dom::FragmentOrElement::CanSkip with FoxyProxy 4.2 Anthony Hughes [DROPPED]
bug 840901 Crash in nsDocLoader::DoFireOnStateChange with Whitesky's ID Vault (bundled with HP computers?) Paul Silaghi [DROPPED]
bug 839629 Fix partner add-ons so that search pref changes comply with add-on guidelines Manuela Muntean [DONE]
bug 837370 Crash in TypeConstraintFreeze::newType @ js::types::TypeCompartment::addPendingRecompile Anthony Hughes [DROPPED]
bug 836263 Crash in AppendUTF8toUTF16 or AppendUTF16toUTF8 with Giant Savings add-on Paul Silaghi [DROPPED]
bug 832091 Crash in xpc::XPCWrappedNativeXrayTraits::resolveDOMCollectionProperty w/Java Virgil Dicu [DROPPED]
bug 830531 [Win8] crash in XPC_WN_Helper_NewResolve mainly with AMD Radeon HD Anthony Hughes [DONE]
bug 824971 [summary hidden] Matt Wobensmith [DROPPED]
bug 819028 [Win7][KB2670838] crash in gfxWindowsPlatform::UpdateRenderMode @ CResource::InvalidAlignment Marcia Knous [DROPPED]
bug 816976 [adbe 3519647][10.8] crash in glcGetIOAccelService in Flash plugin-container process Paul Silaghi [DROPPED]
bug 812695 [D2D] Text Rendering Issues due to Windows 7 Platform Update KB2670838 Juan Becerra [DROPPED]
bug 812683 [Win7][KB2670838] startup crash in _d2d_clear_surface @ CContext::RestorePipelineStateImpl Juan Becerra [DROPPED]
bug 805406 Crash in gfxContext::PushClipsToDT with d2d11.dll 6.2 Juan Becerra [DROPPED]
bug 698296 crash in js::SweepBackgroundThings @ js::gc::Arena::finalize Ioana Budnar [DROPPED]
Totals:
  • 36% [DONE]
  • 56% [DROPPED]
  • 8% [ON TRACK]

UNCONFIRMED Bugs

The following bugs need confirmation that they are reproducible. Select a bug from bugzilla and set the status to NEW if you reproduce it. Be sure to add a comment indicating your operating system, Firefox version(s), and steps you used to reproduce the bug. Add the bug to the list below.

ID Summary Priority Status
841705 deviantart.com Submit/add Title, description or tags hangs Firefox -- RESOLVED
847653 Scaling transform animation can cause severe jank and sometimes misrender on Firefox for Mac -- RESOLVED
851758 missing all bookmarks of subdirectories when use xmarks sync -- RESOLVED
852976 Messages minimised in Gmail composer automatically restore when switching tabs -- RESOLVED
856415 100% CPU core hang in firefox.exe mozjs.dll!?JS_NewStringCopyN@@YAPAVJSString@@PAUJSContext@@PBDI@Z+0x8c26 -- RESOLVED

5 Total; 0 Open (0%); 5 Resolved (100%); 0 Verified (0%);


FHR Release Blockers

  • [DONE] bug 858242 [FHR] Privacy Policy isn't linked from the sharing doorhanger
  • [DONE] bug 856315 [FHR] Outlier startup time destroys chart usability

Final Sign-offs

Final Release

Post Mortem Notes

  • bug 869866 'Fullscreen playback' NVIDIA stereo 3d for youtube not working
    • Should at this particular case to our regression checklist
      • concern that as our list grows we become more distracted (can't automated everything)
      • consider doing the full checklist in Beta 1 and RC1 (will require more QA time with builds but allow for less distraction on intermediary betas)
    • Should look at getting someone like iBeta engaged on an ongoing basis
      • admit that QA can't contend with the scale being demanded of us with current resources
    • Should look at engaging with third parties like NVidia earlier
      • had they be testing Nightly 3 months ago we'd have known about it
    • Should look at having mozilla-central opt tinderbox builds for longer so we can regress further
      • current pushlog is large, Sun Bingo bug regressed 3 months ago, we could regress it further if we had builds
      • rebuilding random changesets in that range is not too feasible
  • bug 842334 Ensure distribution directory is still at the top level after the move of the browser into the browser directory
    • How did this get missed if we ran the Moztrap test?
    • RelEng is adding some automation via bug 870766
    • Can/should QA investigate including Partner Repacks in our Mozmill update tests?
    • Can/should we involve Tomcat/Hong in the sign-off process for future RCs? QA lacks the expertise and tests to be able to reasonably qualify custom distributions
  • bug 870834 Shutdown crash @ PR_Free _pt_thread_death_internal
    • Found by Bob Clary via crash automation, Poker Games first occurred on April 30
    • Why did it take us 10 days to file a bug?
  • bug 871851 Firefox 21.0build3 updates on releasetest are broken, file size inconsistency
    • caused by build3->build4 updates on test channel
    • does RelEng have automation to detect this?