Firefox 3 Beta 5 Test Results

Related Pages

Links to separate test results. Some are referenced further below.

Project Release Tracking Page

Contact Info

PM: Mike Beltzner (beltzner)
Build: John O'Duinn (joduinn)
QA: Tony (tchung)

Schedule

Code Freeze: 3/18
Code Build: ETA 3/26
QA test signoff: ETA 3/26-3/31

Smoketests (3/26) 100%
BFTs (3/26-3/27) 100%
Beta 5 and past focused Tests (ongoing)
Update snippets from Alpha X, Beta 1,2,3,4 -> 5 (3/27-3/28) 100% betatest, 100% releasetest, 100% beta (4 testers)
P1 L10N spotchecks (3/27) 100% (1 tester)
B5 bug verifications (ongoing)
Beta 5 Testday (3/28) 100% (3 testers)
Release notes, Websites, live tests (3/31) 100% (3 testers)

Release Notes

See list of release notes here

Build Location and IDs

RC1: ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/3.0b5-candidates/rc1/

Linux: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9b5) Gecko/2008032604 Firefox/3.0b5
Mac: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-GB; rv:1.9b5) Gecko/2008032604 Firefox/3.0b5

RC2: ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/3.0b5-candidates/rc2/

Windows: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5
Mac: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9b5) Gecko/2008032619 Firefox/3.0b5
Linux: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9b5) Gecko/2008032619 Firefox/3.0b5

Strategy

Areas Covered:
- Smoketests
- BFTs
- Focused Tests
- Update snippets from Alpha X, Beta X -> Beta 5
- Memory Leak Testing (handled by Tomcat on debug builds)
- P1 L10N spotchecks
- Bug verifications
- Beta 5 Testday

Areas Not Covered:
- FFTs
- Top Sites
- Top Plugins
- Top Themes
- Top Extensions
- localization (String changes) - 3rd party will cover
- Distribution
- Major Updates

Test Results

Types of Tests	Win XP	Win Vista	Mac Tiger	Mac Leopard	Linux
Smoketest	Tomcat - PASS	juanb - PASS	Tracy - PASS	marcia - PASS	Tomcat - PASS
BFT	Tomcat - PASS	Tomcat (Vista test suite) - PASS	juanb - PASS	Marcia (Leopard test suite/PPC and Intel) - PASS	Tracy - PASS
Updates Testing (see below)	Tchung - Betatest PASS, Releasetest PASS, Beta PASS	Juanb - Betatest PASS, Releasetest PASS, Beta PASS	Stephend - Betatest PASS, Releasetest PASS, Beta PASS	Marcia - Betatest PASS, Releasetest PASS, Beta PASS	Abillings - Betatest PASS, Releasetest PASS, Beta PASS
l10N Pri 1 locales	Tchung - PASS	N/A	Tchung - PASS	N/A	Tchung - PASS
Bug Verifications	Everyone - result	Everyone - result	Everyone - result	Everyone - result	Everyone - result
Live Checks & Download Links	Tomcat - PASS	Tomcat - PASS	Marcia - PASS	Juanb - PASS	Tchung - PASS

Updates Testing

We will be doing updates of all Alpha and Beta users to Beta 5. Tests will include different locales for beta 1, but only en-US for all past versions. We will be checking partial updates, complete updates, and failed status.

Will be available on betatest, releasetest, and beta channels

Betatest channel - PASS
Releasetest channel - PASS
Beta channel (live) - PASS

Testcases and Results are tracked here.

L10N P1 Spotchecks

Visual inspection: Go through the P1 list of locales, and run a spotcheck to see if everything visually looks fine. Refer to L10N Requirements for more specifics.

Automation inspection: supported locales will be ran through Minotaur, and results will be reviewed by axel and mic.

Testcases and Results are tracked here - PASS.

Bug Verifications

Bugs that are marked RESOLVED FIXED that are checked in for beta. Please verify them.

P1/P2 Firefox bugs - http://tinyurl.com/3bfgoa
P1/P2 Platform bugs - http://tinyurl.com/354n48

Testday

A beta 5 testday will be held and call for the community support to assist with testing. Activities include executing litmus testcases, bug verifications, and focus on the new feature areas that are being implemented. More can be found on QMO.

Testday results are posted here.

RC2 Detailed Results

marcia's results

Smoketests:
- 10.5 Mac:
  - Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9b5) Gecko/2008032619 Firefox/3.0b5
  - Date: 27. March 2008
  - Status: PASS - Results in Litmus
  - Bugs filed:
    - https://bugzilla.mozilla.org/show_bug.cgi?id=425476

Leopard Test Suite (Intel):
- 10.5 Mac:
  - Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9b5) Gecko/2008032619 Firefox/3.0b5
  - Date: 27. March 2008
  - Status: PASS - Results in Litmus
  - Added one new Parental Controls test case to Litmus.
  - Noted horrible error message that renders when a site is not allowed - the text overflows the page.

Leopard Test Suite (PPC):
- 10.5 Mac:
  - Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.5; en-US; rv:1.9b5) Gecko/2008032619 Firefox/3.0b5
  - Date: 27. March 2008
  - Status: PASS - Results in Litmus
  - Noted that when you allow a website that is redirected to another site, the user cannot see it. Safari does this too. Example site is bodemiller.com
  - The text overflow that I saw on Intel Mac I did not see on PPC.
  - I had one issue where I blocked the Firefox app. When I logged back in as Admin I could not run Firefox at all and had to restart. Need to investigate further.

tomcat's results

Smoketests:

Linux:
- Build Id: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9b5) Gecko/2008032619 Firefox/3.0b5
- Date: 27. March 2008
- Status: PASS - Results in Litmus

Windows:
- Build Id: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5
- Date: 27. March 2008
- Status: PASS - Results in Litmus

BFT

Windows XP
- Build Id: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5
- ** Status: PASS

Vista
- Build Id: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5
- Status: PASS

Bugs Filed

- bug 425300 ###!!! ASSERTION: wrong thread: 'NS_IsMainThread()
- bug 425306 Firefox Crash on http://www.dhteumeuleu.com/
- bug 425318 JS Error when i open a new Tab
- bug 425443 Gtk-CRITICAL **: gtk_widget Assertions when i use Firefox 3 Beta 5 Rc2 on Linux
- bug 425562 Firefox Crash on Centos5 when changing tabs @ libpthread-2.5.so@0xbaf0
- bug 425568 Need Option/Pref to change Email address
- bug 425585 Firefox Beta 5 RC 2 Crash on Exit after Printing
- bug 425760 https://www.mozilla.com/certs/mozilla-root.crt is not sending the correct content/type

tracy's results

Mac 10.4 Smoketests (PPC) - PASS
- Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9b5) Gecko/2008032619 Firefox/3.0b5
Linux FC7 BFT - PASS
- Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9b5) Gecko/2008032619 Firefox/3.0b5

whimboo's results

Bugs filed:

Crash when a local SWF file tries to communicate with the internet and Javascript is deactivated Bug 425506
Undo function does not work for deleted tags in the Library Bug 425529
Direct download of add-ons from AMO site isn't available anymore Bug 425839
"Most Visited", "Recently Bookmarked", and "Recent Tags" should be treated as folders bug 425851
Search bar inside Library is cut-off for default Library window width bug 425854
Focus in "Get Add-ons" panel should be set to the search field when panel is opened bug 425856

juanb's

Smoketests (Vista) - PASS
- Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5 ID:2008032620
- Date: 3.27.2008
- Results in Litmus
- Notes: migration wizard not launched in Vista bug 415968
BFT (Tiger 10.4.11 MacIntel) - PASS
- Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9b5) Gecko/2008032619 Firefox/3.0b5
- Date: 3.31.2008
- Notes: Bugs I filed or encountered for which I had test failures.
  - bug 426230 - add-ons seach results don't match those in web site
  - bug 425274 - drag and drop icons in toolbar
  - bug 418235 - acid2 fails when zoomed
  - bug 411585 - mozillazine site does not print correctly
  - bug 426275 clear private data crash

RC1 Detailed Results

Tomcats Results

Smoketests:
- Linux:
  - Build Id: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9b5) Gecko/2008032604 Firefox/3.0b5
  - Date: 26. March 2008
  - Status: PASS - Results in Litmus

marcia's results

Smoketests:
- 10.5 Mac:
  - Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9b4) Gecko/2008032604 Firefox/3.0b5
  - Date: 26. March 2008
  - Status: PASS - Results in Litmus
  - Unable to test extension and themes since none are currently compatible.

Smoketests:
- 10.4 PPC Mac:
  - Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9b5) Gecko/2008032604 Firefox/3.0b5
  - Date: 26. March 2008
  - Fun smoketest: Status: PASS - Results in Litmus
  - Unable to test extension and themes since none are currently compatible.
  - Bug filed: https://bugzilla.mozilla.org/show_bug.cgi?id=425274 -> Should be release noted if not fixed

Live Spot Checks

Check live site download links on all platforms. Please cover en-US, and 2 or 3 additional Tier 1 locales as well. Pay attention to the following areas:

Check for signed installer and applications (windows only)
Firstrun page, whatsnew page, Home page, Search plugin
Install, uninstall
General spotchecks and browser functionality
Confirm Build IDs

tchung (Linux)

No issues found.
- Mozilla/5.0 (X11; U; Linux i686; es-AR; rv:1.9b5) Gecko/2008032619 Firefox/3.0b5

tomcat

a mirror issue was discovered before the release started, filed https://bugzilla.mozilla.org/show_bug.cgi?id=426526

Top Issues Reported in Hendrix Against Firefox 3 Beta 5

Enter here

Top Issues from Reporter

Enter here

QA/Firefox3/TestResults/Beta5

Contents

Firefox 3 Beta 5 Test Results

Related Pages

Contact Info

Schedule

Release Notes

Build Location and IDs

Strategy

Test Results

Updates Testing

L10N P1 Spotchecks

Bug Verifications

Testday

RC2 Detailed Results

marcia's results

tomcat's results

Smoketests:

BFT

Bugs Filed

tracy's results

whimboo's results

juanb's

RC1 Detailed Results

Tomcats Results

marcia's results

Live Spot Checks

Top Issues Reported in Hendrix Against Firefox 3 Beta 5

Top Issues from Reporter

Navigation menu

Search