QA/Firefox3/TestResults/M8

« QA/Firefox3/TestResults

Firefox 3 M8 Test Results

Contact Info

Build: John O'Duinn (joduinn)
QA: Tony (tchung), marcia

Schedule

Code Freeze

9/5

Code Build

9/12

QA test signoff

ETA 9/14

Release Notes

M8 Release Notes

Builds

• http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/1.9a8-candidates/rc1

Strategy

• Areas Covered:
  • Smoketests (results at: http://people.mozilla.com/~twalker/trunkresultslinks.html)
  • BFTs
  • M8 and past focused Tests
  • bug verifications
  • Spotchecks (includes standard plugins, top sites, look and feel)
  • Specific Addons/Extensions/Themes
  • Top Sites

• Areas Not Covered:
  • FFTs
  • L10N
  • Distribution
  • Updates

Tests are ran against en-US locale only.

Test Results

Types of Tests	Win XP	Win Vista (Vista Litmus Test Suite)	Mac	Linux
Smoketest	Tomcat - PASS	tomcat - PASS	marcia - PASS	juanb - pass
BFT	Tomcat PASS	tomcat - PASS	marcia - 90% completed. PASS	tchung - PASS
Focused Testing	stephend	marcia	stephend	tchung

Detailed Results

• Windows XP Smoketest/BFT (tomcat) - PASS
  • Windows XP
  • Testing Date: 9/13/07
  • Build ID: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a8) Gecko/2007091216 GranParadiso/3.0a8
  • Bugs filed during smoketest and BFT:
    • https://bugzilla.mozilla.org/show_bug.cgi?id=396221
    • https://bugzilla.mozilla.org/show_bug.cgi?id=396118
    • https://bugzilla.mozilla.org/show_bug.cgi?id=396042
    • https://bugzilla.mozilla.org/show_bug.cgi?id=396228
    • https://bugzilla.mozilla.org/show_bug.cgi?id=396236

• Top Addon Testing (Tomcat)- PASS
  • Run into https://bugzilla.mozilla.org/show_bug.cgi?id=396502

• Top Site Testing (Tomcat) - PASS
  • Top 35 Alexa Sites PASS

• Windows Vista Test Suite - FAIL
  • Windows Vista
  • Testing Date: 9/13/07
  • https://bugzilla.mozilla.org/show_bug.cgi?id=396509 was found. Firefox does not launch with Parental Controls turned on.

• Mac Smoketest - PASS
  • Intel Mac
  • Testing Date: 9/12/07
  • Build ID: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.9a8) Gecko/2007091211 GranParadiso/3.0a8
  • Bugs filed during smoketest:
    • https://bugzilla.mozilla.org/show_bug.cgi?id=395975
    • https://bugzilla.mozilla.org/show_bug.cgi?id=395972
  • Bugs seen during smoketest
    • Preferences Panel quakes when you move between sections. Josh said bug is on file and targeted for M9, although I could not find that bug on the list of M9 bugs.

• Mac BFT - PASS
  • 1/2 on Intel Mac and 1/2 on PPC Mac
  • Testing Date: 9/14/07, 9/15/07 and 9/17/07
  • Build ID: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.9a8) Gecko/2007091211 GranParadiso/3.0a8
  • Issues Noted:
    • Could not run any of the MS test cases due to the ispiked's test cases not being on people.mozilla.com
    • "Visit Date" column in Organize Bookmarks does not show the date and time for bookmarks that I added today. This is way the functionality was on 2.0.0.x branch. Also https://bugzilla.mozilla.org/show_bug.cgi?id=333141 should be addressed for polish.
    • Safari migration needs attention. Currently a number of prefs are not migrated. I will search for open bugs and make comments. Seth fixed the search history migration on the 2.0.0.x branch but did not check that fix into the trunk.
  • Bugs Noted:
    • https://bugzilla.mozilla.org/show_bug.cgi?id=396485 - Unable to quit browser (NS_ERROR_FAILURE) :: goQuitApplication.
    • Frequent crashes in nsPluginInstanceOwner: are occuring. There are several bugs on file for this, such as https://bugzilla.mozilla.org/show_bug.cgi?id=395972.

• Linux Smoketest (juanb) - Pass
  • Bugs found so far:
    • bug 395994 wrong rss feed icon
    • bug 395990 (duped) form completion box unshaded in Linux
    • bug 396109 master password quality meter not working in Linux
  • Migration story is unclear. Only cookies are imported from Seamonkey, for example. Should we worry about migration in Linux? How much should we worry?

• Linux BFT (tchung) - PASS
  • Issues Found:
    • Report a broken website link is still broken. Bug 385935
    • AJAX is still broken on backbase website. Fails litmus case 4312. Bug 390859

• Mac FT (stephend) - PASS
  • Issues found:
    • Same as Marcia (we collaborated on most)
      • Marcia's "preferences window quakes bug" is bug 394881
  • Areas tested in addition to the above:
    • Location Bar (starring/tagging/bookmarking/autocomplete)
    • Flash r60 plugin
      • Crash on meebo.com
      • Duplicated menu items
    • RSS
      • Ran into bug 329534

Live Spot Checks for M8

When we go live at 3 PM it would be great team members could download an M8 build quickly and make sure everything looks ok.

• Windows XP Tomcat - PASS
• Windows Vista -> Tomcat PASS
• Mac PPC/Intel-> tomcat PASS
• Linux -> Tomcat PASS

Litmus Test Case Issues

• ispiked's Microsummaries test cases are not accessible due to his account not being on people.mozilla.com. https://bugzilla.mozilla.org/show_bug.cgi?id=396237 has been filed to address this.
  • Need an owner for the MS area as this does not fall under the feature categories
  • That owner should correct the test cases to show the new location of the test cases.