QA/Firefox3/TestResults/Beta4: Difference between revisions

From MozillaWiki
< QA‎ | Firefox3‎ | TestResults
Jump to navigation Jump to search
Line 64: Line 64:
|-
|-
| BFT
| BFT
| Tomcat (running leak tests on FFT) - '''result'''  
| Tomcat - '''result'''  
| Tomcat (running leak tests) - '''result'''
| Tomcat (Vista test suite) - '''result'''  
| Tomcat (running leak tests on FFT) - '''result'''  
| Tomcat - '''result'''  
| Marcia - '''result'''   
| Marcia - '''result'''   
| Tracy - '''result'''
| Tracy - '''result'''
|-
|-
| Top Websites Testing (see below)
| Top Plugins Testing (see below)
| Juanb - '''result'''
| Juanb - '''result'''
| N/A  
| N/A  
Line 80: Line 80:
| Tomcat - '''result'''
| Tomcat - '''result'''
| Juanb - '''result'''
| Juanb - '''result'''
| Tomcat - '''result'''
| Tchung - '''result'''
| Marcia - '''result'''
| Marcia - '''result'''
| Tchung - '''result'''  
| Tchung - '''result'''  
Line 99: Line 99:
|}
|}


=== Top Sites Testing ===
=== Top Plugins Testing ===
Select the top 50 sites, and run through a combination of various web applications.
Query the top plugins supported for the web, and verify they are compatible with Firefox 3. See [https://addons.mozilla.org/en-US/firefox/browse/type:7 AMO list] for recommended plugins.
 
* [http://www.alexa.com Alexa Top 50]
* [[QA/Firefox3/TestPlan/Banking|Banking and financial institutions]]


Testcases
Testcases
* Visit various sites on the list, verifying web compatibility
* On a clean environment, install plugins
* Detect Layout issues, Font issues, formatting problems, broken functionality
* Test updating of old plugins
** Some baseline comparisons is to use Fx 2.0.0.11, Safari, and Trunk nightly builds
* Disable plugins in Addons Manager, verify plugins stop working
* Change browser font sizes, languages and verify web compatibility
* Uninstall plugins, verify websites request a plugin to download
* Operate end to end functionality of a website. Examples:
* Verify websites with popular plugins:
** track an auction item on ebay
** www.nintendo8.com (java) - '''result'''
** save a book on amazon checkout
** www.youtube.com (flash) - '''result'''
** register for a new email account on Yahoo Mail
** www.shockwave.com (shockwave) - '''result'''
** Create a document on Google docs
** www.apple.com/trailers (quicktime) - '''result'''
** Visit a secure site and perform online banking
** www.msnbc.com, www.cnn.com (WMV, RealVideo) - '''result'''
** www.earthwatch.org (Adobe PDF) - '''result'''
** Others like divx, virtual tour, online ads, etc.. - '''result'''


=== Updates Testing ===
=== Updates Testing ===
Line 150: Line 149:
=== Bug Verifications ===
=== Bug Verifications ===
Bugs that are marked RESOLVED FIXED that are checked in for beta.  Please verify them.
Bugs that are marked RESOLVED FIXED that are checked in for beta.  Please verify them.


=== Testday ===
=== Testday ===

Revision as of 19:14, 20 February 2008

« QA/Firefox3/TestResults

Firefox 3 Beta 4 Test Results

Related Pages

Links to separate test results. Some are referenced further below.

Contact Info

PM: Mike Beltzner (beltzner)
Build: John O'Duinn (joduinn)
QA: Tony (tchung)

Schedule

Code Freeze
2/28
Code Build
ETA 3/3
QA test signoff
ETA 3/10

Release Notes

Build Location and IDs

Strategy

  • Areas Covered:
    • Smoketests
    • BFTs
    • Beta4 and past focused Tests
    • Top Plugins
    • Update snippets from Alpha X, Beta X -> Beta 4
    • Memory Leak Testing (handled by Tomcat on debug builds)
    • P1 L10N spotchecks
    • Beta 4 bug verifications
    • Beta 4 Testday
  • Areas Not Covered:
    • localization (String changes) - 3rd party will cover
    • Distribution
    • Major Updates

Test Results

Types of Tests Win XP Win Vista Mac Tiger Mac Leopard Linux
Smoketest marcia - result marcia - result Tomcat - result Tomcat - result Tomcat - result
BFT Tomcat - result Tomcat (Vista test suite) - result Tomcat - result Marcia - result Tracy - result
Top Plugins Testing (see below) Juanb - result N/A Juanb - result N/A Tracy - result
Updates Testing (see below) Tomcat - result Juanb - result Tchung - result Marcia - result Tchung - result
l10N Pri 1 locales Tomcat - result N/A Tchung - result N/A Tracy - result
Bug Verifications Everyone - result Everyone - result Everyone - result Everyone - result Everyone - result

Top Plugins Testing

Query the top plugins supported for the web, and verify they are compatible with Firefox 3. See AMO list for recommended plugins.

Testcases

  • On a clean environment, install plugins
  • Test updating of old plugins
  • Disable plugins in Addons Manager, verify plugins stop working
  • Uninstall plugins, verify websites request a plugin to download
  • Verify websites with popular plugins:
    • www.nintendo8.com (java) - result
    • www.youtube.com (flash) - result
    • www.shockwave.com (shockwave) - result
    • www.apple.com/trailers (quicktime) - result
    • www.msnbc.com, www.cnn.com (WMV, RealVideo) - result
    • www.earthwatch.org (Adobe PDF) - result
    • Others like divx, virtual tour, online ads, etc.. - result

Updates Testing

For b4, we will be doing updates of all Alpha and Beta users to Beta 4. Tests will include different locales for beta 1, but only en-US for all past versions. We will be checking partial updates, complete updates, and failed status.

Will be available on betatest, releasetest, and beta channels

  • Betatest channel - result
  • Releasetest channel - result
  • Beta channel (live) - result

Testcases and Results are tracked here.

L10N P1 Spotchecks

Visual inspection: Go through the P1 list of locales, and run a spotcheck to see if everything visually looks fine. Refer to L10N Requirements for more specifics.

Automation inspection: supported locales will be ran through Minotaur, and results will be reviewed by axel and mic.

Axel's Localization Status Page

P1 Locales to be tested:

  • French (fr) result
  • Chinese (zh-CN) result
  • German (de) result
  • Japanese (ja) result
  • Spanish (es-ES) result
  • Netherlands (nl) result
  • Italian (it) result
  • Finnish (fi) result
  • Hindi (pa-IN) result

Testcases and Results are tracked here.

Bug Verifications

Bugs that are marked RESOLVED FIXED that are checked in for beta. Please verify them.

Testday

A beta 4 testday will be held and call for the community support to assist with testing. Activities include executing litmus testcases, bug verifications, and focus on the new feature areas that are being implemented. More can be found on QMO.

Testday results are posted here.

Detailed Results

Live Spot Checks

Live site downloads for Beta4, after bits are pushed out. Please cover en-US, and 2 or 3 additional Tier 1 locales as well. Pay attention to the following areas:

  • Check for signed installer and applications (windows only)
  • Firstrun page, whatsnew page, Home page, Search plugin
  • Installer, uninstall
  • General spotchecks and browser functionality

Windows XP -> abillings - result
Windows Vista -> stephend - result
Mac Tiger -> Juan - result
Mac Leopard -> Marcia - result
Linux -> stephend - result

Top Issues Reported in Hendrix Against Firefox 3 Beta 4

  • Enter here

Top Issues in b3 in Reporter

  • Enter here
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Firefox3/TestResults/Beta4&oldid=84243"