Releases/Firefox 4RC/Test Plan

From MozillaWiki
Jump to: navigation, search

Firefox 4 Test Plan

Firefox 4 Release Candidate Testplan

Coordinating QA: juanb in irc.mozilla.org, jbecerra@mozilla.com, mevans@mozilla.com, tchung@mozilla.com, or abillings@mozilla.com.

Description of Release

This is the first release candidate of Firefox 4. Unless serious issues are found during in-house testing or its cycle of beta testing, this will be what we ship to users. Historically, we have not shipped the first release candidate.

Test Plan for RC

As in previous milestones, we are going to run our automated tests suites, smoketests, basic functional tests, and updates tests on all shipped locales, using Mozmill, and we'll complement those with manual checks on en-US builds. In addition to the regular basic cycle of testing we'll focus on the following activities, some of which we have already done on previous betas, and some of which are specific to the release candidate:

  • We are going to verify bug fixes since beta 12 and test around them with exploratory testing- Waverley/Smartware contractors - Done
  • Plugins testing across platforms: Flash, Silverlight, Quicktime, Adobe Reader (latest versions), Java, RealPlayer, Shockwave, and Windows Media Player (From the list of most popular plugins in https://addons.mozilla.org/) - Waverley - Done
    • We should spot check Netflix - see marcia for testing account - Done
  • Web Compatibility Testing (Round 3): Popular social, news, games, and entertainment sites (See testing matrix) - uTest - Done
  • Hardware compatibility testing including blocklisting: Performed by outside vendor (See on-going) - iBeta Done
  • Top 50 add-ons testing: basic functionality - Done
  • A round of testing Zynga games through Facebook: Mafia Wars, FarmVille, FrontierVille, using accounts provided by Zynga - marcia, CantinaVille - Done
  • Accessibility - MarcoZ - Signed off
  • A cycle of top sites page load testing: top 20k sites - Tomcat - Done
  • Special areas sign-off: Sync (desktop and mobile) - Tracy/Tchung, Test Pilot/Feedback - juanb Signed off'
  • Crash-stats stability assessment (after 3 full week days) - marcia Pending results
  • Major update and downgrade scenario assessment - Waverley/Smartware/juanb - Done
  • Fx4 in-product web pages sign-off - marcia, juanb
    • First Run, What's New, Release Notes, Getting Started, Snippet Server (about:home)
  • Web of Wonder Demos - Spotcheck for crashes and CPU spikes/usage - marcia, juanb - Done
  • Web properties sign-off: AMO, SUMO, Socorro, Input - WebQA
  • An RC-specific testday or possible meetup - QA team - Done
  • Test coverage of quit dialog scenarios - ashughes, need others to help, zpao on hand to answer questions

During the time the RC is in the hand of users we'll be monitoring crash stats as well as feedback looking for regressions that may prompt another release candidate. We'll work closely with the support team to identify any new issues and raise them to the attention of the release drivers should they require another RC.

Any Further RCs

Should there be a need for another release candidate, we are not going to go through all of the above activities on the follow-up RC. We'll devote our time verifying the bug fixes and testing around them depending on the types of issues found. We are going to rely on new unconfirmed bugs and user feedback to assess the stability of RC. We expect the scope of issues, whether they be regressions or new problems, to be very narrow.

Schedule

When it's ready.

Builds

  • Builds for RC2 (build3) candidates can be found: here
  • Builds for RC2 (build2) candidates can be found: here.
  • Builds for all RC1 candidates can be found here.

Test Results

RC2 Build 3 Final (Current)

RC2 Build 2

RC1 Build 1

Retrieved from "https://wiki.mozilla.org/index.php?title=Releases/Firefox_4RC/Test_Plan&oldid=295208"