QA/Firefox 7

From MozillaWiki
< QA
Revision as of 03:35, 10 August 2011 by Ashughes (talk | contribs)
Jump to navigation Jump to search

« Firefox 6 | Firefox 7 | Firefox 8 »

Firefox 7 Test Plan

Summary

Milestone Firefox 7
Status Currently in Aurora -- merging to Beta on August 16th, 2011
Lead Anthony Hughes (irc: ashughes)
2nd Juan Becerra (irc: juanb)

Latest Build: ftp, http, web

The following is the test plan for Firefox 7 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 7. After the release this document will be used as an archive of what was done to validate this release.

If you have some free time, please pick a task below and get in touch with one of the leads.

Details

The following is a description of the tasks which need to be completed throughout this release.

Daily Tasks
  • [ON TRACK] [softvision] Automation: Check dashboards for new failures
  • [ON TRACK] [softvision] Bug Triage: Check Bugzilla for regressions
  • [ON TRACK] [softvision] Feature Sign-off: Test features and triage feature bugs targeted for this releases
One-time-only Tasks

Automation

Triage the Mozmill dashboard for test failures and debug for potential Firefox regressions.

  • Lead: Vlad Maniac (Softvision)
  • Peer: Alex Lakatos (Softvision)
  • Failure Status: NEW (12) | ASSIGNED (7) | FIXED (3) | REGRESSIONS (0)
  • Tasks:
    • Check the Mozmill dashboard for failing tests for the day
    • Cross reference any failure to the list of NEW and ASSIGNED bugs
      • If the bug is NEW, assign it to yourself and try to fix it
      • If the bug is ASSIGNED and there has been no activity on the bug in over a week, offer your assistance to fix the bug
      • If there is NO bug on file, file it using this template
    • If you need assistance, ask ashughes in the #mozmill channel

Bug Triage

Check UNCONFIRMED, NEW, FIXED, REOPEN, and tracking/status bugs

Important Bugs
  • [FIXED] bug 598425 - Asynchronous layer-based plugin painting on Mac
  • [FIXED] bug 661471 - Create preference(s) to allow specific font families to be forced to use GDI Classic rendering
Notes

Features

Track features which have landed in Firefox 7 and are ready to be tested.

Azure Direct2D backend for canvas

This feature is an implementation of an HTML5 canvas context using the new Azure 2D API with the Direct2D backend. It should give us significantly faster performance on canvas benchmarks and demos with no loss in functionality or quality.

  • Leads: Chris Blizzard (Product), Bas Schouten (Feature, Dev), Curtis Koenig (Security)
  • QA Owner: AndreiD
  • Status: ready for release
  • Links: Feature Page, Tracking Bug
  • Notes:

Embedded Add-on Preferences

Add-on developers should be able to create a series of simple preference options that can be embedded in the detail view of their add-on similar to mobile.

  • Leads: Justin Scott (Product), Dave Townsend (Feature), Geoff Lankow (Dev), Jennifer Boriss (UX)
  • QA Owner: Henrik Skupin
  • Status: landed -- needs testing
  • Links: Tracking Bug
  • Notes:

Telemetry

The goal for this feature is to give our developers the ability to know if changes they are making have wide-ranging positive and negative effects at scale. Another goal of this is to give us easy-to-use infrastructure to learn about the structure of the Internet as a whole.

  • Leads: Chris Blizzard (Product), Taras Glek (Feature, Dev), Curtis Koenig (Security), Sid Stamm (Privacy), Alex Limi (UX)
  • QA Owner: Andrei D
  • Status: landed -- needs testing
  • Links: Tracking Bug
  • Notes:

Beta Merge Sign-off

The following tasks need to be completed to sign-off the Aurora merge.

  • {{|}} [ashughes] Automated BFT execution: details
  • {{|}} [softvision] Manual Smoketest execution: details
  • {{|}} [various] Feature spotchecks: details
  • {{|}} [softvision] Bug fix verifications: details
  • {{|}} [ashughes] Updates from Firefox 5 Aurora to Firefox 6 Aurora: details

Builds: ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-aurora/ (Firefox 7.0a2)
Litmus Testrun: https://litmus.mozilla.org/run_tests.cgi?test_run_id=315

Automated BFTs

The following are the detailed results of the automated BFT test runs.

  • {{|}} Windows 2000: [ functional] | [ restart]
  • {{|}} Windows XP: [ functional] | [ restart]
  • {{|}} Windows Vista: [ functional] | [ restart]
  • {{|}} Windows 7: [ functional] | [ restart]
  • {{|}} Windows 7 64-bit: [ functional] | [ restart]
  • {{|}} Mac OS X 10.6: [ functional] | [ restart]
  • {{|}} Ubuntu 11.04: [ functional] | [ restart]
  • {{|}} Ubuntu 11.04 64-bit: [ functional] | [ restart]
  • Reports: [ Litmus], [ Mozmill]
Issues Found

Smoketests

The following are the detailed results of the manual smoketests.

  • {{|}} [tester] Windows 2000: status
  • {{|}} [tester] Windows XP: status
  • {{|}} [tester] Windows Vista: status
  • {{|}} [tester] Windows 7: status
  • {{|}} [tester] Windows 7 64-bit: status
  • {{|}} [tester] Mac OS X 10.6: status
  • {{|}} [tester] Ubuntu 11.04: status
  • {{|}} [tester] Ubuntu 11.04 32-bit: status
  • Litmus [ testrun], [ report]
  • Firefox 7 builds
Issues Found


Exploratory Testing

Conduct exploratory tests on WebGL, Javascript, Flash, and QuickTime plugin content

  • {{|}} [tester] Windows XP: status
  • {{|}} [tester] Windows 7: status
  • {{|}} [tester] Mac OS X 10.6: status
  • {{|}} [tester] Ubuntu 11.04: status
Issues Found

Feature Spotchecks

The following are the details about spotchecks performed on landed features

  • {{|}} [andreid] Azure Direct2D Backend for Canvas: status
  • {{|}} [whimboo] Embedded Add-on Preferences: status
  • {{|}} [andreid] Telemetry: status
Issues Found

Bug Verifications

The following are the details of bug verifications for Firefox 7

Issues Found

Updates

The following are details of update verifications for Firefox 7 merge to Beta

  • {{|}} [tester] Windows XP: status
    • Source:
    • Target:
  • {{|}} [tester] Windows 7: status
    • Source:
    • Target:
  • {{|}} [tester] Mac OS X 10.6: status
    • Source:
    • Target:
  • {{|}} [tester] Ubuntu 11.04: status
    • Source:
    • Target:
Issues Found

Aurora Merge Sign-off

The following tasks need to be completed to sign-off the Aurora merge.

  • [DONE] [gmealer] Automated BFT execution: details
  • [DONE] [softvision] Manual Smoketest execution: details
  • [DONE] [softvision] Feature spotchecks: details
  • [DONE] [softvision] Bug fix verifications: details
  • [DONE] [ashughes] Updates from Firefox 5 Aurora to Firefox 6 Aurora: details

Builds: ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-aurora/ (Firefox 7.0a2)
Litmus Testrun: https://litmus.mozilla.org/run_tests.cgi?test_run_id=315

Automated BFTs

The following are the detailed results of the automated BFT test runs.

Issues Found

Smoketests

The following are the detailed results of the manual smoketests.

  • [DONE] [simona] Windows 2000: no issues found
  • [DONE] [andreid] Windows XP: no issues found
  • [DONE] [simona] Windows Vista: no issues found
  • [DONE] [vladg] Windows 7: no issues found
  • [DONE] [andreid] Windows 7 64-bit: no issues found
  • [DONE] [vladg] Mac OS X 10.6: no issues found
  • [DONE] [george] Ubuntu 11.04: no issues found
  • [DONE] [george] Ubuntu 11.04 32-bit: no issues found
  • Litmus testrun, report
  • Firefox 7 builds
Issues Found

Feature Spotchecks

The following are the details about spotchecks performed on landed features

  • [DONE] [whimboo] Embedded Add-on Preferences: no issues found
  • [DONE] [andreid] Azure Direct2D Backend for Canvas: no issues found
  • [DONE] [george] Sharing Textures Cross-process for Electrolysis: not landed
  • [DONE] [vladmaniac] Web Console 7: not landed
Issues Found

Bug Verifications

The following are the details of bug verifications for Firefox 7

Issues Found

Updates

The following are details of update verifications for Firefox 7 merge to Aurora

Auroratest
  • [DONE] [ashughes] Windows XP: PASS
    • Source: Mozilla/5.0 (Windows NT 5.1; rv:6.0a2) Gecko/20110705 Firefox/6.0a2
    • Target: Mozilla/5.0 (Windows NT 5.1; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
  • [DONE] [ashughes] Windows 7: PASS
    • Source: Mozilla/5.0 (Windows NT 6.1; rv:6.0a2) Gecko/20110705 Firefox/6.0a2
    • Target: Mozilla/5.0 (Windows NT 6.1; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
  • [DONE] [ashughes] Mac OS X 10.6: PASS
    • Source: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:6.0a2) Gecko/20110702 Firefox/6.0a2
    • Target: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
  • [DONE] [ashughes] Ubuntu 11.04: PASS
    • Source: Mozilla/5.0 (X11; Linux x86_64; rv:6.0a2) Gecko/20110628 Firefox/6.0a2
    • Target: Mozilla/5.0 (X11; Linux x86_64; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
Aurora
  • [DONE] [ashughes] Windows XP: PASS
    • Source: Mozilla/5.0 (Windows NT 5.1; rv:6.0a2) Gecko/20110705 Firefox/6.0a2
    • Target: Mozilla/5.0 (Windows NT 5.1; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
  • [DONE] [ashughes] Windows 7: PASS
    • Source: Mozilla/5.0 (Windows NT 6.1; rv:6.0a2) Gecko/20110705 Firefox/6.0a2
    • Target: Mozilla/5.0 (Windows NT 6.1; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
  • [DONE] [ashughes] Mac OS X 10.6: PASS
    • Source: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:6.0a2) Gecko/20110702 Firefox/6.0a2
    • Target: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
  • [DONE] [ashughes] Ubuntu 11.04: PASS
    • Source: Mozilla/5.0 (X11; Linux x86_64; rv:6.0a2) Gecko/20110628 Firefox/6.0a2
    • Target: Mozilla/5.0 (X11; Linux x86_64; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
Issues Found
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Firefox_7&oldid=337983"