Releases/Firefox 7/Test Plan

From MozillaWiki
Jump to: navigation, search

« Firefox 6 | Firefox 7 | Firefox 8 »

Firefox 7 Test Plan

Summary

Milestone Firefox 7
Status Currently in Beta -- releasing on September 27th, 2011
Lead Anthony Hughes (irc: ashughes)
2nd Juan Becerra (irc: juanb)

Latest Build: ftp, http, web

The following is the test plan for Firefox 7 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 7. After the release this document will be used as an archive of what was done to validate this release.

If you have some free time, please pick a task below and get in touch with one of the leads.

Details

The following is a description of the tasks which need to be completed throughout this release.

Daily Tasks
  • [ON TRACK] [softvision] Automation: Check dashboards for new failures
  • [ON TRACK] [softvision] Bug Triage: Check Bugzilla for regressions
  • [ON TRACK] [softvision] Features: Test features and triage feature bugs targeted for this releases
  • [ON TRACK] [softvision] Input: Triage Input.MO for prevalent issues and feature requests
One-time-only Tasks

Automation

BFTs

Triage the Mozmill dashboard for test failures and debug for potential Firefox regressions.

  • Lead: Vlad Maniac (Softvision)
  • Peer: Alex Lakatos (Softvision)
  • Failure Status: NEW (12) | ASSIGNED (7) | FIXED (3) | REGRESSIONS (0)
  • Tasks:
    • Check the Mozmill dashboard for failing tests for the day
    • Cross reference any failure to the list of NEW and ASSIGNED bugs
      • If the bug is NEW, assign it to yourself and try to fix it
      • If the bug is ASSIGNED and there has been no activity on the bug in over a week, offer your assistance to fix the bug
      • If there is NO bug on file, file it using this template
    • If you need assistance, ask ashughes in the #mozmill channel

Endurance

This task is only necessary during the Nightly phase

Triage and report the performance numbers from the daily Endurance tests.

  • Lead: Vlad Maniac (Softvision)
  • Peer: Alex Lakatos (Softvision)
  • Status: SIGNED-OFF
  • Tasks:
    • Get the results from the Endurance reports
    • Record the result in the Endurance spreadsheet
    • Report any spikes in performance degradation by emailing the QA release driver
Issues Found

Bug Triage

Daily triage of the UNCONFIRMED, NEW, FIXED, REOPEN, and tracking/status bugs

  • Lead: Vlad Maniac (Softvision)
  • Peer: Alex Lakatos (Softvision)
  • Status: ON-GOING
  • Bugs: QA+, FIXED
  • Results: VERIFIED, REOPENED
  • Tips:
    • If you don't know how to verify a bug, just skip it, move onto the next bug
    • Ignore code-level bugs, bugs in tests, build issues
Issues Found

Features

Track features which have landed in Firefox 7 and are ready to be tested (spreadsheet)

Azure Direct2D backend for canvas

This feature is an implementation of an HTML5 canvas context using the new Azure 2D API with the Direct2D backend. It should give us significantly faster performance on canvas benchmarks and demos with no loss in functionality or quality.

  • Leads: Chris Blizzard (Product), Bas Schouten (Feature, Dev), Curtis Koenig (Security)
  • QA Owner: AndreiD
  • Status: QA sign-off for the current status of the feature development
  • Links: Feature Page, Tracking Bug
  • Notes:

Embedded Add-on Preferences

Add-on developers should be able to create a series of simple preference options that can be embedded in the detail view of their add-on similar to mobile.

  • Leads: Justin Scott (Product), Dave Townsend (Feature), Geoff Lankow (Dev), Jennifer Boriss (UX)
  • QA Owner: Henrik Skupin
  • Status: landed -- needs testing
  • Links: Tracking Bug
  • Notes:

Telemetry

The goal for this feature is to give our developers the ability to know if changes they are making have wide-ranging positive and negative effects at scale. Another goal of this is to give us easy-to-use infrastructure to learn about the structure of the Internet as a whole.

  • Leads: Chris Blizzard (Product), Taras Glek (Feature, Dev), Curtis Koenig (Security), Sid Stamm (Privacy), Alex Limi (UX)
  • QA Owner: Andrei D
  • Status: QA sign-off for the current status of the feature development
  • Links: Tracking Bug
  • Notes:

Input

The following are the details of daily triage conducted on http://input.mozilla.org.

Popular Feedback

Full Functional Tests

The following tracks testing done by iBeta for full functional test coverage. This task is performed once during Aurora only.

  • Lead: Anthony Hughes
  • [ link to test suite]
  • [ link to results of testing]
  • [ link to bugs filed by iBeta]

Website Compatibility

The following tracks testing done by uTest for website compatibility. This task is performed after every merge.

  • Lead: Anthony Hughes
  • Link to test suite
  • Link to report for Aurora Web Compatibility
  • Link to report for Beta Web Compatibility
  • Link to bugs filed

Aurora Merge

The following tasks need to be completed to sign-off the Aurora merge.

  • [DONE] [gmealer] Automated tests: details
  • [DONE] [softvision] Manual tests: details
  • [DONE] [softvision] Feature sign-off: details
  • [DONE] [ashughes] Updates: details

Builds: ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-aurora/ (Firefox 7.0a2)
Litmus Testrun: https://litmus.mozilla.org/run_tests.cgi?test_run_id=315

Automated Tests

The following are the detailed results of the automated BFT test runs.

Issues Found
  • Two tests are constantly failing and which need to be fixed as soon as possible.

Manual Tests

Smoketests

The following are the detailed results of the manual smoketests

  • [DONE] [simona] Windows 2000: no issues found
  • [DONE] [andreid] Windows XP: no issues found
  • [DONE] [simona] Windows Vista: no issues found
  • [DONE] [vladg] Windows 7: no issues found
  • [DONE] [andreid] Windows 7 64-bit: no issues found
  • [DONE] [vladg] Mac OS X 10.6: no issues found
  • [DONE] [george] Ubuntu 11.04: no issues found
  • [DONE] [george] Ubuntu 11.04 64-bit: no issues found
  • Litmus testrun, report
  • Firefox 7 builds
Issues Found

Feature Sign-off

The following are the details about spotchecks performed on landed features

  • [DONE] [whimboo] Embedded Add-on Preferences: no issues found
  • [DONE] [andreid] Azure Direct2D Backend for Canvas: no issues found
  • [DONE] [george] Sharing Textures Cross-process for Electrolysis: not landed
  • [DONE] [vladmaniac] Web Console 7: not landed
Issues Found

Updates

The following are details of update verifications for Firefox 7 merge to Aurora

Auroratest
  • [DONE] [ashughes] Windows XP: PASS
    • Source: Mozilla/5.0 (Windows NT 5.1; rv:6.0a2) Gecko/20110705 Firefox/6.0a2
    • Target: Mozilla/5.0 (Windows NT 5.1; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
  • [DONE] [ashughes] Windows 7: PASS
    • Source: Mozilla/5.0 (Windows NT 6.1; rv:6.0a2) Gecko/20110705 Firefox/6.0a2
    • Target: Mozilla/5.0 (Windows NT 6.1; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
  • [DONE] [ashughes] Mac OS X 10.6: PASS
    • Source: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:6.0a2) Gecko/20110702 Firefox/6.0a2
    • Target: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
  • [DONE] [ashughes] Ubuntu 11.04: PASS
    • Source: Mozilla/5.0 (X11; Linux x86_64; rv:6.0a2) Gecko/20110628 Firefox/6.0a2
    • Target: Mozilla/5.0 (X11; Linux x86_64; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
Aurora
  • [DONE] [ashughes] Windows XP: PASS
    • Source: Mozilla/5.0 (Windows NT 5.1; rv:6.0a2) Gecko/20110705 Firefox/6.0a2
    • Target: Mozilla/5.0 (Windows NT 5.1; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
  • [DONE] [ashughes] Windows 7: PASS
    • Source: Mozilla/5.0 (Windows NT 6.1; rv:6.0a2) Gecko/20110705 Firefox/6.0a2
    • Target: Mozilla/5.0 (Windows NT 6.1; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
  • [DONE] [ashughes] Mac OS X 10.6: PASS
    • Source: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:6.0a2) Gecko/20110702 Firefox/6.0a2
    • Target: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
  • [DONE] [ashughes] Ubuntu 11.04: PASS
    • Source: Mozilla/5.0 (X11; Linux x86_64; rv:6.0a2) Gecko/20110628 Firefox/6.0a2
    • Target: Mozilla/5.0 (X11; Linux x86_64; rv:7.0a2) Gecko/20110706 Firefox/7.0a2
Issues Found

Beta 1

The following tasks need to be completed to sign-off the Beta merge

  • [DONE] [whimboo] Automated Tests: details
  • [DONE] [softvision] Manual Tests: details
  • [ON TRACK] [ashughes] Updates: details

Builds: Firefox 7.0b1(1)
Litmus:testrun

Automated Tests

The following are the detailed results of the automated BFT test runs.

Issues Found
  • Two tests are constantly failing and which need to be fixed as soon as possible.

Manual Tests

Smoketests

The following are the detailed results of the manual smoketests.

  • [DONE] [Mihaela] Windows 2000: PASS
  • [DONE] [Virgil] Windows XP: PASS
  • [DONE] [Simona] Windows Vista: PASS
  • [DONE] [Simona] Windows 7: PASS
  • [DONE] [Andrei] Windows 7 64-bit: PASS
  • [DONE] [VladG] Mac OS X 10.6: PASS
  • [DONE] [VladG] Mac OS X 10.7: PASS
  • [DONE] [Alin] Ubuntu 11.04: PASS
  • [DONE] [Alin] Ubuntu 11.04 64-bit: PASS
  • Litmus testrun, report
  • Firefox 7.0b1 build1
Exploratory Tests

The following are the detailed results of the exploratory testing - Web App compatibility

  • [DONE] [Mihaela] Windows 2000: PASS with issue
  • [DONE] [Virgil] Windows XP: PASS with issue
  • [DONE] [Simona] Windows Vista: PASS
  • [DONE] [Simona] Windows 7: PASS
  • [DONE] [Andrei] Windows 7 64-bit: PASS
  • [DONE] [VladG] Mac OS X 10.6: PASS with issue
  • [DONE] [VladG] Mac OS X 10.7: PASS with issue
  • [DONE] [Alin] Ubuntu 11.04: PASS
  • [DONE] [Alin] Ubuntu 11.04 64-bit: PASS
  • Litmus report
Issues Found
  • Bug 650741 Downloading pdf files is no longer possible - Confirmed
  • Bug 555694 Autocomplete should not suggest switching to the current tab - Confirmed
  • Bug 679701 Embedded window media player plugin is not displayed properly - NEW

Updates

The following are details of update verifications for Firefox 7 beta 1

Betatest: report

  • [DONE] [ashughes] Windows 2000: SIGNED-OFF
  • [DONE] [ashughes] Windows XP: SIGNED-OFF
  • [DONE] [ashughes] Windows Vista: SIGNED-OFF
  • [DONE] [ashughes] Windows 7: SIGNED-OFF
  • [DONE] [ashughes] Windows 7 64-bit: SIGNED-OFF
  • [DONE] [ashughes] Mac OS X 10.6: SIGNED-OFF
  • [DONE] [ashughes] Mac OS X 10.7: SIGNED-OFF
  • [DONE] [ashughes] Ubuntu 11.04: SIGNED-OFF
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: SIGNED-OFF

Beta: report

  • [DONE] [ashughes] Windows 2000: SIGNED-OFF
  • [DONE] [ashughes] Windows XP: SIGNED-OFF
  • [DONE] [ashughes] Windows Vista: SIGNED-OFF
  • [DONE] [ashughes] Windows 7: SIGNED-OFF
  • [DONE] [ashughes] Windows 7 64-bit: SIGNED-OFF
  • [DONE] [ashughes] Mac OS X 10.6: SIGNED-OFF
  • [DONE] [ashughes] Mac OS X 10.7: SIGNED-OFF
  • [DONE] [ashughes] Ubuntu 11.04: SIGNED-OFF
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: SIGNED-OFF
Issues Found

Beta 2

The following tasks need to be completed to sign-off the Beta

  • [DONE] [whimboo] Automated Tests: details
  • [DONE] [softvision] Manual Tests: details
  • [DONE] [softvision] Bug Verifications: details
  • [DONE] [ashughes] Updates: details

Builds: Firefox 7.0b2
Litmus:testrun

Automated Tests

The following are the detailed results of the automated BFT test runs.

Issues Found

Manual Tests

Smoketests

The following are the detailed results of the manual smoketests.

  • [DONE] [Alin] Windows 2000: PASS
  • [DONE] [IoanaB] Windows XP: PASS
  • [DONE] [Simona] Windows Vista: PASS
  • [DONE] [Andrei] Windows 7: PASS
  • [DONE] [Mihaela] Windows 7 64-bit: PASS
  • [DONE] [VladG] Mac OS X 10.6: PASS
  • [DONE] [VladG] Mac OS X 10.7: PASS
  • [DONE] [Virgil] Ubuntu 11.04: PASS
  • [DONE] [Virgil] Ubuntu 11.04 64-bit: PASS
  • Reports Litmus
Exploratory Tests

The following are the detailed results of the exploratory testing - Web App compatibility

  • [DONE] [Alin] Windows 2000: PASS
  • [DONE] [IoanaB] Windows XP: PASS
  • [DONE] [Simona] Windows Vista: PASS
  • [DONE] [Andrei] Windows 7: PASS
  • [DONE] [Mihaela] Windows 7 64-bit: PASS
  • [DONE] [VladG] Mac OS X 10.6: PASS
  • [DONE] [VladG] Mac OS X 10.7: PASS
  • [DONE] [IoanaB] Ubuntu 11.04: PASS
  • [DONE] [Virgil] Ubuntu 11.04 64-bit: PASS
  • Reports Litmus
Issues Found
  • Bug 681922 In google docs Upload -> Files has no highlighting - NEW
Features Spotchecks

If you own a feature this release and feel the need to spotcheck it, please do so and record any issues you find below.

  • [DONE] [Andrei] Azure Direct2D backend for canvas PASS
  • [DONE] [Andrei] Telemetry PASS
  • [DONE] [VladM] Embedded Add-on Preferences PASS
Issues Found

Bug Verifications

Verify as many of these bug fixes as possible, ignoring build, code-level, and test failure bugs. Report any REOPENED bugs below.

Issues Found
  • bug 670318 - Google+ home page creates multiple SHEntries, and if you go back to the first one, you can no longer go forward

Updates

The following are details of update verifications for Firefox 7 beta 2

Betatest: report

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS

Beta: report

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS
Issues Found

Beta 3

THIS RELEASE IS A CHEMSPILL

The following tasks need to be completed to sign-off the Beta

  • [DONE] [geo/whimboo] Automated Tests: details
  • [skipped] [softvision] Manual Tests: details
  • [skipped] [softvision] Bug Verifications: details
  • [ON TRACK] [ashughes] Updates: details

Builds: Firefox 7.0b3
Litmus:testrun

Automated Tests

The following are the detailed results of the automated BFT test runs.

  • [DONE] Windows 2000: PASS
  • [DONE] Windows XP: PASS
  • [DONE] Windows Vista: PASS
  • [DONE] Windows 7: PASS
  • [DONE] Windows 7 64-bit: PASS
  • [DONE] Mac OS X 10.6: PASS
  • [DONE] Mac OS X 10.7: PASS
  • [DONE] Ubuntu 10.10: PASS
  • [DONE] Ubuntu 10.10 64-bit: PASS
  • Reports: Mozmill (1 Known Failure)
Issues Found

Manual Tests [SKIPPED]

Smoketests

The following are the detailed results of the manual smoketests.

  • [ON TRACK] [tester] Windows 2000: result
  • [ON TRACK] [tester] Windows XP: result
  • [ON TRACK] [tester] Windows Vista: result
  • [ON TRACK] [tester] Windows 7: result
  • [ON TRACK] [tester] Windows 7 64-bit: result
  • [ON TRACK] [tester] Mac OS X 10.6: result
  • [ON TRACK] [tester] Mac OS X 10.7: result
  • [ON TRACK] [tester] Ubuntu 11.04: result
  • [ON TRACK] [tester] Ubuntu 11.04 64-bit: result
  • Reports Litmus
Exploratory Tests

The following are the detailed results of the exploratory testing - Web App compatibility

  • [ON TRACK] [tester] Windows 2000: result
  • [ON TRACK] [tester] Windows XP: result
  • [ON TRACK] [tester] Windows Vista: result
  • [ON TRACK] [tester] Windows 7: result
  • [ON TRACK] [tester] Windows 7 64-bit: result
  • [ON TRACK] [tester] Mac OS X 10.6: result
  • [ON TRACK] [tester] Mac OS X 10.7: result
  • [ON TRACK] [tester] Ubuntu 11.04: result
  • [ON TRACK] [tester] Ubuntu 11.04 64-bit: result
  • Reports Litmus
Issues Found
Features Spotchecks

If you own a feature this release and feel the need to spotcheck it, please do so and record any issues you find below.

  • [ON TRACK] [Andrei] Azure Direct2D backend for canvas result
  • [ON TRACK] [Andrei] Telemetry result
  • [ON TRACK] [VladM] Embedded Add-on Preferences result
Issues Found

Bug Verifications [SKIPPED]

Verify as many of these bug fixes as possible, ignoring build, code-level, and test failure bugs. Report any REOPENED bugs below.

Issues Found

Updates

The following are details of update verifications for Firefox 7 beta 3

Releasetest: report

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS

Beta: report]

  • [ON TRACK] [ashughes] Windows 2000: result
  • [ON TRACK] [ashughes] Windows XP: result
  • [ON TRACK] [ashughes] Windows Vista: result
  • [ON TRACK] [ashughes] Windows 7: result
  • [ON TRACK] [ashughes] Windows 7 64-bit: result
  • [ON TRACK] [ashughes] Mac OS X 10.6: result
  • [ON TRACK] [ashughes] Mac OS X 10.7: result
  • [ON TRACK] [ashughes] Ubuntu 11.04: result
  • [ON TRACK] [ashughes] Ubuntu 11.04 64-bit: result
Issues Found

Beta 4

THIS RELEASE IS A CHEMSPILL

The following tasks need to be completed to sign-off the Beta

  • [ON TRACK] [geo/hskupin] Automated Tests: details
  • [ON TRACK] [hskupin/juanb] Bug Verifications: details
  • [ON TRACK] [hskupin] Updates: details

Builds: Firefox 7.0b4
Litmus:testrun, report

Automated Tests

The following are the detailed results of the automated BFT test runs.

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS
  • Reports: Mozmill(1 Known Failure)
Issues Found

Bug Verifications

  • bug 669792 - gov.uk is not considered to be a public suffix
  • bug 682927 - Dis-trust DigiNotar root certificate
  • bug 683261 - Better coverage for DigiNotarGate in NSS
  • bug 683449 - Remove the exemptions for the Staat der Nederlanden root
  • bug 683883 - Improve DigitNotarGate handling in PSM

Updates

The following are details of update verifications for Firefox 7 beta 4

Releasetest: report

  • [DONE] [whimboo] Windows 2000: PASS
  • [DONE] [whimboo] Windows XP: PASS
  • [DONE] [whimboo] Windows Vista: PASS
  • [DONE] [whimboo] Windows 7: PASS
  • [DONE] [whimboo] Windows 7 64-bit: PASS
  • [DONE] [whimboo] Mac OS X 10.6: PASS
  • [DONE] [whimboo] Mac OS X 10.7: PASS
  • [DONE] [whimboo] Ubuntu 11.04: PASS
  • [DONE] [whimboo] Ubuntu 11.04 64-bit: PASS

Beta: report

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: 'PASS
Issues Found

Exploratory testing

  • Resources
    • (please state your names)
  • Testing activities
    • Tests (please provide a link with the exploratory tests/scenarios you are testing around)
    • Other testing activities (corner cases, interactions with other features)
  • Results (tester 1)
    • OS 1
    • OS 2
    • ...
  • Results (tester 2)
    • OS ..

Beta 5

The following tasks need to be completed to sign-off the Beta

  • [DONE] [ashughes] Automated Tests: details
  • [DONE] [softvision] Manual Tests: details
  • [DONE] [softvision] Bug Verifications: details
  • [DONE] [ashughes] Updates: details

Builds: Firefox 7.0b5
Litmus:testrun

Automated Tests

The following are the detailed results of the automated BFT test runs.

  • [DONE] Windows 2000: PASS
  • [DONE] Windows XP: PASS
  • [DONE] Windows Vista: PASS
  • [DONE] Windows 7: PASS
  • [DONE] Windows 7 64-bit: PASS
  • [DONE] Mac OS X 10.6: PASS
  • [DONE] Mac OS X 10.7: PASS
  • [DONE] Ubuntu 11.04: PASS
  • [DONE] Ubuntu 11.04 64-bit: PASS
  • Reports: Litmus, Mozmill
Issues Found

Manual Tests

Smoketests

The following are the detailed results of the manual smoketests.

  • [DONE] [Simona] Windows 2000: PASS
  • [DONE] [Ioana] Windows XP: PASS
  • [DONE] [Simona] Windows Vista: PASS
  • [DONE] [AndreiD] Windows 7: PASS
  • [DONE] [AndreiD] Windows 7 64-bit: PASS
  • [DONE] [VladG] Mac OS X 10.6: PASS
  • [DONE] [VladG] Mac OS X 10.7: PASS
  • [DONE] [Virgil] Ubuntu 11.04 32-bit: PASS
  • [DONE] [Mihaela] Ubuntu 11.04 64-bit: PASS
  • Reports Litmus
  • [DONE] [gmontagu] Windows 7 (es-AR): PASS
Web App Compatibility

The following are the detailed results of Web App compatibility testing

  • [DONE] [Simona] Windows 2000: PASS
  • [DONE] [Ioana] Windows XP: PASS
  • [DONE] [Simona] Windows Vista: PASS
  • [DONE] [AndreiD Windows 7: PASS
  • [DONE] [AndreiD] Windows 7 64-bit: PASS
  • [DONE] [VladG] Mac OS X 10.6: PASS
  • [DONE] [VladG] Mac OS X 10.7: PASS
  • [DONE] [Virgil] Ubuntu 11.04 32-bit: PASS
  • [DONE] [Mihaela] Ubuntu 11.04 64-bit: PASS
  • Reports Litmus
  • [DONE] [gmontagu] Windows 7 (es-AR): PASS
Exploratory Testing

The following are the detailed results of any exploratory testing

  • [DONE] [Virgil] Add-on Compatibility: PASS
    • Check for incompatible add-ons after updating from a previous Firefox
    • Sample SUMO issues: 862160, 862461, 862143
  • [DONE] [Ioana] Session Store: Pass - bug #68573 was updated
    • Check restoration of session after updating from a previous version of Firefox
    • Sample SUMO issues: 869407, 862722, 863248
    • Reference Bugs: bug 685731
  • [DONE] [Mihaela] Profile Retention: PASS
    • Check for loss of places/signon/cookies sqlite files or complete loss of profile after updating from a previous version of Firefox
    • Sample SUMO issues: 864148, 862133, 863067
  • Results: Exploratory work spreadsheet
Issues Found

Bug Verifications

Verify as many of these bug fixes as possible, ignoring build, code-level, and test failure bugs. Report any REOPENED bugs below.

Issues Found

Updates

The following are details of update verifications for Firefox 7 beta 5

Betatest: report

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS

Releasetest: report

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS

Beta: report

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS
Issues Found

Beta 6

The following tasks need to be completed to sign-off the Beta

  • [DONE] [ashughes] Automated Tests: details
  • [ON TRACK] [softvision] Manual Tests: details
  • [ON TRACK] [softvision] Bug Verifications: details
  • [ON TRACK] [ashughes] Updates: details

Builds: Firefox 7.0b6
Litmus:testrun

Automated Tests

The following are the detailed results of the automated BFT test runs.

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS
  • Reports: Litmus, Mozmill

l10n: report

  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 9.10: PASS
Issues Found

Manual Tests

Smoketests

The following are the detailed results of the manual smoketests.

  • [DONE] [Simona] Windows 2000: PASS
  • [DONE] [Ioana] Windows XP: PASS
  • [DONE] [Simona] Windows Vista: PASS
  • [DONE] [Ioana] Windows 7: PASS
  • [DONE] [Mihaela] Windows 7 64-bit: PASS
  • [DONE] [VladG] Mac OS X 10.6: PASS
  • [DONE] [VladG] Mac OS X 10.7: PASS
  • [DONE] [Virgil] Ubuntu 11.04: PASS
  • [DONE] [Virgil] Ubuntu 11.04 64-bit: PASS
  • Testrun: testrun
  • Report: Litmus
Web App Compatibility

The following are the detailed results of Web App compatibility testing

  • [DONE] [Simona] Windows 2000: PASS
  • [DONE] [Ioana] Windows XP: PASS
  • [DONE] [Simona] Windows Vista: PASS
  • [DONE] [Ioana] Windows 7: PASS
  • [DONE] [Mihaela] Windows 7 64-bit: PASS
  • [DONE] [VladG] Mac OS X 10.6: PASS
  • [DONE] [VladG] Mac OS X 10.7: PASS
  • [DONE] [Virgil] Ubuntu 11.04: PASS
  • [DONE] [Virgil] Ubuntu 11.04 64-bit: PASS
  • Testrun: testrun
  • Report: Litmus
Exploratory Testing

The following are the detailed results of any exploratory testing

  • [DONE] [Ioana] Tabbed browsing: PASS
  • [DONE] [VladG] Add-ons Manager: PASS
  • [DONE] [Virgil] Panorama: PASS
  • [DONE] [Simona] Menus&Themes: PASS
  • [DONE] [Andrei] Azure|Canvas: PASS
  • [DONE] [Mihaela] Bookmarks&History: PASS
  • Results: Exploratory work spreadsheet
Issues Found
  • Bug 687469 - Multiple empty nodes are appended to history sidebar after opening a website from it

Bug Verifications

Verify as many of these bug fixes as possible, ignoring build, code-level, and test failure bugs. Report any REOPENED bugs below.

Issues Found
  • Bug 676690 - Could not change status to VERIFIED, since a Javascript code freezes Firefox only on Win7

Updates

The following are details of update verifications for Firefox 7 beta 6

Releasetest: report

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS
  • NOTE: failures of fallback updates for 7.0b5 locales are not reproducible manually (bug 687231)

Beta: report

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS
  • NOTE: failures of fallback updates for 7.0b5 locales are not reproducible manually (bug 687231)

l10n: report (betatest)

  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Ubuntu 10.10: PASS
Issues Found

Final

The following tasks need to be completed to sign-off the Beta

Automated Tests

The following are the detailed results of the automated BFT test runs.

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS
  • Reports: Litmus, Mozmill

l10n: report

  • [DONE] [whimboo] Windows XP: PASS
  • [DONE] [whimboo] Mac OS X 10.6: PASS
  • [DONE] [whimboo] Ubuntu 10.10: PASS
Issues Found

Manual Tests

Smoketests

The following are the detailed results of the manual smoketests.

  • [DONE] [Simona] Windows 2000: PASS
  • [DONE] [Mihaela] Windows XP: PASS
  • [DONE] [Simona] Windows Vista: PASS
  • [DONE] [Ioana] Windows 7: PASS
  • [DONE] [Mihaela] Windows 7 64-bit: PASS
  • [DONE] [VladG] Mac OS X 10.6: PASS
  • [DONE] [VladG] Mac OS X 10.7: PASS
  • [DONE] [Ioana] Ubuntu 11.04: PASS
  • [DONE] [Virgil] Ubuntu 11.04 64-bit: PASS
  • Links: testrun, report
Exploratory Testing

The following are the detailed results of any exploratory testing

  • Suggested areas: plugins, video, social networking, games, banking, shopping, webmail
  • [DONE] Updated flash version (10.3.183.10)
    • run flash content on an older version of flash, use the updater, rerun flash content with the latest version of flash
    • [andreid] Platform: Windows 7 PASS
    • [Virgil Dicu] Platform: Ubuntu 64 PASS
    • [Ioana] Platform: Windows XP PASS
    • [VladG] Platform: Mac OS X 10.6 PASS
  • [DONE] Session store on update
    • create a loaded session in an older version of Firefox, update to 7.0RC and see if session is restored
    • [Ioana] Platform: Windows 7 PASS
    • [Virgil Dicu] Platform: Ubuntu 64 PASS
    • [Ioana] Platform: Windows XP PASS
    • [VladG] Platform: Mac OS X 10.6 PASS
  • [DONE] Other focus areas
    • [andreid] Platform: Windows 7 PASS
    • [Virgil Dicu] Platform: Ubuntu 64 PASS
    • [Ioana] Platform: Windows XP PASS
    • [VladG] Platform: Mac OS X 10.6 PASS
Issues Found - [7rc] whiteboard
  • Bug 675645 - Firefox freezes when using Pop out while in Fullscreen mode on Youtube - Confirmed
  • bug 688710 - Middle click on a video link opens a blank tab - NEW INVALID
  • Bug 687828 - Print through the Adobe Flash Plugin freezes Firefox on ikea.com - NEW
  • Bug 622417 - Flash plugin overlaps content element on youtube - NEW

Bug Verifications

Verify all of these bugs

  • verify the fix on Firefox 7, 8 and 9 as necessary
  • add the verified-beta keyword if fixed on Firefox 7
  • add the verified-aurora keyword if fixed on Firefox 8
  • set the status to VERIFIED FIXED if fixed on the version of Firefox Nightly at the time of mozilla-central check-in
  • set [qa+] whiteboard tag to [qa!] if verified across all branches
  • Refer to Triage Howto, comment on the bug or ask ashughes if unsure about anything
Results
Issues Found
  • bug 652721 - Minor robustness improvements to Ogg - REOPENED

Updates

The following are details of update verifications for Firefox 7 Final

Releasetest: report

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS

Release: report

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS
Issues Found

7.0.1

The following tasks need to be completed to sign-off

Automated Tests

The following are the detailed results of the automated BFT test runs.

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS
  • Reports: Litmus, Mozmill
Issues Found

Manual Tests

Smoketests

The following are the detailed results of the manual smoketests.

  • [DONE] [Simona] Windows 2000: PASS
  • [DONE] [Ioana] Windows XP: PASS
  • [DONE] [Simona] Windows Vista: PASS
  • [DONE] [Mihaela] Windows 7: PASS
  • [DONE] [Ioana] Windows 7 64-bit: PASS
  • [DONE] [VladG] Mac OS X 10.6: PASS
  • [DONE] [VladG] Mac OS X 10.7: PASS
  • [DONE] [Ioana] Ubuntu 11.04: PASS
  • [DONE] [Virgil] Ubuntu 11.04 64-bit: PASS
  • Links: report
Issues Found

Bug Verifications

The following bugs need verification across ALL platforms.

  • bug 680802 - Upgrade Firefox when there is an add-on update waiting to be installed uninstalls the add-on
  • Also check the Restartless Add-on Fixer works for broken builds
Results
  • [DONE] [Simona] Windows 2000: PASS
  • [DONE] [MIhaela] Windows XP: PASS
  • [DONE] [Simona] Windows Vista: PASS
  • [DONE] [Mihaela] Windows 7: PASS
  • [DONE] [Ioana] Windows 7 64-bit: PASS
  • [DONE] [VladG] Mac OS X 10.6: PASS
  • [DONE] [VladG] Mac OS X 10.7: PASS
  • [DONE] [Virgil] Ubuntu 11.04: PASS
  • [DONE] [Virgil] Ubuntu 11.04 64-bit: PASS
Issues Found

Updates

The following are details of update verifications for Firefox 7 Final

Releasetest: report

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS

Release: report

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS
Issues Found

Post-Mortem Notes

  • Need a way to test add-on updates
    • latest version
    • old version updated to latest
    • old version update downloaded but not installed
    • old version of an add-on
    • compatible
    • incompatible
    • Test Strategy: Manual tests in Litmus Smoketests, automate where we can
  • QA aggressive reaction
    • Director escalate issues to Lead immediately
    • Lead needs to develop plan, get Softvision working on bug reproduction, and communicate aggressively
Retrieved from "https://wiki.mozilla.org/index.php?title=Releases/Firefox_7/Test_Plan&oldid=353387"