Releases/Firefox 8/Test Plan: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
Line 641: Line 641:




'''Releasetest:''' [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=8.0&channel=releasetest&from=2011-10-11&to=2011-10-13&target=8.0 report]
'''Releasetest:''' [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=8.0&channel=releasetest&from=2011-10-13&to=2011-10-13&target=8.0 report]
* {{ok|}} [ashughes] Windows 2000: '''RESULT'''
* {{done|}} [ashughes] Windows 2000: '''PASS'''
* {{ok|}} [ashughes] Windows XP: '''RESULT'''
* {{done|}} [ashughes] Windows XP: '''PASS'''
* {{ok|}} [ashughes] Windows Vista: '''RESULT'''
* {{done|}} [ashughes] Windows Vista: '''PASS'''
* {{ok|}} [ashughes] Windows 7: '''RESULT'''
* {{done|}} [ashughes] Windows 7: '''PASS'''
* {{ok|}} [ashughes] Windows 7 64-bit: '''RESULT'''
* {{done|}} [ashughes] Windows 7 64-bit: '''PASS'''
* {{ok|}} [ashughes] Mac OS X 10.6: '''RESULT'''
* {{done|}} [ashughes] Mac OS X 10.6: '''PASS'''
* {{ok|}} [ashughes] Mac OS X 10.7: '''RESULT'''
* {{done|}} [ashughes] Mac OS X 10.7: '''PASS'''
* {{ok|}} [ashughes] Ubuntu 11.04: '''RESULT'''
* {{done|}} [ashughes] Ubuntu 11.04: '''PASS'''
* {{ok|}} [ashughes] Ubuntu 11.04 64-bit: '''RESULT'''
* {{done|}} [ashughes] Ubuntu 11.04 64-bit: '''PASS'''


'''Beta:''' [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=8.0&channel=beta&from=2011-10-11&to=2011-10-13&target=8.0 report]
'''Beta:''' [http://mozmill-release.brasstacks.mozilla.com/#/update/detail?branch=8.0&channel=beta&from=2011-10-11&to=2011-10-13&target=8.0 report]

Revision as of 17:07, 13 October 2011

« Firefox 7 | Firefox 8 | Firefox 9 »

Firefox 8 Test Plan

Summary

Milestone Firefox 8
Status Currently in Aurora -- merging to Beta on September 27th, 2011
Lead Juan Becerra (irc: juanb)
2nd Anthony Hughes (irc: ashughes)

Latest Build: ftp, http, web

The following is the test plan for Firefox 8 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 8. After the release this document will be used as an archive of what was done to validate this release.

If you have some free time, please pick a task below and get in touch with one of the leads.

Details

The following is a description of the tasks which need to be completed throughout this release.

Daily Tasks
  • [ON TRACK] [softvision] Automation: Check dashboards for new failures
  • [ON TRACK] [softvision] Bug Triage: Check Bugzilla for regressions
  • [ON TRACK] [softvision] Features: Test features and triage feature bugs targeted for this releases
  • [ON TRACK] [softvision] Input: Triage Input.MO for prevalent issues and feature requests
One-time-only Tasks

Automation

BFTs

Triage the Mozmill dashboard for test failures and debug for potential Firefox regressions.

  • Lead: Vlad Maniac (Softvision)
  • Peer: Alex Lakatos (Softvision)
  • Failure Status: NEW (12) | ASSIGNED (7) | FIXED (3) | REGRESSIONS (0)
  • Tasks:
    • Check the Mozmill dashboard for failing tests for the day
    • Cross reference any failure to the list of NEW and ASSIGNED bugs
      • If the bug is NEW, assign it to yourself and try to fix it
      • If the bug is ASSIGNED and there has been no activity on the bug in over a week, offer your assistance to fix the bug
      • If there is NO bug on file, file it using this template
    • If you need assistance, ask ashughes in the #mozmill channel

Endurance

Performed during NIGHTLY phase only

Triage and report the performance numbers from the daily Endurance tests.

  • Lead: Vlad Maniac (Softvision)
  • Peer: Alex Lakatos (Softvision)
  • Endurance reports
  • Spreadsheet for tracking results
  • NOTE: Report any spikes in performance degradation to the QA release driver
Analysis (20110722 - 20110815)
  • Windows Memory Usage: +15.58% (77MB -> 89MB)
  • Mac Memory Usage: +3.27% (153MB -> 158MB)
  • Linux Memory Usage: +0.00% (84MB -> 84MB)
Issues Found

Bug Triage

Check UNCONFIRMED, NEW, FIXED, REOPEN, and tracking/status bugs

  • Lead: Vlad Maniac (Softvision)
  • Peer: Alex Lakatos (Softvision)
  • Bugs: QA+, FIXED
  • Results: VERIFIED, REOPENED
  • Tips:
    • If you don't know how to verify a bug, just skip it, move onto the next bug
    • Ignore code-level bugs, bugs in tests, build issues
Notes

Features

Track features which have landed in Firefox 8 and are ready to be tested (spreadsheet)

Feature

Description of feature

  • Leads: Person (role)
  • QA Owner: Owner
  • Status: Status
  • Links: Feature page, test plan, tracking bug, etc
  • Notes:

Electrolysis Texture Sharing

  • Leads: Chris Blizzard (Product), Cedric Vivier (Feature, Lead Engineer),
  • QA Owner: irc: VirgilDicu
  • Status: In progress. Health: OK - Feature in development stage
  • Links: Feature Page,Tracking Bug
  • Notes:

Tab move detach animations

When dragging tabs on the tab bar, we should show the tab and animate when you rearrange them, instead of showing a placeholder indicator like we currently do. We should also show a thumbnail of the page on detach operations, and make sure we have a good threshold, so this isn't triggered accidentally.

Third Party Add-on Warnings

We currently do not provide adequate warning to users that new third-party provided add-ons have been installed. This project will ensure that users opt-in to all add-ons that aren't installed through the Firefox UI. This feature falls primarily in the Experience category (from the "Discover, Experience, and Connect" vision statement.)

  • Leads: Asa Dotzler (Product), Dave Townsend (Feature, Lead), Curtis Koenig (Security lead), Jennifer Boriss (UX Lead)
  • QA Owner: Henrik Skupin
  • Status: Complete. Health: OK - Feature Landed.
  • Links: Feature Page, Tracking Bug
  • Notes:


People Search

Users can currently search for general web content, ecommerce items, and reference material, but they cannot search for people. We should add some form of People search to our search list. Twitter search is probably a good first step.

  • Leads: Chris Lee (Product)
  • QA Owner: Ioana Budnar
  • Status: In progress. Health: OK - Feature in development stage
  • Links: Feature PageTracking Bug
  • Notes: Tracking Bug is Verified Fixed. Not clear if it needs more QA work.

In-content UI Visual Unification

Do preparation work for having new in-content UI have a consistent and unified visual appearance.

  • Leads: Asa Dotzler (Product), Blair McBride (Feature, Lead Engineer), Jennifer Boriss (UX Lead)
  • QA Owner: Henrik Skupin
  • Status: In progress. Health: OK - Feature in development stage
  • Links: Feature Page
  • Notes:Initial work done & landed, in process of getting reviews and converting additional pages. The feature here is to create a unified template so that future pages (add-ons, permission, etc.) have a consistent look and feel. This will not include user facing changes.

Confirm add-ons on upgrade

Then next time existing users upgrade Firefox they should be presented with a list of add-ons they have installed where they can choose to disable them. By default add-ons from third-parties which may have been installed without the users' knowledge will be disabled. This replaces the compatibility wizard for that one time upgrade, subsequent upgrades use the compatibility wizard still.
This feature falls primarily in the Experience category (from the "Discover, Experience, and Connect" vision statement.)

  • Leads: Justin Scott (Product), Dave Townsend (Feature, Lead), Curtis Koenig (Security lead), Alex Faaborg (UX Lead)
  • QA Owner: Henrik Skupin
  • Status: Complete. Health: OK - Feature Landed.
  • Links: Feature Page, Tracking Bug
  • Notes:

Expose prefs for tabs-on-demand

Firefox 4 has the hidden ability to load tabs on-demand by setting the maximum number of concurrent tabs being restored to "0" — now that the feature has been in the wild for a while, we'd like to expose it as the default behavior for session restore. It makes startup a lot faster, and uses a lot less memory.
This feature falls primarily in the Experience category (from the "Discover, Experience, and Connect" vision statement.)

  • Leads: Asa Dotzler (Product), Alex Limi (Feature), Paul O'Shannessy (Lead Engineer)
  • QA Owner: Mihaela Velimiroviciu
  • Status: Complete. Health: OK - Feature Landed.
  • Links: Feature Page, Tracking Bug
  • Notes:

Input

The following are the details of daily triage conducted on http://input.mozilla.org.

Popular Feedback

Full Functional Tests

The following tracks testing done by iBeta for full functional test coverage. This task is performed once during Aurora only.

  • Lead: Juan Becerra
  • [ link to test suite]
  • [ link to results of testing]
  • [ link to bugs filed by iBeta]

Website Compatibility

The following tracks testing done by uTest for website compatibility. This task is performed after every merge.

  • Lead: Juan Becerra
  • Link to test suite
  • Link to report for Aurora Web Compatibility
  • Link to report for Beta Web Compatibility
  • Link to bugs filed

Aurora Merge

Verify that the builds generated from the merge to Aurora have updated properly.

Auroratest
  • [DONE] [ashughes] Windows XP: PASS
    • Source: Mozilla/5.0 (Windows NT 5.1; rv:7.0a2) Gecko/20110813 Firefox/7.0a2
    • Target: Mozilla/5.0 (Windows NT 5.1; rv:8.0a2) Gecko/20110817 Firefox/8.0a2
  • [DONE] [ashughes] Windows 7: PASS
    • Source: Mozilla/5.0 (Windows NT 6.1; rv:7.0a2) Gecko/20110814 Firefox/7.0a2
    • Target: Mozilla/5.0 (Windows NT 6.1; rv:8.0a2) Gecko/20110817 Firefox/8.0a2
  • [DONE] [ashughes] Mac OS X 10.6: PASS
    • Source: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0a2) Gecko/20110812 Firefox/7.0a2
    • Target: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:8.0a2) Gecko/20110817 Firefox/8.0a2
  • [DONE] [ashughes] Ubuntu 11.04: PASS
    • Source: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:7.0a2) Gecko/20110814 Firefox/7.0a2
    • Target: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:8.0a2) Gecko/20110817 Firefox/8.0a2
Aurora
  • [DONE] [ashughes] Windows XP: PASS
    • Source: Mozilla/5.0 (Windows NT 5.1; rv:7.0a2) Gecko/20110813 Firefox/7.0a2
    • Target: Mozilla/5.0 (Windows NT 5.1; rv:8.0a2) Gecko/20110819 Firefox/8.0a2
  • [DONE] [ashughes] Windows 7: PASS
    • Source: Mozilla/5.0 (Windows NT 6.1; rv:7.0a2) Gecko/20110814 Firefox/7.0a2
    • Target: Mozilla/5.0 (Windows NT 6.1; rv:8.0a2) Gecko/20110819 Firefox/8.0a2
  • [DONE] [ashughes] Mac OS X 10.6: PASS
    • Source: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0a2) Gecko/20110812 Firefox/7.0a2
    • Target: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:8.0a2) Gecko/20110819 Firefox/8.0a2
  • [DONE] [ashughes] Ubuntu 11.04: PASS
    • Source: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:7.0a2) Gecko/20110814 Firefox/7.0a2
    • Target: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:8.0a2) Gecko/20110819 Firefox/8.0a2
Issues Found

Beta 1

The following tasks need to be completed to sign-off the Beta merge

  • [ON TRACK] [ashughes] Automated Tests: details
  • [ON TRACK] [softvision] Manual Tests: details
  • [ON TRACK] [ashughes] Updates: details

Builds: Firefox 8.0b1(1)
Litmus:testrun

Automated Tests

The following are the detailed results of the automated BFT test runs.

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS
  • Reports: Litmus, Mozmill
Issues Found
  • Two tests are constantly failing and which need to be fixed as soon as possible.

Manual Tests

Smoketests

The following are the detailed results of the manual smoketests.

  • [DONE] [Simona] Windows 2000: PASS
  • [DONE] [Ioana] Windows XP: PASS
  • [DONE] [Simona] Windows Vista: PASS
  • [DONE] [Mihaela] Windows 7: PASS
  • [DONE] [Andrei] Windows 7 64-bit: PASS
  • [DONE] [VladG] Mac OS X 10.6: PASS
  • [DONE] [VladG] Mac OS X 10.7: PASS
  • [DONE] [Ioana] Ubuntu 11.04: PASS
  • [DONE] [Virgil] Ubuntu 11.04 64-bit: PASS
  • Litmus testrun, report
  • Firefox 8.0b1 build1
Exploratory Tests

The following are the detailed results of the exploratory testing - Web App compatibility

  • [DONE] [Simona] Windows 2000: PASS
  • [DONE] [Ioana] Windows XP: PASS
  • [DONE] [Simona] Windows Vista: PASS
  • [DONE] [Mihaela] Windows 7: PASS
  • [DONE] [Andrei] Windows 7 64-bit: PASS
  • [DONE] [VladG] Mac OS X 10.6: PASS
  • [DONE] [VladG] Mac OS X 10.7: PASS
  • [DONE] [Ioana] Ubuntu 11.04: PASS
  • [DONE] [Virgil] Ubuntu 11.04 64-bit PASS
  • Litmus report
Issues Found
  • Html5
    • bug 690356 - Lower half of html5 youtube videos flickers in full screen
  • Tab animation
    • bug 689554 - The use of closing browser window shortcut key (ctrl+shift+w) while dnd a tab disturbs tab animation [NEW]
    • bug 689566 - Keyboard shortcut cmd+shift+w disturbs tab animation - dnd a tab from a window to another window [mac] [NEW]
    • bug 689575 - Disable keyboard shortcuts while tab animation is active [NEW]
    • bug 690341 - Using the close tab option from the menu opens a new tab when dragging a tab [NEW]

Updates

The following are details of update verifications for Firefox 8 beta 1

Releasetest: report

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS

Beta: report

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS
Issues Found

Beta 2 (build1)

The following tasks need to be completed to sign-off the Beta merge

  • [ON TRACK] [ashughes] Automated Tests: details
  • [ON TRACK] [softvision] Manual Tests: details
  • [ON TRACK] [softvision] Bug fix verifications details
  • [ON TRACK] [ashughes] Updates: details

Builds: Firefox 8.0b2(1)
Litmus:testrun

Automated Tests

The following are the detailed results of the automated BFT test runs.

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS
  • Reports: Litmus, Mozmill
Issues Found
  • /restartTests/testAddons_installMultipleExtensions/test1.js is failing on Win XP

Manual Tests

Smoketests

The following are the detailed results of the manual smoketests.

  • [DONE] [Simona] Windows 2000: PASS
  • [DONE] [Ioana] Windows XP: PASS
  • [DONE] [Simona] Windows Vista: PASS
  • [DONE] [Mihaela] Windows 7: PASS
  • [DONE] [Mihaela] Windows 7 64-bit: PASS
  • [DONE] [VladG] Mac OS X 10.6: PASS
  • [DONE] [VladG] Mac OS X 10.7: PASS
  • [DONE] [Ioana] Ubuntu 11.04: PASS
  • [DONE] [Virgil] Ubuntu 11.04 64-bit: PASS
  • Litmus testrun, report
  • Firefox 8.0b2 build1
Exploratory Tests

The following are the detailed results of the exploratory testing - Web App compatibility

  • [DONE] [Simona] Windows 2000: PASS
  • [DONE] [Ioana] Windows XP: PASS
  • [DONE] [Simona] Windows Vista: PASS
  • [DONE] [Mihaela] Windows 7: PASS
  • [DONE] [Mihaela] Windows 7 64-bit: PASS
  • [DONE] [VladG] Mac OS X 10.6: PASS
  • [DONE] [VladG] Mac OS X 10.7: PASS
  • [DONE] [Ioana] Ubuntu 11.04: PASS
  • [DONE] [Virgil] Ubuntu 11.04 64-bit: PASS
  • Litmus report
Issues Found
  • bug 692418 -Print error is triggered the first time when attempting to print a yahoo mail message (New)
  • bug 692446 -Firefox prompts to install windows media player plugin even though it is already installed (New)

Bug Fix Verification

Verify 100% QA+ bugs, as many as possible FIXED bugs

Updates

The following are details of update verifications for Firefox 8 beta 2

Betatest:

  • run some manual exploratory tests around updating with add-ons, plugins, session/profile data, etc
  • How to set update channel to betatest:
    • Find the defaults/prefs/channel-prefs.js file in your Firefox application folder
    • Open it with a text editor and change the "beta" channel to "betatest", save and exit
    • Start Firefox and check app.update.channel in about:config to verify it changed
    • NOTE: On Windows Vista and 7 will not let you save the changes unless you run your text editor as an Administrator -- do so by right clicking the launcher for the text editor and select "Run as Administrator"
  • What to test
    • Build a realistic amount of profile and session data on one of the previous betas (4.0b8 through 8.0b1):
    • try installing several add-ons (some out of date)
    • try installing several plugins (some out of date)
    • go to various top sites like Youtube, Banking sites, Social networks, news sites, games, webmail etc
    • put some of the tabs in App Tabs and Panorama groupings
    • Ultimately just play around with your browser for an hour or so like you would on a day to day basis
    • After a while, check for and install the Firefox update to 8.0b2 and restart
  • Result -- check after restart:
    • no loss of session data
    • no loss of profile data
    • no loss of history, bookmarks
    • no loss of add-ons
    • no obvious breakage with Firefox and the sites you use most often
  • What to report
    • File bugs using the [8b2] whiteboard tag


Releasetest: report

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS

Beta: [ report]

  • [ON TRACK] [tester] Windows 2000: result
  • [ON TRACK] [tester] Windows XP: result
  • [ON TRACK] [tester] Windows Vista: result
  • [ON TRACK] [tester] Windows 7: result
  • [ON TRACK] [tester] Windows 7 64-bit: result
  • [ON TRACK] [tester] Mac OS X 10.6: result
  • [ON TRACK] [tester] Mac OS X 10.7: result
  • [ON TRACK] [tester] Ubuntu 11.04: result
  • [ON TRACK] [tester] Ubuntu 11.04 64-bit: result
Issues Found

Beta 2 (build 2)

The following tasks need to be completed to sign-off on this beta milestone.

  • [ON TRACK] [ashughes] Automated Tests: details
  • [ON TRACK] [softvision] Manual Tests: details
  • [ON TRACK] [softvision] Bug fix verifications details
  • [ON TRACK] [ashughes] Updates: details

Builds: Firefox 8.0b2(build 2)
Litmus:testrun

Automated Tests

The following are the detailed results of the automated BFT test runs.

  • [DONE] [ashughes] Windows 2000: RESULT
  • [DONE] [ashughes] Windows XP: RESULT
  • [DONE] [ashughes] Windows Vista: RESULT
  • [DONE] [ashughes] Windows 7: RESULT
  • [DONE] [ashughes] Windows 7 64-bit: RESULT
  • [DONE] [ashughes] Mac OS X 10.6: RESULT
  • [DONE] [ashughes] Mac OS X 10.7: RESULT
  • [DONE] [ashughes] Ubuntu 11.04: RESULT
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: RESULT
  • Reports: Litmus, Mozmill
Issues Found
  • /testSecurity/testSubmitUnencryptedInfoWarning.js failed on Mac 10.7 and Windows 2000
  • /restartTests/testAddons_installMultipleExtensions failed on Mac 10.7
  • /testTabbedBrowsing/testBackgroundTabScrolling.js failed on Windows Vista

Manual Tests

Smoketests

The following are the detailed results of the manual smoketests.

  • [DONE] [Simona] Windows 2000: RESULT
  • [DONE] [Ioana] Windows XP: RESULT
  • [DONE] [Simona] Windows Vista: RESULT
  • [DONE] [Mihaela] Windows 7: RESULT
  • [DONE] [Mihaela] Windows 7 64-bit: RESULT
  • [DONE] [VladG] Mac OS X 10.6: RESULT
  • [DONE] [VladG] Mac OS X 10.7: RESULT
  • [DONE] [Ioana] Ubuntu 11.04: RESULT
  • [DONE] [Virgil] Ubuntu 11.04 64-bit: RESULT
  • Litmus testrun, report
  • Firefox 8.0b2 build2
Exploratory Tests

The following are the detailed results of the exploratory testing - Web App compatibility

  • [DONE] [Simona] Windows 2000: RESULT
  • [DONE] [Ioana] Windows XP: RESULT
  • [DONE] [Simona] Windows Vista: RESULT
  • [DONE] [Mihaela] Windows 7: RESULT
  • [DONE] [Mihaela] Windows 7 64-bit: RESULT
  • [DONE] [VladG] Mac OS X 10.6: RESULT
  • [DONE] [VladG] Mac OS X 10.7: RESULT
  • [DONE] [Ioana] Ubuntu 11.04: RESULT
  • [DONE] [Virgil] Ubuntu 11.04 64-bit: RESULT
  • Litmus report
Issues Found
  • bug 692418 -Print error is triggered the first time when attempting to print a yahoo mail message (New)
  • bug 692446 -Firefox prompts to install windows media player plugin even though it is already installed (New)

Bug Fix Verification

Verify 100% QA+ bugs, as many as possible FIXED bugs

Updates

The following are details of update verifications for Firefox 8 beta 2

Betatest:

  • run some manual exploratory tests around updating with add-ons, plugins, session/profile data, etc
  • How to set update channel to betatest:
    • Find the defaults/prefs/channel-prefs.js file in your Firefox application folder
    • Open it with a text editor and change the "beta" channel to "betatest", save and exit
    • Start Firefox and check app.update.channel in about:config to verify it changed
    • NOTE: On Windows Vista and 7 will not let you save the changes unless you run your text editor as an Administrator -- do so by right clicking the launcher for the text editor and select "Run as Administrator"
  • What to test
    • Build a realistic amount of profile and session data on one of the previous betas (4.0b8 through 8.0b1):
    • try installing several add-ons (some out of date)
    • try installing several plugins (some out of date)
    • go to various top sites like Youtube, Banking sites, Social networks, news sites, games, webmail etc
    • put some of the tabs in App Tabs and Panorama groupings
    • Ultimately just play around with your browser for an hour or so like you would on a day to day basis
    • After a while, check for and install the Firefox update to 8.0b2 and restart
  • Result -- check after restart:
    • no loss of session data
    • no loss of profile data
    • no loss of history, bookmarks
    • no loss of add-ons
    • no obvious breakage with Firefox and the sites you use most often
  • What to report
    • File bugs using the [8b2] whiteboard tag

Exploratory testing

  • [DONE] [Simona] Windows XP:PASS
  • [DONE] [Virgil] Ubuntu 11.04 x64:PASS
  • [DONE] [Ioana] Windows 7:PASS
  • [DONE] [VladG] MacOS X 10.6:PASS
    • Exploratory work spreadsheet Link

Releasetest: report

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS

Beta: report

  • [ON TRACK] [tester] Windows 2000: PASS
  • [ON TRACK] [tester] Windows XP: PASS
  • [ON TRACK] [tester] Windows Vista: PASS
  • [ON TRACK] [tester] Windows 7: PASS
  • [ON TRACK] [tester] Windows 7 64-bit: PASS
  • [ON TRACK] [tester] Mac OS X 10.6: PASS
  • [ON TRACK] [tester] Mac OS X 10.7: PASS
  • [ON TRACK] [tester] Ubuntu 11.04: PASS
  • [ON TRACK] [tester] Ubuntu 11.04 64-bit: PASS
Issues Found
  • There were three failures in the automation, which I wasn't able to reproduce manually (juanb).

Beta 3

The following tasks need to be completed to sign-off the Beta merge

  • [ON TRACK] [ashughes] Automated Tests: details
  • [ON TRACK] [softvision] Manual Tests: details
  • [ON TRACK] [softvision] Bug fix verifications details
  • [ON TRACK] [ashughes] Updates: details

Builds: Firefox 8.0b3(build1)
Litmus:testrun

Automated Tests

The following are the detailed results of the automated BFT test runs.

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS
  • Reports: Litmus, Mozmill
Issues Found

Manual Tests

Smoketests

The following are the detailed results of the manual smoketests.

  • [DONE] [Simona] Windows 2000: PASS
  • [DONE] [AndreiD] Windows XP: PASS
  • [DONE] [Simona] Windows Vista: PASS
  • [DONE] [Mihaela] Windows 7: PASS
  • [DONE] [AndreiD] Windows 7 64-bit: PASS
  • [DONE] [VladG] Mac OS X 10.6: PASS
  • [DONE] [VladG] Mac OS X 10.7: PASS
  • [DONE] [Ioana] Ubuntu 11.04: PASS
  • [DONE] [Mihaela] Ubuntu 11.04 64-bit: PASS
  • Litmus testrun, report
  • Firefox 8.0b3 build1
Exploratory Tests

The following are the detailed results of the exploratory testing - Web App compatibility

  • [DONE] [Simona] Windows 2000: PASS
  • [DONE] [AndreiD] Windows XP: PASS
  • [DONE] [Simona] Windows Vista: PASS
  • [DONE] [Mihaela] Windows 7: PASS
  • [DONE] [AndreiD] Windows 7 64-bit: PASS
  • [DONE] [VladG] Mac OS X 10.6: PASS
  • [DONE] [VladG] Mac OS X 10.7: PASS
  • [DONE] [Ioana] Ubuntu 11.04: PASS
  • [DONE] [Mihaela] Ubuntu 11.04 64-bit: PASS
  • Litmus report
Issues Found
  • bug 694288 - Vimeo.com - embedded video player is not displayed on Windows 2000 (New)

Bug Fix Verification

Verify 100% QA+ bugs, as many as possible FIXED bugs

Updates

The following are details of update verifications for Firefox 8 beta 3

Betatest:

  • run some manual exploratory tests around updating with add-ons, plugins, session/profile data, etc
  • How to set update channel to betatest:
    • Find the defaults/prefs/channel-prefs.js file in your Firefox application folder
    • Open it with a text editor and change the "beta" channel to "betatest", save and exit
    • Start Firefox and check app.update.channel in about:config to verify it changed
    • NOTE: On Windows Vista and 7 will not let you save the changes unless you run your text editor as an Administrator -- do so by right clicking the launcher for the text editor and select "Run as Administrator"
  • What to test
    • Build a realistic amount of profile and session data on one of the previous betas (4.0b8 through 8.0b1):
    • try installing several add-ons (some out of date)
    • try installing several plugins (some out of date)
    • go to various top sites like Youtube, Banking sites, Social networks, news sites, games, webmail etc
    • put some of the tabs in App Tabs and Panorama groupings
    • Ultimately just play around with your browser for an hour or so like you would on a day to day basis
    • After a while, check for and install the Firefox update to 8.0b2 and restart
  • Result -- check after restart:
    • no loss of session data
    • no loss of profile data
    • no loss of history, bookmarks
    • no loss of add-ons
    • no obvious breakage with Firefox and the sites you use most often
  • What to report
    • File bugs using the [8b3] whiteboard tag

Exploratory testing

  • [DONE] [Simona] Windows XP:PASS
  • [DONE] [Ioana] Ubuntu 11.04 x86:PASS - with known issues: bug 658913
  • [DONE] [Mihaela] Windows 7:PASS
  • [DONE] [VladG] MacOS X 10.6:PASS
    • Exploratory work spreadsheet Link


Releasetest: report

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS

Beta: report

  • [ON TRACK] [tester] Windows 2000: result
  • [ON TRACK] [tester] Windows XP: result
  • [ON TRACK] [tester] Windows Vista: result
  • [ON TRACK] [tester] Windows 7: result
  • [ON TRACK] [tester] Windows 7 64-bit: result
  • [ON TRACK] [tester] Mac OS X 10.6: result
  • [ON TRACK] [tester] Mac OS X 10.7: result
  • [ON TRACK] [tester] Ubuntu 11.04: result
  • [ON TRACK] [tester] Ubuntu 11.04 64-bit: result
Issues Found
Retrieved from "https://wiki.mozilla.org/index.php?title=Releases/Firefox_8/Test_Plan&oldid=357963"