QA/Templates/Firefox Desktop Testplan

From MozillaWiki
< QA‎ | Templates
Jump to: navigation, search
TEST PLAN SHOULD BE CREATED AT wiki.mozilla.org/Releases/Firefox_VERSION/Test_Plan

« Firefox PREVIOUS | Firefox CURRENT | Firefox NEXT »

Firefox VERSION Test Plan

Summary

Milestone [LINK_TO_FLIGHT_TRACKING_PAGE Firefox VERSION]
Status Currently in BRANCH -- merging to NEXT_BRANCH on DATE_OF_MERGE
Lead [MAILTO_LINK_FOR_LEAD] (irc: NICK)
Peer [MAILTO_LINK_FOR_PEER] (irc: NICK)

Latest Build: [ ftp], [ http], [ web]

The following is the test plan for Firefox VERSION from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox VERSION. After the release this document will be used as an archive of what was done to validate this release.

If you have some free time, please pick a task below and get in touch with one of the leads.

Details

The following is a description of the tasks which need to be completed throughout this release.

Daily Tasks
  • {{|}} [tester] Automation: Check dashboards for new failures
  • {{|}} [tester] Bug Triage: Check Bugzilla for regressions
  • {{|}} [tester] Features: Test features and triage feature bugs targeted for this releases
  • {{|}} [tester] Input: Triage Input.MO for prevalent issues and feature requests
One-time-only Tasks
  • {{|}} [tester] Nightly Merge: Sign-off initial merge to Nightly
  • {{|}} [tester] Aurora Merge: Sign-off merge from Nightly to Aurora
  • {{|}} [tester] Beta Merge: Sign-off merge from Aurora to Beta
  • {{|}} [tester] Release: Sign-off final release

Automation

BFTs

Triage the [ Mozmill dashboard] for test failures and debug for potential Firefox regressions.

  • Lead: Vlad Maniac (Softvision)
  • Peer: Alex Lakatos (Softvision)
  • Failure Status: NEW (0) | ASSIGNED (0) | FIXED (0) | REGRESSIONS (0)
  • Tasks:
    • Check the Mozmill dashboard for failing tests for the day
    • Cross reference any failure to the list of NEW and ASSIGNED bugs
      • If the bug is NEW, assign it to yourself and try to fix it
      • If the bug is ASSIGNED and there has been no activity on the bug in over a week, offer your assistance to fix the bug
      • If there is NO bug on file, file it using this template
    • If you need assistance, ask ashughes in the #mozmill channel

Endurance

Triage and report the performance numbers from the daily Endurance tests.

  • Endurance reports
  • Spreadsheet for tracking results
  • NOTE: Report any spikes in performance degradation to the QA release driver
Issues Found

Bug Triage

Check UNCONFIRMED, NEW, FIXED, REOPEN, and tracking/status bugs

Important Bugs
  • [STATUS] bug - Bug summary
Notes

Features

Track features which have landed in Firefox VERSION and are ready to be tested.

FEATURE

SUMMARY OF THE FEATURE, GOALS, ETC

  • Leads: NAME (ROLE)
  • QA Owner: NAME
  • Status: STATUS
  • Links: TEST PLAN, TRACKING BUG, ETC
  • Notes:

Input

The following are the details of daily triage conducted on http://input.mozilla.org.

Tasks
  • Go to http://input.mozilla.org
  • Filter the results by Firefox version
  • Any feedback which is spiking should listed below and escalated to the QA release driver
Popular Feedback

Release

The following tasks need to be completed to sign-off the Beta merge.

  • {{|}} [tester] Automated tests: details
  • {{|}} [tester] Manual tests: details
  • {{|}} [tester] Feature spotchecks: details
  • {{|}} [tester] Bug fix verifications: details
  • {{|}} [tester] Updates: details

Automated Tests

The following are the detailed results of the automated BFT test runs.

  • {{|}} Windows 2000: [ functional] | [ restart]
  • {{|}} Windows XP: [ functional] | [ restart]
  • {{|}} Windows Vista: [ functional] | [ restart]
  • {{|}} Windows 7: [ functional] | [ restart]
  • {{|}} Windows 7 64-bit: [ functional] | [ restart]
  • {{|}} Mac OS X 10.6: [ functional] | [ restart]
  • {{|}} Ubuntu 11.04: [ functional] | [ restart]
  • {{|}} Ubuntu 11.04 64-bit: [ functional] | [ restart]
  • Reports: [ Litmus], [ Mozmill]
Issues Found

Manual Tests

The following are the detailed results of the manual smoketests.

  • {{|}} [tester] Windows 2000: status
  • {{|}} [tester] Windows XP: status
  • {{|}} [tester] Windows Vista: status
  • {{|}} [tester] Windows 7: status
  • {{|}} [tester] Windows 7 64-bit: status
  • {{|}} [tester] Mac OS X 10.6: status
  • {{|}} [tester] Ubuntu 11.04: status
  • {{|}} [tester] Ubuntu 11.04 32-bit: status
  • Litmus [ testrun], [ report]
  • Firefox VERSION [ builds]
Issues Found
Exploratory Testing

Conduct exploratory tests on FOCUS AREA

  • {{|}} [tester] Windows XP: status
  • {{|}} [tester] Windows 7: status
  • {{|}} [tester] Mac OS X 10.6: status
  • {{|}} [tester] Ubuntu 11.04: status
Issues Found

Features

The following are the details about spotchecks performed on landed features

  • {{|}} [tester] FEATURE: status
Issues Found

Bug Verifications

The following are the details of bug verifications for Firefox VERSION

  • Bugs: [ tracking-firefoxN+]
  • Results: [ VERIFIED], [ REOPENED]
Issues Found

Updates

The following are details of update verifications for Firefox VERSION

Betatest: report

  • {{|}} Windows XP: status
  • {{|}} Windows Vista: status
  • {{|}} Windows 7: status
  • {{|}} Windows 7 64-bit: status
  • {{|}} Mac OS X 10.6: status
  • {{|}} Mac OS X 10.7: status
  • {{|}} Ubuntu 11.04: status
  • {{|}} Ubuntu 11.04 64-bit: status

Beta: report

  • {{|}} Windows XP: status
  • {{|}} Windows Vista: status
  • {{|}} Windows 7: status
  • {{|}} Windows 7 64-bit: status
  • {{|}} Mac OS X 10.6: status
  • {{|}} Mac OS X 10.7: status
  • {{|}} Ubuntu 11.04: status
  • {{|}} Ubuntu 11.04 64-bit: status

Releasetest: report

  • {{|}} Windows XP: status
  • {{|}} Windows Vista: status
  • {{|}} Windows 7: status
  • {{|}} Windows 7 64-bit: status
  • {{|}} Mac OS X 10.6: status
  • {{|}} Mac OS X 10.7: status
  • {{|}} Ubuntu 11.04: status
  • {{|}} Ubuntu 11.04 64-bit: status

Release: report

  • {{|}} Windows XP: status
  • {{|}} Windows Vista: status
  • {{|}} Windows 7: status
  • {{|}} Windows 7 64-bit: status
  • {{|}} Mac OS X 10.6: status
  • {{|}} Mac OS X 10.7: status
  • {{|}} Ubuntu 11.04: status
  • {{|}} Ubuntu 11.04 64-bit: status
Issues Found

Beta Merge

The following tasks need to be completed to sign-off the Beta merge.

  • {{|}} [tester] Automated Tests: details
  • {{|}} [tester] Manual Tests: details
  • {{|}} [tester] Features: details
  • {{|}} [tester] Bug fix verifications: details
  • {{|}} [tester] Updates: details

Automated Tests

The following are the detailed results of the automated BFT test runs.

  • {{|}} Windows 2000: [ functional] | [ restart]
  • {{|}} Windows XP: [ functional] | [ restart]
  • {{|}} Windows Vista: [ functional] | [ restart]
  • {{|}} Windows 7: [ functional] | [ restart]
  • {{|}} Windows 7 64-bit: [ functional] | [ restart]
  • {{|}} Mac OS X 10.6: [ functional] | [ restart]
  • {{|}} Ubuntu 11.04: [ functional] | [ restart]
  • {{|}} Ubuntu 11.04 64-bit: [ functional] | [ restart]
  • Reports: [ Litmus], [ Mozmill]
Issues Found

Manual Tests

The following are the detailed results of the manual smoketests.

  • {{|}} [tester] Windows 2000: status
  • {{|}} [tester] Windows XP: status
  • {{|}} [tester] Windows Vista: status
  • {{|}} [tester] Windows 7: status
  • {{|}} [tester] Windows 7 64-bit: status
  • {{|}} [tester] Mac OS X 10.6: status
  • {{|}} [tester] Ubuntu 11.04: status
  • {{|}} [tester] Ubuntu 11.04 32-bit: status
  • Litmus [ testrun], [ report]
  • Firefox VERSION [ builds]
Issues Found
Exploratory Testing

Conduct exploratory tests on FOCUS AREA

  • {{|}} [tester] Windows XP: status
  • {{|}} [tester] Windows 7: status
  • {{|}} [tester] Mac OS X 10.6: status
  • {{|}} [tester] Ubuntu 11.04: status
Issues Found

Features

The following are the details about spotchecks performed on landed features

  • {{|}} [tester] FEATURE: status
Issues Found

Bug Verifications

The following are the details of bug verifications for Firefox VERSION

  • Bugs: [ tracking-firefoxN+]
  • Results: [ VERIFIED], [ REOPENED]
Issues Found

Updates

The following are details of update verifications for Firefox VERSION merge to Beta

Betatest
  • {{|}} [tester] Windows XP: status
    • Source:
    • Target:
  • {{|}} [tester] Windows 7: status
    • Source:
    • Target:
  • {{|}} [tester] Mac OS X 10.6: status
    • Source:
    • Target:
  • {{|}} [tester] Ubuntu 11.04: status
    • Source:
    • Target:
Beta
  • {{|}} [tester] Windows XP: status
    • Source:
    • Target:
  • {{|}} [tester] Windows 7: status
    • Source:
    • Target:
  • {{|}} [tester] Mac OS X 10.6: status
    • Source:
    • Target:
  • {{|}} [tester] Ubuntu 11.04: status
    • Source:
    • Target:
Issues Found

Aurora Merge

The following tasks need to be completed to sign-off the Aurora merge.

  • {{|}} [tester] Automated Tests: details
  • {{|}} [tester] Manual Tests: details
  • {{|}} [tester] Features: details
  • {{|}} [tester] Bug fix verifications: details
  • {{|}} [tester] Updates: details

Automated Tests

The following are the detailed results of the automated BFT test runs.

  • {{|}} Windows 2000: [ functional] | [ restart]
  • {{|}} Windows XP: [ functional] | [ restart]
  • {{|}} Windows Vista: [ functional] | [ restart]
  • {{|}} Windows 7: [ functional] | [ restart]
  • {{|}} Windows 7 64-bit: [ functional] | [ restart]
  • {{|}} Mac OS X 10.6: [ functional] | [ restart]
  • {{|}} Ubuntu 11.04: [ functional] | [ restart]
  • {{|}} Ubuntu 11.04 64-bit: [ functional] | [ restart]
Issues Found

Manual Tests

The following are the detailed results of the manual smoketests.

  • {{|}} [tester] Windows 2000: result
  • {{|}} [tester] Windows XP: result
  • {{|}} [tester] Windows Vista: result
  • {{|}} [tester] Windows 7: result
  • {{|}} [tester] Windows 7 64-bit: result
  • {{|}} [tester] Mac OS X 10.6: result
  • {{|}} [tester] Ubuntu 11.04: result
  • {{|}} [tester] Ubuntu 11.04 32-bit: result
  • Litmus [ testrun], [ report]
  • Firefox VERSION [ builds]
Issues Found

Features

The following are the details about spotchecks performed on landed features

  • {{|}} [tester] FEATURE: status
Issues Found

Bug Verifications

The following are the details of bug verifications for Firefox VERSION

  • Bugs: [ tracking-firefoxN+]
  • Results: [ VERIFIED], [ REOPENED]
Issues Found

Updates

The following are details of update verifications for Firefox VERSION merge to Aurora

Auroratest
  • {{|}} [tester] Windows XP: result
    • Source:
    • Target:
  • {{|}} [tester] Windows 7: result
    • Source:
    • Target:
  • {{|}} [tester] Mac OS X 10.6: result
    • Source:
    • Target:
  • {{|}} [tester] Ubuntu 11.04: result
    • Source:
    • Target:
Aurora
  • {{|}} [tester] Windows XP: result
    • Source:
    • Target:
  • {{|}} [tester] Windows 7: result
    • Source:
    • Target:
  • {{|}} [tester] Mac OS X 10.6: result
    • Source:
    • Target:
  • {{|}} [tester] Ubuntu 11.04: result
    • Source:
    • Target:
Issues Found

Nightly Merge

The following tasks need to be completed to sign-off the Nightly merge.

  • {{|}} [tester] Automated Tests: details
  • {{|}} [tester] Manual Tests: details
  • {{|}} [tester] Features: details
  • {{|}} [tester] Bug fix verifications: details
  • {{|}} [tester] Updates: details

Automated Tests

The following are the detailed results of the automated BFT test runs.

  • {{|}} Windows 2000: [ functional] | [ restart]
  • {{|}} Windows XP: [ functional] | [ restart]
  • {{|}} Windows Vista: [ functional] | [ restart]
  • {{|}} Windows 7: [ functional] | [ restart]
  • {{|}} Windows 7 64-bit: [ functional] | [ restart]
  • {{|}} Mac OS X 10.6: [ functional] | [ restart]
  • {{|}} Ubuntu 11.04: [ functional] | [ restart]
  • {{|}} Ubuntu 11.04 64-bit: [ functional] | [ restart]
Issues Found

Manual Tests

The following are the detailed results of the manual BFTs.

  • {{|}} [tester] Windows 2000: result
  • {{|}} [tester] Windows XP: result
  • {{|}} [tester] Windows Vista: result
  • {{|}} [tester] Windows 7: result
  • {{|}} [tester] Windows 7 64-bit: result
  • {{|}} [tester] Mac OS X 10.6: result
  • {{|}} [tester] Ubuntu 11.04: result
  • {{|}} [tester] Ubuntu 11.04 32-bit: result
  • Litmus [ testrun], [ report]
  • Firefox VERSION [ builds]
Issues Found

Features

The following are the details about spotchecks performed on landed features

  • {{|}} [tester] FEATURE: status
Issues Found

Bug Verifications

The following are the details of bug verifications for Firefox VERSION

  • Bugs: [ tracking-firefoxN+]
  • Results: [ VERIFIED], [ REOPENED]
Issues Found

Updates

The following are details of update verifications for Firefox VERSION merge to Nightly

Nightly
  • {{|}} [tester] Windows XP: result
    • Source:
    • Target:
  • {{|}} [tester] Windows 7: result
    • Source:
    • Target:
  • {{|}} [tester] Mac OS X 10.6: result
    • Source:
    • Target:
  • {{|}} [tester] Ubuntu 11.04: result
    • Source:
    • Target:
Issues Found
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Templates/Firefox_Desktop_Testplan&oldid=338501"