Releases/Firefox 30/Test Plan

« Firefox 29 | Firefox 30 | Firefox 31 »

Firefox 30 Test Plan

QA Lead: Tracy Walker

Schedule

• 2014.02.03: [ON TRACK] Nightly
• 2014.03.17: Aurora
• 2014.04.25: Pre-Beta feature sign-offs
• 2014.04.29: Beta 1
• 2014.05.06: Beta 2
• 2014.05.08: Beta 3
• 2014.05.13: Beta 4
• 2014.05.15: Beta 5
• 2014.05.20: Beta 6
• 2014.05.22: Beta 7
• 2014.05.27: Beta 8
• 2014.05.29: Beta 9
• 2014.06.03: Beta 10
• 2014.06.05: Release Candidate
• 2014.06.10: Final

Nightly

Features

Feature Tracking

Tentative - we'll be switching to 2 week sprint model for a portion of desktop Fx work and will link to those sprint tracks here

Sign-off Criteria

• All user stories covered by manual or automated tests
• Manual testing complete with no blocking regressions
• Automated testing complete with no blocking regressions
• 0 untriaged unconfirmed bugs
• 0 untriaged unverified fixes

Status

Feature

Owner

Documentation

Tracked Bugs

Aurora Merge Sign-off

Pre-beta Sign-off

Notes.

Fixed Bugs

How to Help

• Download and install the latest Firefox Nightly
• Load this list of bug reports
• Read each bug report until you find one you understand and want to test
• Download and install the version of Firefox used to report the bug
• Using the older build, test the claim made in the bug report to see if you can reproduce it
  • If you cannot reproduce the bug move on to another bug or ask the reporter to verify the fix
• Using the latest Nightly, test to see if you can reproduce the bug
  • If you cannot reproduce the bug set the STATUS to VERIFIED FIXED and add a comment describing your testing
  • If you can still reproduce the bug set the STATUS to REOPENED and add a comment describing your testing
• Add any bug reopened to the list below using the bug number template (verified fixes will be tracked elsewhere)
• If you need help, send email to Tracy Walker with the bug summary in the Subject field

Reopened Bugs

Reported Bugs

Triage unconfirmed bugs.

How to Help

• Download and install the latest Firefox Nightly
• Load this list of bug reports
• Read each bug report until you find one you understand and want to test
• Test the claim made in the bug report and see if you can reproduce it
• If you can reproduce the bug, set the STATUS to NEW, add a comment describing your testing, and try to find a regression window
• Any confirmed bugs should be added to the list below using the bug number template
• If you need help, send email to Tracy Walker with the bug summary in the Subject field

Confirmed Bugs

Stability

How to Help

• Review the Firefox 30 Explosiveness Report
  • Any signatures rising significantly (>= 2.0 explosiveness rating) without a bug number should be reported to Bugzilla and nominated for tracking
  • If there is already a bug report, update the report with the latest numbers
  • Ensure the bug report has the topcrash-platform keyword added (where platform is the platform where crashes occur -- ie. topcrash-win for Windows)
  • Ensure the bug is listed below using the bug number template
• Review the Firefox 30 Topcrash Report
  • Any signature in the top 10 without a bug number should be reported to Bugzilla and nominated for tracking
  • If there is already a bug report, update the report with the latest ranking information
  • Ensure the bug report has the topcrash-platform keyword added (where platform is the platform where crashes occur -- ie. topcrash-win for Windows)
  • Ensure the bug is listed below using the bug number template
• Review the Firefox 30 tracked bugs that aren't tracked in 29 or 28.
  • Each bug listed needs to have its rank updated once per week
  • Use the topcrash reports from crash-stats to get the rank for each branch
  • Add this information to the bug report
  • Ensure the bug report has the topcrash-platform keyword added (where platform is the platform where crashes occur -- ie. topcrash-win for Windows)
  • Conversely, remove the topcrash keyword if the crash is no longer in the top-10
• For each of these tasks, ensure the etherpad is updated with your name and the date checked

Topcrash Bugs

QA Wanted

How to Help

• Download and install the latest Firefox Nightly
• Review this list of bug reports to find a bug you both understand and want to test (each should contain a comment detailing what's needed from QA)
  • If the bug needs steps to reproduce, try to find a reproducible set of steps
  • If the bug needs a regression window, try to find a regression window
  • If the bug is fixed and needs verification, test to confirm you can reproduce the bug on an older build but not the latest build
• Any tested bugs should be added to the list below using the bug number template
• Remove the qawanted keyword from any fulfilled request
• If you need help, send email to Tracy Walker with the bug summary in the Subject field

Tested Bugs

Aurora

Branch Migration

Pre Sign-off Tasks

• {{|}} Firefox branch migrated (Release Management)
• {{|}} Mozmill Tests branch migrated (Automation Development)
• {{|}} Updates throttled on aurora channel (Release Engineering)
• {{|}} Updates verified throttled on aurora channel (QA)

Sign-off Tasks

• {{|}} Mozmill Daily functional testruns pass with no failures ([report])
• {{|}} Mozmill Daily auroratest update testruns pass with no failures ([report])
• {{|}} Scope features tested and signed off for Aurora uplift
  • feature...

Post Sign-off Tasks

• Aurora sign-off email sent (QA)
• Updates unthrottled on aurora channel (Release Engineering)
• Updates verified unthrottled on aurora channel (QA)