Releases/Firefox 27/Test Plan: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
Line 91: Line 91:
  | Chrome Stable on Windows 7
  | Chrome Stable on Windows 7
  | Firefox 27.0a1 on Windows 8
  | Firefox 27.0a1 on Windows 8
  |  
  | '''PASS'''
  |-
  |-
  | Firefox 27.0a1 on Windows 7
  | Firefox 27.0a1 on Windows 7

Revision as of 12:15, 20 September 2013

« Firefox 26 | Firefox 27 | Firefox 28 »

Firefox 27 Test Plan

Summary

Milestone Firefox 27
Status Currently in Nightly -- merging to Aurora on 2013-10-28
Lead Juan Becerra (irc: juanb)
2nd Anthony Hughes (irc: ashughes)
Builds [ftp] [http] [web]

The following is the test plan for Firefox 27 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 27. After the release this document will be used as an archive of what was done to validate this release.

If you have some free time, please pick a task below and get in touch with one of the leads.

Schedule

  • 2013.08.05: Nightly
  • 2013.09.16: Aurora
  • 2013.10.25: Pre-Beta feature sign-offs
  • 2013.10.31: Beta 1
  • 2013.11.05: Beta 2
  • 2013.11.07: Beta 3
  • 2013.11.12: Beta 4
  • 2013.11.14: Beta 5
  • 2013.11.19: Beta 6
  • 2013.11.21: Beta 7
  • 2013.11.26: Beta 8
  • 2013.11.28: Beta 9
  • 2013.12.03: Beta 10
  • 2013.12.09: Release Candidate
  • 2013.12.10: Final

Features

Sign-off Criteria
  • All user stories covered by manual or automated tests
  • Manual testing complete with no blocking regressions
  • Automated testing complete with no blocking regressions
  • 0 untriaged unconfirmed bugs
  • 0 untriaged unverified fixes
Status
Feature Owner Documentation Tracked Bugs Status
Firefox Metro Juan Becerra Bug Testing }}

Nightly

WebRTC

Using the Moztrap test as a guideline, verify that Firefox 27.0a1 is no worse than Firefox 22.0 when making AppRTC calls of >5 minutes in length. When testing, here are the guidelines to follow:

  • Have only one browser open on each machine at any given time
  • Have only one call running at any given time on a particular machine
  • Make sure the caller and callee are always on *different* machines.
  • Please don't test any 3+-way calls for this sanity check -- We just want to see the results for 1:1 (basic) calling
  • If you find regressions, report a bug and CC Randell Jesup and Maire Reavy; they can help track down if the regressions are real or not
    • Tip: be sure to provide extremely detailed steps to reproduce and witnessed results, as well as detailed information about your test environment; more information is better than not enough information.
Caller Callee Result
Firefox 27.0a1 on Windows 7 Firefox 27.0a1 on Windows 7
Firefox 22.0 on Windows 7 Firefox 27.0a1 on Windows XP
Chrome Stable on Windows 7 Firefox 27.0a1 on Windows 8 PASS
Firefox 27.0a1 on Windows 7 Firefox 22.0 on Mac OSX 10.8
Firefox 27.0a1 on Windows 7 Chrome Stable on Ubuntu 12.04
Firefox 22.0 on Windows XP Firefox 27.0a1 on Windows 7
Firefox 27.0a1 on Windows XP Firefox 22.0 on Windows XP
Firefox 27.0a1 on Windows XP Chrome Stable on Windows 8
Chrome Stable on Windows XP Firefox 27.0a1 on Mac OSX 10.8
Firefox 22.0 on Windows XP Firefox 27.0a1 on Ubuntu 13.04
Chrome Stable on Windows 8 Firefox 27.0a1 on Windows 7 PASS
Firefox 27.0a1 on Windows 8 Chrome Stable on Windows XP
Firefox 22.0 on Windows 8 Firefox 27.0a1 on Windows 8
Firefox 27.0a1 on Windows 8 Firefox 22.0 on Windows 8
Firefox 22.0 on Windows 8 Firefox 27.0a1 on Mac OSX 10.8
Chrome Stable on Windows 8 Firefox 27.0a1 on Ubuntu 13.04 FAIL (1*)
Firefox 27.0a1 on Mac OSX 10.8 Firefox 22.0 on Windows 7
Chrome Stable on Mac OSX 10.8 Firefox 27.0a1 on Windows XP
Firefox 22.0 on Mac OSX 10.8 Firefox 27.0a1 on Windows 8
Firefox 27.0a1 on Mac OSX 10.8 Chrome Stable on Mac OSX 10.8
Firefox 27.0a1 on Mac OSX 10.8 Firefox 22.0 on Ubuntu 13.04
Firefox 27.0a1 on Ubuntu 12.04 Chrome Stable on Windows 7
Firefox 22.0 on Ubuntu 12.04 Firefox 27.0a1 on Windows XP
Chrome Stable on Ubuntu 12.04 Firefox 27.0a1 on Windows 8
Firefox 27.0a1 on Ubuntu 12.04 Firefox 22.0 on Mac OSX 10.8
Firefox 27.0a1 on Ubuntu 12.04 Chrome Stable on Ubuntu 13.04

Detailed Results: (1*) Callee sometimes hears and sees the caller with a delay (~1sec)

Aurora

Branch Migration

Pre Sign-off Tasks

  • {{|}} Firefox branch migrated (Release Management)
  • {{|}} Mozmill Tests branch migrated (Automation Development)
  • {{|}} Updates throttled on aurora channel (Release Engineering)
  • {{|}} Updates verified throttled on aurora channel (QA)

Sign-off Tasks

  • {{|}} Mozmill Daily functional testruns pass with no failures (report)
  • {{|}} Mozmill Daily auroratest update testruns pass with no failures (report)
  • {{|}} Scope features tested and signed off for Aurora uplift
    • feature...

Post Sign-off Tasks

  • Aurora sign-off email sent (QA)
  • Updates unthrottled on aurora channel (Release Engineering)
  • Updates verified unthrottled on aurora channel (QA)

Beta

Bug Triage

Query Description Remaining
QAWANTED Bugs requiring immediate investigative assistance 1
VERIFYME Bugs with fixes requiring immediate verification
FIXED Bugs with fixes which should be verified as soon as possible
REGRESSION Bugs requiring identification of the first affected build
UNCONFIRMED Bugs requiring confirmation if they are reproducible in a current build

Known Issues

Bug QA Contact Status
Retrieved from "https://wiki.mozilla.org/index.php?title=Releases/Firefox_27/Test_Plan&oldid=712520"