QA/Text to Speech

From MozillaWiki
< QA
Jump to: navigation, search

Overview

Purpose

The purpose of this wiki is to serve as a general test plan for verifying that the Text to Speech works as expected.

Scope

The testing will be focus on

  • testing that the "media.webspeech.synth.enabled" pref is set to true by default on Nightly 46 and false on Aurora 45.

Testing will be performed manually, by QA Release team.

Ownership

Developer contacts: Eitan Isaacson
QA: Camelia Badau

Testing Summary

Scope of testing

In Scope

The main testing scope is to assure that the Text to Speech works correct.[TBD]

Requirements for testing

Environments

Testing will be performed on following OSes (both x86 & x64 infrastructures):

  • Mac OS X 10
  • Windows 7
  • Windows 10
  • Ubuntu 13.10

Testing on specific devices is not needed.

Test Strategy

Test Objectives

Following table contains main features to be tested:

Ref Function Test Objective Evaluation Criteria
1 basic scenarios TBD

Test Types

Type of Testing Required (Yes/No) Owners Details/Comments
Manual Yes Moz QA Rel team
Automation No
Performance No
Security No
Usability No
Metrics/Telemetry TBD

Builds

This section should contain links for builds with the feature

Nightly builds can be found at link

Aurora builds can be found at link

Test Execution Schedule

The following table identifies the anticipated testing period available for test execution.

Project phase Start Date End Date
Start project
Study documentation/specs received from developers -
QA - Test plan creation
QA - Test cases/Env preparation
QA - Nightly Testing
QA - Aurora Testing
QA - Beta Testing
Release Date

Testing Tools

Detail the tools to be used for testing, for example see the following table:

Process Tool
Test plan creation Mozilla wiki
Test case creation Google docs
Test case execution Google docs
Bugs management Bugzilla

Status

Overview

[LANDED] Nightly 45: first landed on - 09.11.2015

Risk analysis

- No Risks found at the moment

References

Meta: Bug 1003439

Test cases

Overview

  • Tested that speech synthesis correctly works on several demo pages
  • Verified the "media.webspeech.synth.enabled" pref

Test suites

Full Test Suite - can be reached here

Bugs Work

Tracking bug: Meta bug 1003439

Bug fix verification
Bug No Summary Status Firefox Version
1003452 Text to Speech API on Firefox for Mac RESOLVED FIXED Firefox 44
1003457 Text to Speech API on Firefox for Windows RESOLVED FIXED Firefox 42
1003464 Text to Speech API on Firefox for Linux RESOLVED FIXED Firefox 44
Bug 1230533 Speech synthesis doesn't stop when the page is reloaded VERIFIED FIXED Firefox 46
Bug 1238522 Intermittently, the speech synthesis doesn't start after changing the Pitch VERIFIED FIXED Firefox 49
Bug 1252076 Speech synthesis doesn't stop when tab is closed or Firefox quit RESOLVED DUPLICATE of bug 1230533
Bug 1252048 mozilla::dom::Event::InitEvent RESOLVED WORKSFORME
Bug 1252051 Abort | NS_DebugBreak | mozilla::dom::PSpeechSynthesisRequestParent::Send OnBoundary RESOLVED WORKSFORME
New bugs

Bug 1238538 - [Linux] Speech synthesis doesn't work correctly on http://eeejay.github.io/webspeechdemos/
Bug 1221443 - Speech synthesis doesn't stop when tab is muted

Sign off

Criteria

  • All test cases should be executed
  • All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)

Results

pref = true by default

Pre-Aurora Sign-off (2016-06-13)

    • Manual testing covered Windows 7 x64, Windows 10 x64, Ubuntu 14.04 x32 and Mac OS X 10.9.5.
    • No new issues were found

Mid-Aurora Sign-off (2016-07-07)

    • Manual testing covered Windows 7 x64, Windows 10 x64, Ubuntu 13.10 x32 and Mac OS X 10.9.5.
    • No new issues were found

Pre-Beta Sign-off (2016-07-29)

    • Manual testing covered Windows 7 x64, Windows 10 x32, Ubuntu 14.04 x64 and Mac OS X 10.9.5.
    • No new issues were found

Pre-Release Sign-off (2016-09-02)

    • Manual testing covered Windows 7 x64, Windows 10 x64, Ubuntu 16.04 x64 and Mac OS X 10.11.6.
    • No new issues were found


pref = false by default

Mid-Aurora Sign-off (2016-01-12)

    • Manual testing covered Windows 7 x64, Ubuntu 13.10 x32 and Mac OS X 10.9.5.
    • There were 2 new issues found: bug 1238538 and bug 1238522
    • Conditional Sign-Off

Checklist

  • TBD