HTML5 Speech API

From MozillaWiki
Jump to navigation Jump to search

What and Why?

The HTML Speech Incubator group has proposed the implementation of speech technology in browsers in the form of uniform, cross-platform APIs that can be used to build rich web applications. The API itself contains 2 components -

  • Speech Input API
  • Text to Speech API


1.Speech Input API

The speech input API aims to provide an alternative input method for web applications, without using a keyboard or other physical device. This API can be used to input commands, fill input elements, give directions etc. It is based on SpeechRequest[[1]]

The API consists of 2 main components -

  • A Media Capture API to capture a stream of raw audio. Its implementation is platform dependent. Mac, Windows and Linux are to be supported first, eventually adding support for android.
  • A streaming API to asynchronously stream microphone data to a speech recognition server and to get the results back. This will be similar to how XMLHttpRequest is implemented. The api should be able to support both local and remote engines or a combination of both depending on the network connection available.


Security/Privacy issues

  • A speech input session should be allowed only with the user's consent. This could be provided using a doorhanger notification.
  • The user should be notified when audio is being recorded possibly using a record symbol somewhere in the web browser UI itself like the URL bar or status bar.


API Design -

The API will look like the interface described in the SpeechRequest proposal.

  • The developer should be able to specify a grammar(using SRGS) for the speech which is useful when the set of possible commands is limited. The recognition response would be in the form of an EMMA document.
  • The developer should be allowed to set a threshold for accuracy and sensitivity to improve performance.
  • The developer should be able to choose what speech engine to use.
  • The developer should be able to start, stop, handle errors and multiple requests as required.


2.Text To Speech API

The text to speech API will be based on google's proposal(http://lists.w3.org/Archives/Public/public-xg-htmlspeech/2011Feb/att-0022/htmltts-draft.html).This API can be used for speech translation, turn by turn navigation, dialog systems etc.

API Design -

  • The API will introduce a new element <tts> that extends HTMLMediaElement. It will be similar to how the <audio> and and <video> tags are implemented.
  • A playback UI should allow the user to start, stop and disable text to speech. The current spoken word can be highlighted.
  • The developer should be able to specify the language, position, start/stop playback and handle errors programmatically.
  • The API should itself be independent of the underlying speech synthesizer. If speech synthesis is not supported, appropriate text should be displayed.
  • What speech engines is yet to be decided.
Retrieved from "https://wiki.mozilla.org/index.php?title=HTML5_Speech_API&oldid=304969"