Confirmed users
656
edits
Audio Data API Review Version: Difference between revisions
Jump to navigation
Jump to search
→DOM Implementation
| Line 413: | Line 413: | ||
interface nsIDOMNotifyAudioAvailableEvent : nsIDOMEvent | interface nsIDOMNotifyAudioAvailableEvent : nsIDOMEvent | ||
{ | { | ||
// mozFrameBuffer is really a Float32Array | // mozFrameBuffer is really a Float32Array | ||
readonly attribute | readonly attribute jsval frameBuffer; | ||
readonly attribute float | readonly attribute float time; | ||
}; | }; | ||
</pre> | </pre> | ||
The ''' | The '''frameBuffer''' attribute contains a typed array ('''Float32Array''') with the raw audio data (32-bit float values) obtained from decoding the audio (e.g., the raw data being sent to the audio hardware vs. encoded audio). This is of the form <nowiki>[channel1, channel2, ..., channelN, channel1, channel2, ..., channelN, ...]</nowiki>. All audio frames are normalized to a length of channels * 1024 by default, but could be any power of 2 between 512 and 32768 if the user has set a different length using the '''mozFrameBufferLength''' attribute. | ||
The ''' | The '''time''' attribute contains a float representing the time in seconds since the start. | ||
===== nsIDOMHTMLMediaElement additions ===== | ===== nsIDOMHTMLMediaElement additions ===== | ||
Audio metadata is made available via three new attributes on the HTMLMediaElement. By default these attributes | Audio metadata is made available via three new attributes on the HTMLMediaElement. By default these attributes throw if accessed before the '''LoadedMetadata''' event occurs. Users who need this info before the audio starts playing should not use '''autoplay''', since the audio might start before a loadmetadata handler has run. | ||
The three new attributes are defined as follows: | The three new attributes are defined as follows: | ||
| Line 435: | Line 435: | ||
</pre> | </pre> | ||
The '''mozChannels''' attribute contains the number of channels in the audio resource (e.g., 2). The '''mozSampleRate''' attribute contains the number of samples per second that will be played, for example 44100. Both are | The '''mozChannels''' attribute contains the number of channels in the audio resource (e.g., 2). The '''mozSampleRate''' attribute contains the number of samples per second that will be played, for example 44100. Both are read-only. | ||
The '''mozFrameBufferLength''' attribute indicates the number of samples that will be returned in the framebuffer of each ''' | The '''mozFrameBufferLength''' attribute indicates the number of samples that will be returned in the framebuffer of each '''MozAudioAvailable''' event. This number is a total for all channels, and by default is set to be the number of channels * 1024 (e.g., 2 channels * 1024 samples = 2048 total). | ||
The '''mozFrameBufferLength''' attribute can also be set to a new value, if users want lower latency, or larger amounts of data, etc. The size | The '''mozFrameBufferLength''' attribute can also be set to a new value, if users want lower latency, or larger amounts of data, etc. The size given '''must''' be a power of 2 between 512 and 32768. The following are all valid lengths: | ||
* 512 | * 512 | ||
| Line 449: | Line 449: | ||
* 32768 | * 32768 | ||
Using any other size will result in an exception being thrown. The best time to set a new length is after the '''loadedmetadata''' event fires, when the audio info is known, but before the audio has started or ''' | Using any other size will result in an exception being thrown. The best time to set a new length is after the '''loadedmetadata''' event fires, when the audio info is known, but before the audio has started or '''MozAudioAvailable''' events begun firing. | ||
===== nsIDOMHTMLAudioElement additions ===== | ===== nsIDOMHTMLAudioElement additions ===== | ||
| Line 461: | Line 461: | ||
</pre> | </pre> | ||
The '''mozSetup()''' method allows an <audio> element to be setup for writing from script. This method '''must''' be called before '''mozWriteAudio''' can be called, since an audio stream has to be created for the media element. It takes | The '''mozSetup()''' method allows an <audio> element to be setup for writing from script. This method '''must''' be called before '''mozWriteAudio''' or '''mozCurrentSampleOffset''' can be called, since an audio stream has to be created for the media element. It takes two arguments: | ||
# '''channels''' - the number of audio channels (e.g., 2) | # '''channels''' - the number of audio channels (e.g., 2) | ||
| Line 470: | Line 470: | ||
The '''mozSetup()''' method, if called more than once, will recreate a new audio stream (destroying an existing one if present) with each call. Thus it is safe to call this more than once, but unnecessary. | The '''mozSetup()''' method, if called more than once, will recreate a new audio stream (destroying an existing one if present) with each call. Thus it is safe to call this more than once, but unnecessary. | ||
The '''mozWriteAudio()''' method can be called after '''mozSetup()'''. It allows audio data to be written directly from script. It takes one argument, '''array'''. This is a JS Array (i.e., new Array()) or a typed float array (i.e., new Float32Array()) containing the audio data (floats) you wish to write. It must be 0 or N elements in length, where N % channels == 0, otherwise | The '''mozWriteAudio()''' method can be called after '''mozSetup()'''. It allows audio data to be written directly from script. It takes one argument, '''array'''. This is a JS Array (i.e., new Array()) or a typed float array (i.e., new Float32Array()) containing the audio data (floats) you wish to write. It must be 0 or N elements in length, where N % channels == 0, otherwise an exception is thrown. | ||
The '''mozWriteAudio()''' method returns the number of samples that were just written, which may or may not be the same as the number in '''array'''. Only the number of samples that can be written without blocking the audio hardware will be written. It is the responsibility of the caller to deal with any samples that don't get written in the first pass (e.g., buffer and write in the next call). | The '''mozWriteAudio()''' method returns the number of samples that were just written, which may or may not be the same as the number in '''array'''. Only the number of samples that can be written without blocking the audio hardware will be written. It is the responsibility of the caller to deal with any samples that don't get written in the first pass (e.g., buffer and write in the next call). | ||