11
edits
Plugins:PepperAudioAPI: Difference between revisions
no edit summary
(Created page with '= Status = Under consideration. = Problem Summary = Low level audio component of the Pepper API. === Specification === '''Low Level Audio in PEPPER''' * Last modified: Dece…') |
No edit summary |
||
| Line 7: | Line 7: | ||
Low level audio component of the Pepper API. | Low level audio component of the Pepper API. | ||
=== Specification === | === Specification === | ||
=== Low Level Audio in PEPPER === | |||
* Last modified: December 7, 2009 | *Last modified: December 7, 2009 | ||
* Author: Nicholas Fullagar (Google) | *Author: Nicholas Fullagar (Google) | ||
* Contributors: | *Contributors: | ||
=====Objective:===== | |||
*First pass implementation is to provide 16bit, stereo, CD (44.1kHz) or DAT (48kHz) quality audio. | |||
*Deliver low latency audio on systems that have good audio drivers. (<=20ms) | |||
*Be flexible enough to scale additional features in future versions | |||
**additional sample formats | |||
**additional speaker configurations | |||
**audio input | |||
*It will be the application's responsibility to provide higher level audio functionality. | |||
**If an audio device is available, this API will guarantee | |||
**stereo output (regardless of physical speaker configuration) | |||
**the availability of both 44.1kHz and 48kHz output. | |||
*the availability of int16 as a sample format. | |||
*The basic audio interface is a pure "C" API. | |||
*Only linear PCM formats will be supported. | |||
*Straight forward to secure. | |||
*Implemented in Pepper's device API | |||
The browser might be responsible for applying additional user volume or muting control,<br>and may upsample/downsample the audio data before mixing with other browser audio<br>sources.<br> | |||
All future versions of PEPPER will support version 1 (this proposal) as a bare minimum.<br> | |||
Trusted PEPPER modules will be able to open more than one audio context.<br>Untrusted PEPPER modules might be restricted to a single audio context.<br>In either environment, it is assumed multi-threading is available.<br> | |||
This document contains two models - one based on callbacks, and one based on a blocking<br>push model. While both models are fundamentally the same, they do carry some subtle<br>differences:<br> | |||
*Blocking Push API provides more control, and assumes threading will be explicitly handled by the application. When writing trusted PEPPER modules without a platform independent threading library, this may imply the application writer to #if WIN32, #if PTHREADS, etc. around the application's audio thread setup code. | |||
*Callback API does not require application to setup/teardown audio thread, and the API entry points are all on the NPAPI thread -- only the callback occurs on an implicit, dedicated audio thread. | |||
*There might be some subtle implementation details when it comes to how each model needs to signal between the audio device and the PEPPER audio interface. These signals will need to be as fast and low latency as possible, so minimizing how many signals are required will be important.<br> | |||
=====Background:===== | |||
Native Client plug-ns have up until now used an audio API very similar to a blocking<br>flushContext() call, usually done on a thread dedicated to audio. The API call was<br>implemented as a syscall, and at the service runtime (kernel) level used SDL to deliver low<br>latency audio across multiple platforms. As Native Client moves into a more secure<br>sandbox environment, untrusted code will no longer be able to communicate directly with<br>audio devices in the same process. In the new audio model, trusted code in another<br>process will be attached to the audio device, and the sandboxed Native Client untrusted<br>application will use shared memory and sockets to communicate its audio needs outside the<br>sandbox. The new API for this model will be part of PEPPER, a variant of NPAPI that is<br>designed to be platform neutral.<br> | |||
Please refer to Appendix A for a list of links pertaining to audio.<br>Please refer to Appendix B for PEPPER design wiki -- where this document will migrate once<br>approved.<br> | |||
Both models will need additional PEPPER events. Please refer to Appendix C for audio<br>related events.<br> | |||
Both models support multi-channel configurations. Please refer to Appendix E for multi-<br>channel output scenarios.<br>Please refer to Appendix F for input scenarios.<br>Please refer to Appendix G for contributers. | |||
===Changes to Pepper's Device API=== | |||
Previously, Pepper's device API consisted of the methods initializeContext, flushContext, and destroyContext. Audio extends this API to queryCapability, queryConfig, initializeContext, flushContext, getState, setState, and destroyContext. | |||
*queryCapability - ask a device about a specific capability | |||
*queryConfig - ask a device about a configuration (a set of capabilities) | |||
*initializeContext - given a configuration, initialize a device context | |||
*flushContext - flush context to device, either blocking or with a completion callback | |||
*getStateContext - get state information from device context | |||
*setStateContext - set device context state | |||
*destroyContext - shutdown and destroy device context | |||
===NPAPI Thead, Plug-ins, and Plug-in Instances=== | |||
The following device methods must be called from the NPAPI thread (Tp). They should not<br>be called from any other thread, unless it is via NPN_ThreadAsyncCall.<br> | |||
*queryCapability, queryConfig, initializeContext, getStateContext, setStateContext,destroyContext<br> | |||
The only device method that can be called from another thread is flushContext when using<br>the push model. When using the callback model, it is expected that the user supplied<br>callback will be invoked on a thread dedicated to audio. This dedicated audio thead is not<br>an NPAPI thread.<br> | |||
When creating an audio context, usually this is done as part of a plug-in instance. In<br>untrusted Pepper modules, there will be one instance per sandboxed process. However,<br>this could change in the future, and an untrusted Pepper plug-in could host multiple<br>instances in the same sandbox. In this case, it may be desirable to have one audio device,<br>with it's own application supplied mixer, to be shared amongst mutliple untrusted plug-in<br>instances. This audio context should be created at Plug-in initialization, using NULL as the<br>npp instance argument.<br> | |||