MozillaWiki

Plugins:PepperAudioAPI: Difference between revisions

Page Discussion

Plugins:PepperAudioAPI (view source)

Revision as of 16:02, 26 April 2010

553 bytes added ,  26 April 2010
→‎Problem Summary
 
(14 intermediate revisions by one other user not shown)
Line 7: Line 7:
Low level audio component of the Pepper API.
Low level audio component of the Pepper API.


For more on Pepper, please see [[Plugins:PlatformIndependentNPAPI|Pepper Overview]]
For more on Pepper, please see [[NPAPI:Pepper|Pepper Overview]]


=== Specification  ===
=== Specification  ===
Line 32: Line 32:
*Only linear PCM formats will be supported.  
*Only linear PCM formats will be supported.  
*Straight forward to secure.  
*Straight forward to secure.  
*Implemented in Pepper's device API  
*Implemented in Pepper's NPDevice API  


The browser might be responsible for applying additional user volume or muting control,and may upsample/downsample the audio data before mixing with other browser audio sources.  
The browser might be responsible for applying additional user volume or muting control,and may upsample/downsample the audio data before mixing with other browser audio sources.  
Line 48: Line 48:
=====Background:=====
=====Background:=====


Up until now, NPAPI plug-ins have traditionally done audio output either via third party audio libraries or directly with platform specific audio APIs.  This document describes a unified audio API for PEPPER, an extension of NPAPI for platform independent developement.
NPAPI plug-ins have traditionally done audio output either via third party audio libraries or directly with platform specific audio APIs.  This document describes a unified audio API for PEPPER, an extension of NPAPI for platform independent developement.


Please refer to Appendix A for a list of links pertaining to audio.  Please refer to Appendix B for PEPPER design wiki -- where this document will migrate once approved.  
Please refer to Appendix A for a list of links pertaining to audio.  Please refer to Appendix B for PEPPER design wiki -- where this document will migrate once approved.  
Line 84: Line 84:
<pre>
<pre>
NPPepperExtensions *pepper;
NPPepperExtensions *pepper;
NPAudio *audio = NULL;
NPDevice *audio = NULL;
/* Get PEPPER extensions */
/* Get PEPPER extensions */
NPERR ret = NPN_GetValue(instance, NPNVPepperExtensions, &pepper);
NPERR ret = NPN_GetValue(instance, NPNVPepperExtensions, &pepper);
if (NPERR_NO_ERROR == ret) {
if (NPERR_NO_ERROR == ret) {
   /* successfully obtained Pepper extensions, now acquire audio... */
   /* successfully obtained Pepper extensions, now acquire audio... */
   audio = pepper->acquireDevice(NPAudioDeviceID);
   audio = pepper->acquireDevice(NPPepperAudioDevice);
} else {
} else {
   /* Pepper extensions are not available */
   /* Pepper extensions are not available */
Line 99: Line 99:
</pre>
</pre>


Once an interface to the audio device is acquired, an application can query capabilities of the device, and from there, create a configuration (a set of capabilities.) In this simple example, the application picks 44.1kHz, stereo output using int16 sample types. It uses query Capability to retrieve a recommended sample frame count for the output buffer.
Once an interface to the audio device is acquired, an application can query capabilities of the device, and from there, create a configuration (a set of capabilities.) In this simple example, the application picks 44.1kHz, stereo output using int16 sample types. It uses queryCapability() to retrieve a recommended sample frame count for the output buffer.


<pre>
<pre>
...
...
/* fill out request, use sample buffer size from configuration */
/* fill out request, use sample buffer size from configuration */
NPAudioConfig request;
NPDeviceContextAudioConfig request;
NPAudioConfig config;
NPDeviceContextAudioConfig config;
int32 ret;
int32 ret;
request.sampleRate = NPAudioSampleRate44100Hz;
request.sampleRate = NPAudioSampleRate44100Hz;
Line 145: Line 145:


/* fill out request, use sample buffer size from configuration */
/* fill out request, use sample buffer size from configuration */
NPAudioConfig request;
NPDeviceContextAudioConfig request;
NPAudioConfig config;
NPDeviceContextAudioConfig config;
NPAudioContext context;
NPDeviceContextAudio context;
int32 ret;
int32 ret;
request.sampleRate = NPAudioSampleRate44100Hz;
request.sampleRate = NPAudioSampleRate44100Hz;
Line 186: Line 186:


<pre>
<pre>
void appCallback(const NPAudioContext *context) {
void appCallback(const NPDeviceContextAudio *context) {
   /* assume using int16 format */
   /* assume using int16 format */
   int16 *outBuffer16 = (int16 *)context->outBuffer;
   int16 *outBuffer16 = (int16 *)context->outBuffer;
Line 207: Line 207:
/* From the application's NPAPI thread... */
/* From the application's NPAPI thread... */
/* setup request structure, use sample_buffer_size from configuration */
/* setup request structure, use sample_buffer_size from configuration */
NPAudioConfiguration request;
NPDeviceContextAudioConfig request;
NPAudioConfiguration config;
NPDeviceContextAudioConfig config;
pthread_t appAudioThreadID = 0;
pthread_t appAudioThreadID = 0;
volatile appAudioDone = false;
volatile appAudioDone = false;
Line 243: Line 243:
/* application's thread dedicated to audio */
/* application's thread dedicated to audio */
void* appAudioThread(void *userData) {
void* appAudioThread(void *userData) {
   /* context is sent via pthread's userData function arg */
   /* context & interface is sent via pthread's userData function arg */
   NPAudioContext *context = (NPAudioContext *)userData;
   NPDeviceContextAudio *context = (NPDeviceContextAudio *)userData;
   /* get pepper audio extensions (so we can call flushContext) */
   /* get pepper audio extensions (so we can call flushContext) */
   NPAudio *audio = (NPAudio *)context->userData;
   NPDevice *audio = (NPDevice *)context->userData;
   /* simple audio loop for this example, poll global variable */
   /* simple audio loop for this example, poll global variable */
   while (!appAudioDone) {
   while (!appAudioDone) {
Line 268: Line 268:
}
}
</pre>
</pre>


===Switching to Low Power (Silence)===
===Switching to Low Power (Silence)===
Line 323: Line 322:


/* forward decls */
/* forward decls */
typedef struct NPAudioContext NPAudioContext;
typedef struct NPDeviceContextAudio NPDeviceContextAudio;
typedef struct NPAudioConfig NPAudioConfig;
typedef struct NPDeviceContextAudioConfig NPDeviceContextAudioConfig;


/* user supplied callback function */
/* user supplied callback function */
typedef void (*NPAudioCallback)(const NPAudioContext *context);
typedef void (*NPAudioCallback)(const NPDeviceContextAudio *context);


/* Audio config structure */
/* Audio config structure */
struct NPAudioConfig {
struct NPDeviceContextAudioConfig {
int32 sampleRate;
int32 sampleRate;
int32 sampleType;
int32 sampleType;
Line 342: Line 341:


/* Audio context structure */
/* Audio context structure */
struct NPAudioContext {
struct NPDeviceContextAudio {
NPP npp;
NPP npp;
NPAudioConfig config;
NPDeviceContextAudioConfig config;
void *outBuffer;
void *outBuffer;
void *inBuffer;
void *inBuffer;
void *private;
void *privatePtr;
};
};
</pre>
</pre>
Line 374: Line 373:
</pre>
</pre>
'''NPError queryConfig(NPP npp, NPAudioConfig *request, NPAudioConfig *obtain)'''
'''NPError queryConfig(NPP npp, NPDeviceContextAudioConfig *request, NPDeviceContextAudioConfig *obtain)'''
<pre>
<pre>
   inputs:
   inputs:
NPP npp
NPP npp
Plug-in instance npp
Plug-in instance npp
NPAudioConfig *request
NPDeviceContextAudioConfig *request
A requested configuration, which is a set of capabilities
A requested configuration, which is a set of capabilities
outputs:
outputs:
NPAudioConfig *obtain
NPDeviceContextAudioConfig *obtain
The set of capabilities obtained, which may or may not match
The set of capabilities obtained, which may or may not match
request input.
request input.
Line 391: Line 390:
Okay if request & obtain pointers are the same.
Okay if request & obtain pointers are the same.
</pre>
</pre>
'''NPError initializeContext(NPP npp, const NPAudioConfig *config,
'''NPError initializeContext(NPP npp, const NPDeviceContextAudioConfig *config,
NPAudioContext *context)'''
NPDeviceContextAudio *context)'''
<pre>
<pre>
inputs:
inputs:
NPP npp
NPP npp
Plug-in instance npp
Plug-in instance npp
NPAudioConfig *config - a structure with which to configure the
NPDeviceContextAudioConfig *config - a structure with which to configure the
audio device
audio device
int32 sampleRate
int32 sampleRate
Line 434: Line 433:
pointer to user data for callback. Can be NULL.
pointer to user data for callback. Can be NULL.
outputs:
outputs:
NPAudioContext *context
NPDeviceContextAudio *context
Filled in, used to identify audio context.
Filled in, used to identify audio context.
returns:
returns:
Line 455: Line 454:
</pre>
</pre>
'''NPError setStateContext(NPP npp, NPAudioContext *context, int32 state, int32 value)'''
'''NPError setStateContext(NPP npp, NPDeviceContextAudio *context, int32 state, int32 value)'''
<pre>
<pre>
         inputs:
         inputs:
NPP npp
NPP npp
Plug-in instance npp
Plug-in instance npp
NPAudioContext *context
NPDeviceContextAudio *context
audio context to apply state
audio context to apply state
int32 state
int32 state
Line 482: Line 481:
</pre>
</pre>
'''NPError getStateContext(NPP npp, NPAudioContext *context, int32 state, int32 *value)'''
'''NPError getStateContext(NPP npp, NPDeviceContextAudio *context, int32 state, int32 *value)'''
<pre>
<pre>
inputs:
inputs:
NPP npp
NPP npp
Plug-in instance npp
Plug-in instance npp
NPAudioContext *context
NPDeviceContextAudio *context
audio context to apply state
audio context to apply state
int32 state
int32 state
Line 505: Line 504:
</pre>
</pre>
'''NPError flushContext(NPP npp, NPAudioContext *context)'''
'''NPError flushContext(NPP npp, NPDeviceContextAudio *context)'''
<pre>
<pre>
         inputs:
         inputs:
Line 511: Line 510:
For audio, this function is invoked from a non-NPAPI thread,
For audio, this function is invoked from a non-NPAPI thread,
so npp is ignored.
so npp is ignored.
NPAudioContext *context
NPDeviceContextAudio *context
context->outBuffer
context->outBuffer
pointer to output data. If this field is NULL, no output
pointer to output data. If this field is NULL, no output
Line 546: Line 545:
</pre>
</pre>
'''NPError destroyContext(NPP npp, NPAudioContext *context)'''
'''NPError destroyContext(NPP npp, NPDeviceContextAudio *context)'''
<pre>
<pre>
         inputs:
         inputs:
NPP npp
NPP npp
Plug-in instance npp
Plug-in instance npp
NPAudioContext context
NPDeviceContextAudio *context
audio context to shutdown
audio context to shutdown
returns:
returns:
Line 572: Line 571:
</pre>
</pre>
'''void (NPAudioCallback *)(NPAudioContext *context);'''
'''void (NPAudioCallback *)(NPDeviceContextAudio *context);'''
<pre>
<pre>
         inputs:
         inputs:
NPAudioContext context
NPDeviceContextAudio context
Audio context that generated this callback. Fields within the
Audio context that generated this callback. Fields within the
context
context
Line 633: Line 632:
queryCapability(npp, NPAudioCapabilitySampleRate, &value)
queryCapability(npp, NPAudioCapabilitySampleRate, &value)
output value:
output value:
Current 'native' sample rate of the underlying audio system. If an
Current 'native' sample rate of the underlying audio system. If an
application can use this rate, less upsampling/downsampling is
application can use this rate, less upsampling/downsampling is
likely to occur at the driver level.
likely to occur at the driver level.
NPAudioSampleRate44100Hz
NPAudioSampleRate44100Hz
NPAudioSampleRate48000Hz
NPAudioSampleRate48000Hz
NPAudioSampleRate96000Hz
NPAudioSampleRate96000Hz
notes:
notes:
Both NPAudioSampleRate44100Hz and NPAudioSampleRate48000Hz are
Both NPAudioSampleRate44100Hz and NPAudioSampleRate48000Hz are
Line 741: Line 740:
the implementation doesn't support audio input.
the implementation doesn't support audio input.
</pre>
</pre>


=== Appendix A - Links to audio related info  ===
=== Appendix A - Links to audio related info  ===
Line 871: Line 869:
+----------+
+----------+
       +-----+       +-----+
       +-----+               +-----+
       | ch2 |              | ch3 |
       | ch2 |              | ch3 |
left  +-----+              +-----+ right
left  +-----+              +-----+ right
Line 955: Line 953:
left  +-----+                +-----+ right
left  +-----+                +-----+ right
surround                              surround
surround                              surround
(Ls)                                 (Rs)
(Ls)       +------+      +------+    (Rs)
 
    left  | ch3  |      | ch4  | right
    back  +------+      +------+ back
    (Lb)                        (Rb)
</pre>
</pre>


Line 983: Line 983:
                 | listener |
                 | listener |
       +-----+  +----------+  +-----+
       +-----+  +----------+  +-----+
       | ch4 |                | ch5 |
       | ch6 |                | ch7 |
left  +-----+                +-----+ right
left  +-----+                +-----+ right
surround                              surround
surround                              surround
(Ls)                                 (Rs)
(Ls)       +------+      +------+    (Rs)
    left  | ch4  |      | ch5  | right
    back  +------+      +------+ back
    (Lb)                        (Rb)
 
</pre>
</pre>


Josh
Confirmed users, Bureaucrats and Sysops emeriti
1,680

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/188521...218382"