Changes

Under consideration.

Over the past few months a number of us have also been discussing some of the issues facing NPAPI as a platform independent browser plugin framework. First, we feel there are some changes needed to NPAPI to keep up with recent developments in browser technology, such as out of process plugin execution. Second, while NPAPI provides an extensive framework for writing plugins, many end up relying on operating system or browser platform specific features. This is especially true of windowed plugins that implement 2D or 3D graphics, where almost the entirety of the plugin may consist of operating system specific graphics or event APIs. Third, both windowed and windowless plugins are difficult to composite with other layers correctly today, making it difficult for a page author to achieve the same look and feel across browsers and operating systems. This proposal intends to be a starting point for addressing these issues, and consists of four topics.

A somewhat related issue is how to extend NPAPI to provide platform independent access to new hardware or software features such as digital cameras, SD cards, bar code readers, accelerometers, webcams, microphones, etc. While certainly important, this is deferred to future discussions.

To provide these improvements to NPAPI we propose that the changes below be another platform, called "Pepper", that may be implemented by a browser and requested by a plugin at startup.

</pre>

A vector of function pointers is used to provide access to Pepper-specific functionality. This vector, called NPExtensions, currently looks like

</pre>

The current NPAPI model defines the threading model within one process such that whatever thread invokes NPP_New is the only thread that may invoke NPN_* functions. Several browsers either currently support or are considering moving NPAPI plugins to a separate process from the renderer process (responsible for rendering a page 's content). With these processes distinct, we have the following:

</pre>

We propose that NPN_* calls, with only one exception, can only be made from Tp. This allows us to maintain an important invariant, namely that if Tp is executing any NPP code, then Tr is blocked. This eliminates the possibility of simultaneous JavaScript and plugin activities.

The one exception to NPN_* calls is NPN_PluginThreadAsyncCall.

We observe that implementing NPN_PluginThreadAsyncCall this way results in additional call/callback/call latency for NPN_* calls. We believe this is reasonable to provide a well-defined interaction, but realize we may need to address the efficiency of this mechanism.

We will use device context to refer to platform independent devices (e.g., audio, 3D graphics, hardware accelerators). To facilitate this, we propose the addition of a new struct type.

</pre>

<pre>

Use queryCapability to ask a device about a specific capability. Use queryConfig to ask a device about a set of capabilities. The obtained configuration is the closest match to the requested configuration. If the obtained configuration is undesirable, the application is free to attempt additional queryConfig calls until a satisfactory configuration is discovered. Use the obtained configuration as input when initializing a device.

<pre>

Causes the creation of a device context for use by the specified instance.

<pre>

Get/Set sideband state information with respect to device context.

<pre>

To facilitate low latency devices, flushContext() can be invoked from non-NPAPI threads.

<pre>

Causes the context specified to be destroyed. The memory pointed to by context may be freed and accesses are undefined.

There is a possible performance opportunity that should be noted.

Given the threading model we have defined, it is desirable to allow InitializeRenderContext and FlushRenderContext to be called from other plugin threads than Tp. This eliminates the need to do the associated NPN_PluginThreadAsyncCall, and avoids the context switch penalties that might be associated.

Old-style NPAPI plugins display 2D graphics in one of two modes: windowless or windowed, each with their own features and limitations. The plugin can specify which mode it would prefer to execute in.

* From the browser's perspective, it is difficult to manage the child windows. For example, keeping them and the page in sync when the page scrolls is challenging. There are also responsiveness challenges, in that a hung plugin can hang the entire browser UI by not processing messages in its window, even if it runs in another process or thread.

We propose that only windowless plugins be supported, that compositing should be done outside the plugin, and that no native OS events should be delivered to the plugin. New APIs and events will be added to fill in the resulting holes in functionality.

Animation can be implemented in the 2D device API using the callback from the flush function. After the browser paints the image to the screen, it will call the plugin's specified callback. At this point, the plugin can generate a new frame. This callback notification is the rate limiting signal so the plugin does not generate images faster than they can be painted to the screen.

<pre>typedef struct _NPDeviceContext2D

} NPDeviceContext2D;</pre>

<pre>/* Get the 2D device function pointers. */

device2d->flushContext(npp, &context, NULL, NULL);</pre>

The details of 3D rendering are still being worked out. The general idea is that there will be a serialized set of OpenGL commands sent from the plugin to the browser for display on the video card. It is currently not decided whether the format of this command stream is standardized, or whether there will be a standardized API that generates a browser-specific command stream.

There is currently only one configuration option for the 3D device, which is the size of the command buffer. Its likely additional configuration options will be added later.

} NPDeviceContext3DState;</pre>

Whereas currently NPP_SetWindow passes both some geometry information and a handle to a native window (for windowed plugins), in our proposal, it only conveys geometry and format information. For example.

</pre>

Plugins should receive all their events from NPP_HandleEvent. We believe that standardizing the event types and payloads based on, for example, DOM events, would be sufficient to provide browser and OS independent events. If not, the goal can be accomplished by possibly extending the set of event types or data on the event structures to convey more information.

We will use specifically sized types (int32_t, int16_t, int8_t, etc.) to avoid sizeof(int) and sizeof(wchar) differences between compilers and machine architectures, and have very strict padding and alignment specifications.

The plugin can use NPN_GetValue() & NPN_SetValue() to query and set future event models. Currently, only version 1.0 model is supported, and it should be safe to assume that version 1.0 model will always be available, and will be the default at startup.

There is no timer event, see NPN_ScheduleTimer() and NPN_UnscheduleTimer() for timer support. There is no quit event during application shutdown, see JavaScript's "onunload" There is no resize event; see NPN_SetWindow(), which will be correctly serialized with respect to event order.

Currently, we are proposing a platform neutral event for focus gained/lost, and a similar event for minimize/background tab. There is also a proposal for NPAPI to support three new NPN functions: NPN_UnfocusInstance(), NPN_LostFocus(), and NPN_GotFocus(). The NPAPI proposal uses NPN functions in leu of events for a few reasons, two of which are: 1) they didn't want to define what the focus event should be for each OS platform, 2) allow the application to explicitly give up focus. Since PEPPER events are platform neutral, the 1st reason doesn't really apply here. The 2nd reason might be good enough to consider using AKH instead of focus events. The downside is that as more events become NPN_* functions, the event model starts to become an inconsistent mix of traditional events and NPN_* functions. https://wiki.mozilla.org/Plugins:AdvancedKeyHandling

It is important that all compilers, on all supported platforms, both trusted and untrusted, preserve structure order, field size, and alignment padding, as these structures might be used as a bit copyable wire format between different processes. These seperate processses could be compiled using different compilers and/or compilier options, while sharing these structures via shared memory. For the definitions below, the byte offset for each field is in square brackets. Unused portions of the event structure should be treated as undefined data.

</pre>

The current process of determining which plugins are available to the browser typically involves loading various shared libraries and querying them for types and extensions addressed by the plugin. We propose that more efficient method would be to add a special section to the shared library for the plugin.

This enables browsers to scan the plugins without loading them, which becomes more difficult with out-of-process plugins and sandbox protection.

Brief: Pepper will feature very low level audio support, with an emphasis on low latency. To start, initial implementations can support 44.1kHz and 48kHz stereo int16 audio streams. The API can scale to include higher sample rates (96kHz), multi-channel output (5ch, 5.1ch, +more), floating point samples and audio input. It assumes higher level audio functionality, such as software mixing, will be layered on top with additional libraries.

The whole Pepper audio design document is too long to embed here.&nbsp; [[Plugins:PepperAudioAPI|Please click here for the Pepper Audio API]]<br>