1,295
edits
User:Roc/AudioBufferProposal: Difference between revisions
Jump to navigation
Jump to search
no edit summary
No edit summary |
No edit summary |
||
| Line 1: | Line 1: | ||
== Overview == | == Overview == | ||
This proposal aims to solve the data-race problem with <tt>AudioBuffer</tt>'s mutable | This proposal aims to solve the data-race problem with <tt>AudioBuffer</tt>'s mutable <tt>AudioContext</tt>s, in a way that provides a high degree of compatibility with existing API usage, but avoids requiring memory copies in almost all cases (even for code using existing APIs). | ||
== Specification Changes == | == Specification Changes == | ||
| Line 9: | Line 9: | ||
<tt>AudioBuffer</tt> is extended with one new method: | <tt>AudioBuffer</tt> is extended with one new method: | ||
partial interface | partial interface AudioBuffer { | ||
void | void CopyChannelDataTo(long channelNumber, unsigned long start, unsigned long length, AudioContext destination); | ||
} | } | ||
The <tt>CopyChannelDataTo</tt> method copies a range of samples from the specified channel of the <tt>AudioBuffer</tt> to the destination array. If start is less than zero, start plus length is greater than the <tt>AudioBuffer</tt>'s length, or length is greater than the destination array's length, an INDEX_SIZE_ERR exception must be thrown. | The <tt>CopyChannelDataTo</tt> method copies a range of samples from the specified channel of the <tt>AudioBuffer</tt> to the <tt>destination</tt> array. If <tt>start</tt> is less than zero, <tt>start</tt> plus <tt>length</tt> is greater than the <tt>AudioBuffer</tt>'s <tt>length</tt>, or <tt>length</tt> is greater than the <tt>destination</tt> array's length, an <tt>INDEX_SIZE_ERR</tt> exception must be thrown. | ||
''<small>Note: This method can be used to fill part of an array by passing in a | ''<small>Note: This method can be used to fill part of an array by passing in a <tt>AudioContext</tt> that's a view onto the larger array.</small>'' | ||
''<small>Note: When reading data from an <tt>AudioBuffer</tt>'s channels, and the data can be processed in chunks, <tt>CopyChannelDataTo</tt> should be preferred to calling getChannelData | ''<small>Note: When reading data from an <tt>AudioBuffer</tt>'s channels, and the data can be processed in chunks, <tt>CopyChannelDataTo</tt> should be preferred to calling <tt><tt>getChannelData</tt></tt> and accessing the resulting array, because it may avoid unnecessary memory allocation and copying.</small>'' | ||
An internal operation "acquire the contents of an <tt>AudioBuffer</tt>" is invoked when the contents of an <tt>AudioBuffer</tt> are needed by some API implementation. This operation returns immutable channel data to the invoker. When an "acquire the contents" operation occurs, run the following steps: | An internal operation "acquire the contents of an <tt>AudioBuffer</tt>" is invoked when the contents of an <tt>AudioBuffer</tt> are needed by some API implementation. This operation returns immutable channel data to the invoker. When an "acquire the contents" operation occurs, run the following steps: | ||
# If any of the <tt>AudioBuffer</tt>'s ArrayBuffers have been neutered, abort these steps and return zero-length channel data buffers to the invoker. | # If any of the <tt>AudioBuffer</tt>'s ArrayBuffers have been neutered, abort these steps and return zero-length channel data buffers to the invoker. | ||
# Neuter all ArrayBuffers for arrays previously returned by getChannelData | # Neuter all ArrayBuffers for arrays previously returned by <tt><tt>getChannelData</tt></tt> on this <tt>AudioBuffer</tt>. | ||
# The underlying data buffers for those ArrayBuffers are retained by the <tt>AudioBuffer</tt> and returned to the invoker. | # The underlying data buffers for those ArrayBuffers are retained by the <tt>AudioBuffer</tt> and returned to the invoker. | ||
# New ArrayBuffers containing copies of the data are attached to the <tt>AudioBuffer</tt> to be returned by the next call to getChannelData | # New ArrayBuffers containing copies of the data are attached to the <tt>AudioBuffer</tt> to be returned by the next call to <tt><tt>getChannelData</tt></tt>. | ||
''<small>Note: Acquiring the contents of an <tt>AudioBuffer</tt> can be implemented without copying channel data. In particular, the last step can be performed lazily at the next getChannelData | ''<small>Note: Acquiring the contents of an <tt>AudioBuffer</tt> can be implemented without copying channel data. In particular, the last step can be performed lazily at the next <tt><tt>getChannelData</tt></tt> call (if there is one; there often won't be). A sequence of consecutive "acquire the contents" operations with no intervening <tt><tt>getChannelData</tt></tt> (e.g. multiple AudioBufferSourceNodes playing the same <tt>AudioBuffer</tt>) can be implemented with no allocations or copying.</small>'' | ||
''<small>Note: Implementations can perform an additional optimization: if getChannelData | ''<small>Note: Implementations can perform an additional optimization: if <tt><tt>getChannelData</tt></tt> is called on an <tt>AudioBuffer</tt>, fresh ArrayBuffers have not yet been allocated, but all invokers of previous "acquire the contents" operations on an <tt>AudioBuffer</tt> have stopped using the <tt>AudioBuffer</tt>'s data, the raw data buffers can be recycled for use with new AudioBuffers, avoiding any reallocation or copying of the channel data.</small>'' | ||
The "acquire the contents of an <tt>AudioBuffer</tt>" operation is invoked in the following cases: | The "acquire the contents of an <tt>AudioBuffer</tt>" operation is invoked in the following cases: | ||
| Line 38: | Line 38: | ||
== Additional Notes == | == Additional Notes == | ||
A good implementation following the advice above will not allocate or copy buffers of channel data any more than an implementation of the "freely share memory" proposal, except when an application calls getChannelData | A good implementation following the advice above will not allocate or copy buffers of channel data any more than an implementation of the "freely share memory" proposal, except when an application calls <tt><tt>getChannelData</tt></tt> on an <tt>AudioBuffer</tt> that is "in use" ("associated with a live AudioNode", in Jer's proposal). If such an application writes to the returned array, that is deprecated behavior under any proposal, but this proposal defines more predictable results than the "freely share memory" proposal. If such an application only reads the returned array, it can probably be modified to use <tt>CopyChannelDataTo</tt> instead, which will reduce the memory overhead to a negligible level. | ||
This proposal relies heavily on ArrayBuffer neutering. Some people want to avoid the use of neutering, but the TAG declined to endorse that position when asked. | This proposal relies heavily on ArrayBuffer neutering. Some people want to avoid the use of neutering, but the TAG declined to endorse that position when asked. | ||
| Line 46: | Line 46: | ||
I argue that this proposal is no more complicated than other proposals. Compared to Jer's proposal, this proposal has less API surface, and less complexity for Web developers who don't need to read or write channel data after an <tt>AudioBuffer</tt> has been used (since they don't have to think about node liveness), or who are not very concerned about performance. For developers who are very concerned about performance, and who want to modify channel data between uses of an <tt>AudioBuffer</tt>, the proposals are very similar; they'll have to think about node liveness. (Jer's proposal throws an exception when writing data to an in-use <tt>AudioBuffer</tt>, which is informative for Web authors who want to avoid copies; but with my proposal we can provide a similar alert through Web developer tools.) My proposal may be more complicated for implementers --- or not, since Jer's proposal requires implementations to track precisely when an <tt>AudioBuffer</tt> is associated with a live AudioNode, and this proposal does not. | I argue that this proposal is no more complicated than other proposals. Compared to Jer's proposal, this proposal has less API surface, and less complexity for Web developers who don't need to read or write channel data after an <tt>AudioBuffer</tt> has been used (since they don't have to think about node liveness), or who are not very concerned about performance. For developers who are very concerned about performance, and who want to modify channel data between uses of an <tt>AudioBuffer</tt>, the proposals are very similar; they'll have to think about node liveness. (Jer's proposal throws an exception when writing data to an in-use <tt>AudioBuffer</tt>, which is informative for Web authors who want to avoid copies; but with my proposal we can provide a similar alert through Web developer tools.) My proposal may be more complicated for implementers --- or not, since Jer's proposal requires implementations to track precisely when an <tt>AudioBuffer</tt> is associated with a live AudioNode, and this proposal does not. | ||
Compared to the "freely share memory" proposal, assuming that proposal is fully fleshed out to define what authors and implementations are allowed to do, this proposal is almost the same for Web developers. The only extra complexity for Web developers is that they should call <tt>CopyChannelDataTo</tt> to read channel contents instead of reading from getChannelData | Compared to the "freely share memory" proposal, assuming that proposal is fully fleshed out to define what authors and implementations are allowed to do, this proposal is almost the same for Web developers. The only extra complexity for Web developers is that they should call <tt>CopyChannelDataTo</tt> to read channel contents instead of reading from <tt><tt>getChannelData</tt></tt> arrays, if an <tt>AudioBuffer</tt> could be in use. (Web developers writing to <tt><tt>getChannelData</tt></tt> arrays should ensure that the <tt>AudioBuffer</tt> is not in use, under both proposals.) For implementers, the "freely share memory" proposal is probably less complex, although the "ArrayBuffer is neutered while we're playing it" problem must be solved trickily while this proposal solves it easily. (Of course, implementers who don't wish to rely on undefined C++ behavior, or who wish to use non-shared-memory hardware, will have a hard time with the "freely share memory" proposal (or be forced to make copies at inopportune times).) | ||