Confirmed users
770
edits
No edit summary |
|||
Line 20: | Line 20: | ||
[[File:FoxEye - Overview.png|800px]] | [[File:FoxEye - Overview.png|800px]] | ||
<br> | <br> | ||
Basically, the spirit of this project is four parts. The first part is extend the MediaStreamTrack to associate a | Basically, the spirit of this project is four parts. The first part is extend the MediaStreamTrack to associate a Worker. This part provide a way to do image processing job frame by frame. The second part is ImageBitmap extension. This part extended the ImageBitmap interface to allow JavaScript developer to read underlying data out and set an external data into an ImageBitmap in a set of supported color formats. The third part is OfflineMediaContext. It is for offline stream to render as fast as possible. The last part is WebImage, it is a hardware accelerated-able API on computer vision area. The Web developers can use it to combine high performance vision processing. | ||
<br> | <br> | ||
Thanks for the amazing asm.js and emscripten work, we also provide an asm.js version of OpenCV called OpenCV.js. The web developers can leverage the power of OpenCV in a simpler way. | Thanks for the amazing asm.js and emscripten work, we also provide an asm.js version of OpenCV called OpenCV.js. The web developers can leverage the power of OpenCV in a simpler way. | ||
Line 26: | Line 26: | ||
=Concept= | =Concept= | ||
==MediaStreamTrack with | ==MediaStreamTrack with Worker: == | ||
The new design is a simple and minimal change for current API. By extending MediaStreamTrack and adding | The new design is a simple and minimal change for current API. By extending MediaStreamTrack and adding Worker related API, we can let MediaStream be able to support video processing functionality through the script code in Worker. Below is the draft WebIDL codes. Please see [2] for the draft specification. | ||
<source lang="webidl"> | <source lang="webidl"> | ||
[Constructor(DOMString scriptURL)] | [Constructor(DOMString scriptURL)] | ||
partial interface MediaStreamTrack { | partial interface MediaStreamTrack { | ||
void addWorkerMonitor ( | void addWorkerMonitor (Worker worker); | ||
void removeWorkerMonitor ( | void removeWorkerMonitor (Worker worker); | ||
MediaStreamTrack addWorkerProcessor ( | MediaStreamTrack addWorkerProcessor (Worker worker); | ||
void removeWorkerProcessor (); | void removeWorkerProcessor (); | ||
}; | }; | ||
interface | partial interface WorkerGlobalScope { | ||
attribute EventHandler onvideoprocess; | attribute EventHandler onvideoprocess; | ||
}; | }; | ||
interface | interface VideoMonitorEvent : Event { | ||
readonly attribute DOMString trackId; | readonly attribute DOMString trackId; | ||
readonly attribute double playbackTime; | readonly attribute double playbackTime; | ||
readonly attribute ImageBitmap inputImageBitmap; | readonly attribute ImageBitmap inputImageBitmap; | ||
}; | }; | ||
interface VideoProcessorEvent : Event { | |||
readonly attribute DOMString trackId; | |||
readonly attribute double playbackTime; | |||
readonly attribute ImageBitmap inputImageBitmap; | |||
attribute ImageBitmap? outputImageBitmap = null; | |||
}; | |||
</source> | </source> | ||
<br> | <br> |