TPEPlatform/MediaRecorder: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(21 intermediate revisions by 2 users not shown)
Line 7: Line 7:
MediaRecorder takes media stream, which may come from web audio or media source or user media, as input. User may process all these media streams before recording. In mozCamera, there is nothing that user can do between capturerer and recorder, except canned video effect.
MediaRecorder takes media stream, which may come from web audio or media source or user media, as input. User may process all these media streams before recording. In mozCamera, there is nothing that user can do between capturerer and recorder, except canned video effect.


Regarding to architecture of Encoding framework, please visit [[TPEPlatform/MediaEncoderFramework|"MediaEncoder Framework"]].
To totally replace mozCamera, we need to support these three WebAPI
#MediaRecorder: replace video recording function in mozCameraControl
#ImageCapture: replace takePhoto function in mozCameraControl
#CameraControl: except the first two function, we still need an API to control camera for 3A/ Zoom/ etc..


==Track Encoder Milestone==
 
The following table reveals encode and associate container format we are going to support:
On B2G, there are two strategies
#B2G 1.3 on JerryBean: Base on JB::MediaCodec, we will support OmxVideoTrackEncoder and OmxAudioTrackEncoder to leverage HW encoder.
#B2G 1.3 on ICS: in this case, we still use VP8 and WebM.
 
==Tracking Issues==
META bugs for Encoding framework:
{| class="fullwidth-table sortable" style="width: 100%;"
{| class="fullwidth-table sortable" style="width: 100%;"
|-
|-
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" width="50px" | '''Type'''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" width="100px" | '''Bug No'''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Encoder'''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Description'''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Detailed'''
|-
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Container'''
| {{bug|803414}}
| MediaRecorder WebAPI
|-
|-
| rowspan=4 valign="top" | Audio
| {{bug|879688}}
| Opus
| MediaEncoder pipeline
| mono/ stereo
| Ogg
|-
|-
| Vobis
| {{bug|888177}}
| mono/ stereo
| ImageCapture WebAPI
| Ogg
|}
 
Main tracking issues
{| class="fullwidth-table sortable" style="width: 100%;"
|-
|-
| AAC_LC
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Bug No'''
| mono/ stereo/ 5.0/ 5.1
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Description'''
| MPEG-4
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Statue'''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Assigned'''
|-
|-
| PCM
| 16 bit LPCM
| Wave
|-
|-
| rowspan=2 valign="top" | Video
| {{bug|842243}}
| H.264
| Audio encoding path
| Baseline Profile
| FIXED
| MPEG-4
| Shelly
|-
|-
| VP8
| {{bug|919905}}
|  
| Refactor encoder architecture to support the video encoder module
| WebM 1.0
| FIXED
| Randy
|-
|-
| VP9
| <font color="red">BLOCKER: </font>{{bug|879669}}
|  
| Video encoding path
| WebM 2.0
| Open
|}
| Randy
List pros and cons for each audio/ video codec. We need more clear view to prioritize codec support sequence.
 
Requirements for a video codec, which is the way to we prioritize encoder support:
#Image quality
#Performance
#Power consumption
#Hardware support
#Licensing
 
==META Issues==
META bugs for Encoding framework:
{| class="fullwidth-table sortable" style="width: 100%;"
|-
|-
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" width="100px" | '''Bug No'''
| {{bug|891705}}
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Description'''
| WebM 1.0 container writer
| Open
| Randy
|-
|-
| {{bug|803414}}
| {{bug|883749}}
| WebAPI: Implementation MediaRecorder API
| Vorbis encoding
| Open
| Benjamin
|-
|-
| {{bug|879688}}
| {{bug|881840}}
| Framework: Implement MediaEncoder
| VP8 encoding
| Open
| Randy
|-
|-
| {{bug|842243}}
| {{bug|879668}}
| Framework: Implement Audio encoding path
| VideoOmxEncoder encoding
| Open
| Shelly, John
|-
|-
| {{bug|879669}}
| {{bug|920934}}
| Framework: Implement Video encoding path
| AudioOmxEncoder encoding
| Open
| Shelly, John
|-
|-
| {{bug|888177}}
| {{bug|891704}}
| WebAPI: Implement ImageCapture API
| MP4Writer container writer
| Open
| Alfredo
|}
|}


Line 104: Line 113:
|}
|}


==Codec/ Container support==
==Schedule==
Schedule is not well define yet. All the information list bellow is not confirmed.<br>
 
===B2G===
{| class="fullwidth-table sortable" style="width: 100%;"
{| class="fullwidth-table sortable" style="width: 100%;"
|-
|-
Line 112: Line 120:
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Milestone'''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Milestone'''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Target '''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Target '''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" align="center" width="100px"| '''Issue'''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" align="center" | '''Platform'''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Assignee'''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Assignee'''
|-
|-
|rowspan=3|Audio
|rowspan=5|Audio
| Opus/ OGG
| Opus
| Gecko 24
| Gecko 24
| {{bug|842243}}
| All
| Shelly
|-
| Vorbis
| Gecko 28
| All
| Benjamin
|-
| OmxAudio
| B2G 1.3
| B2G 1.3 with Android Jerry Bean
| Shelly
| Shelly
|-
|-
| LPCM/ Wav
| <strike>LPCM/ Wav</strike>
| TBD
| <strike>  </strike>
|  
| <strike>  </strike>
|  
| <strike>  </strike>
|-
|-
| <strike>MP3</strike>
| <strike>MP3</strike>
| <strike>Not defined</strike>
| <strike>   </strike>
| <strike>{{bug|884162}}</strike>
| <strike>   </strike>
| <strike>  </strike>
|-
|-
|rowspan=3|Video
|rowspan=3|Video
| H.264/ AAC/ MP4
| OmxVideo
| B2G 1.3
| B2G 1.3
| {{bug|879668}}
| B2G 1.3 with Android Jerry Bean
| Alfredo Yang
| Shelly Lin, John Lin
|-
|-
| VP8/ Vobis/ WebM
| VP8
| Gecko 26
| Gecko 28
| {{bug|881840}}
| All
| Randy Lin
| Randy Lin, Benjamin Chen
|-
|-
| VP9/ Opus/ WebM
| VP9
| Depend on decoding module
| TBD
| TBD
| TBD
|  
|  
|}
===Android===
{| class="fullwidth-table sortable" style="width: 100%;"
|-
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Milestone'''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Target '''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Issue'''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Assignee'''
|-
| Opus/ OGG
| 6/14
| {{bug|842243}}
| Shelly
|}
===Linux===
{| class="fullwidth-table sortable" style="width: 100%;"
|-
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Milestone'''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Target '''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Issue '''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Assignee'''
|-
| Opus/ OGG
| 6/14
| {{bug|842243}}
| Shelly
|}
===Windows===
{| class="fullwidth-table sortable" style="width: 100%;"
|-
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Milestone'''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Target Date'''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Issue No'''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Assignee'''
|-
| Opus/ OGG
| 6/14
| {{bug|842243}}
| Shelly
|}
===MacOS===
{| class="fullwidth-table sortable" style="width: 100%;"
|-
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Milestone'''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Target Date'''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Issue No'''
| style="background: none repeat scroll 0% 0% rgb(239, 239, 239);" | '''Assignee'''
|-
| Opus/ OGG
| 6/14
| {{bug|842243}}
| Shelly
|}
|}


Latest revision as of 06:40, 28 October 2013

Summary

We already have mozCamera for take photo or video recording. mozCamera itself is not part of any standard web spec, which means

  1. It works on Firefox platform only.
  2. Totally isolate in media world. There is no way to connect data stream from mozCamera with media processing module, such as WebAudio, or recording module, such as MediaRecorder.

MediaRecorder takes media stream, which may come from web audio or media source or user media, as input. User may process all these media streams before recording. In mozCamera, there is nothing that user can do between capturerer and recorder, except canned video effect.

To totally replace mozCamera, we need to support these three WebAPI

  1. MediaRecorder: replace video recording function in mozCameraControl
  2. ImageCapture: replace takePhoto function in mozCameraControl
  3. CameraControl: except the first two function, we still need an API to control camera for 3A/ Zoom/ etc..


On B2G, there are two strategies

  1. B2G 1.3 on JerryBean: Base on JB::MediaCodec, we will support OmxVideoTrackEncoder and OmxAudioTrackEncoder to leverage HW encoder.
  2. B2G 1.3 on ICS: in this case, we still use VP8 and WebM.

Tracking Issues

META bugs for Encoding framework:

Bug No Description
bug 803414 MediaRecorder WebAPI
bug 879688 MediaEncoder pipeline
bug 888177 ImageCapture WebAPI

Main tracking issues

Bug No Description Statue Assigned
bug 842243 Audio encoding path FIXED Shelly
bug 919905 Refactor encoder architecture to support the video encoder module FIXED Randy
BLOCKER: bug 879669 Video encoding path Open Randy
bug 891705 WebM 1.0 container writer Open Randy
bug 883749 Vorbis encoding Open Benjamin
bug 881840 VP8 encoding Open Randy
bug 879668 VideoOmxEncoder encoding Open Shelly, John
bug 920934 AudioOmxEncoder encoding Open Shelly, John
bug 891704 MP4Writer container writer Open Alfredo

Testing Plan

  1. Jason Smith: https://etherpad.mozilla.org/media-recording-testing
Bug No Description Statue Target Assigned
bug 889772 Audio Recording - Marionette Test Case Open Gecko 25 Randy Lin
TBD Video Recording - Marionette Test Case Open Gecko 26 Randy Lin

Schedule

Type Milestone Target Platform Assignee
Audio Opus Gecko 24 All Shelly
Vorbis Gecko 28 All Benjamin
OmxAudio B2G 1.3 B2G 1.3 with Android Jerry Bean Shelly
LPCM/ Wav
MP3
Video OmxVideo B2G 1.3 B2G 1.3 with Android Jerry Bean Shelly Lin, John Lin
VP8 Gecko 28 All Randy Lin, Benjamin Chen
VP9 TBD TBD

Reference

Retrieved from "https://wiki.mozilla.org/index.php?title=TPEPlatform/MediaRecorder&oldid=738907"