TPEPlatform/MediaRecorder

Summary

We already have mozCamera for take photo or video recording. mozCamera itself is not part of any standard web spec, which means

1. It works on Firefox platform only.
2. Totally isolate in media world. There is no way to connect data stream from mozCamera with media processing module, such as WebAudio, or recording module, such as MediaRecorder.

MediaRecorder takes media stream, which may come from web audio or media source or user media, as input. User may process all these media streams before recording. In mozCamera, there is nothing that user can do between capturerer and recorder, except canned video effect.

The following table reveals encode and associate container format we are going to support:

Type	Encoder	Detailed	Container
Audio	Opus	mono/ stereo	Ogg
	Vobis	mono/ stereo	Ogg
	AAC	mono/ stereo/ 5.0/ 5.1	MPEG-4
	PCM	16 bit LPCM	Wave
Video	H.264	Baseline Profile	MPEG-4
	VP8		WebM

Encoder framework

Encoding framework META issues:

• bug 879688: Implement MediaEncoder
• bug 842243: Implement Audio encoding path
• bug 879669: Implement Video encoding path
• bug 888177: Implement ImageCapture API

Dependency among modules

• GetUserMedia
  • MediaRecodering
    • Audio encoding
    • VideoEncoding
  • ImageCapture
    • Effort is still unknow

The class diagram of encoder framework

The sequence diagram of receiving input source from MediaStreamGraph, and outputting final container data from MediaEncoder to MediaRecorder

VP9 or H.264

List pros and cons for each audio/ video codec. We need more clear view to prioritize codec support sequence.

H.264

• Pros: Sharable/ HW support

VP9

• Pros: network bandwidth

Codec/ Container support

B2G

Milestone	Target	Issue	Assignee
(Audio) Opus/ OGG	Gecko 24	bug 842243	Shelly
(Video) H.264/ AAC/ MP4	Gecko 28	bug 879668
(Video) VP8/ Vobis/ WebM	Gecko 27	bug 881840
(Video) VP9/ Opus/ WebM	??/??	TBD
(Audio) MP3	Not defined	bug 884162

Android

Milestone	Target	Issue	Assignee
Opus/ OGG	6/14	bug 842243	Shelly

Linux

Milestone	Target	Issue	Assignee
Opus/ OGG	6/14	bug 842243	Shelly

Windows

Milestone	Target Date	Issue No	Assignee
Opus/ OGG	6/14	bug 842243	Shelly

Mac

Milestone	Target Date	Issue No	Assignee
Opus/ OGG	6/14	bug 842243	Shelly

Reference

• Meeting Minutes

https://taiwan.etherpad.mozilla.org/247

• Spec: GetUserMedia

http://dev.w3.org/2011/webrtc/editor/getusermedia.html

• Spec: MediaStream Recording

http://www.w3.org/TR/2013/WD-mediastream-recording-20130205/

• Spec: Image Capture

API for taking photo shot

https://dvcs.w3.org/hg/dap/raw-file/tip/media-stream-capture/ImageCapture.html

• Spec: HTML media capture (only supports simple capture functionalities)

http://www.w3.org/TR/html-media-capture/

bug 741393

• Android JB encoder matrix

http://developer.android.com/guide/appendix/media-formats.html

• iOS encorder matrix

http://developer.apple.com/library/ios/#Documentation/AudioVideo/Conceptual/MultimediaPG/UsingAudio/UsingAudio.html

• Firefox playback format

https://developer.mozilla.org/en-US/docs/HTML/Supported_media_formats

• VP9

http://www.youtube.com/watch?v=K6JshvblIcM&noredirect=1