MozillaWiki

Accessibility/Captioning Work Plan: Difference between revisions

Page Discussion

Accessibility/Captioning Work Plan (view source)

Revision as of 14:56, 4 August 2008

2,058 bytes added ,  4 August 2008
m
→‎Background: Fix summary
m (→‎Background: Fix summary)
 
(10 intermediate revisions by 2 users not shown)
Line 1: Line 1:
To do:
To do:
# Separate into phase 1 and phase 2. I suggest that the work plan say in bold for each item, what the deliverable is, e.g. "Deliverable: test cases"
# Separate into phase 1 and phase 2. How should we split? Captioning vs. audio description? Or, make technical choices in phase 1 and create test cases/docs etc. in phase 2?
# I suggest that the work plan say in bold for each item, what the deliverable is, e.g. "Deliverable: test cases"
# Add technology specifics -- any specifics we need
# Add technology specifics -- any specifics we need
# Add points for audio description
# Add points for audio description
# Describe the true complexity of the problem.  
# Describe the true complexity of the problem.  
# Question: is this really a grant to fix captioning in Ogg? If other formats already have captioning built-in, what else might need to be done? How does this relate to the greater HTML 5 effort?


= Background =
There is both a container and a set of codecs involved for each video format. Typically there's a video codec and an audio codec. The captioning format can be thought of as a timed text codec.
For a given container, you need to have a defined (and supported by tools and other video players for network effects) mapping for muxing a data stream for a given codec into the container. Typically, such mapping don't exist for all codec/container combinations, and there are established combinations that work.
The format choice is driven by the video codec, so the container is then the container typically paired with the chosen video codec.
The choice of captioning format then depends on what's conventional for the container.
In theory, given a muxing rule, you can put any video codec and any captiong format in any container, but in practice, video codec tends to have a conventional native container, so the video codec dictates the container and then different containers have different conventional timed text formats and the timed text formats might not have muxing rules for non-native containers.
Gecko will embed an Ogg-specific playback framework called liboggplay. It only supports the Ogg container format.
The Flash plug-in effectively provides a specialized playback framework for .flv and .mp4 containers.
Typically, desktop environments come with a more general timed media playback framework. These frameworks can load extension libraries that enable support for various containers and codecs.
{| summary="Frameworks by desktop" border="1"
! Desktop
! Framework
|-
| Windows
| DirectShow
|-
| Mac OS X
| QuickTime
|-
| Gnome
| GStreamer
|-
| KDE
| Phonon
|}
Example: Ogg and MP4 are containers, whereas Theora and H.264 are codecs. Gstreamer and QuickTime are both timed media frameworks, which each can play various container/codec combinations. Ogg, Theora and CMML are a natural match. MP4, H.264 and 3GPP TT are a natural match. While technically, you 'could' define a way to put 3GPP TT inside Ogg, the disadvantage to doing this is that the result might not interoperate well with authoring tools and other player due to the combination being unusual.
{| summary="Map of video technology for the web" border="1"
! Container
! Codecs
! Authoring tools
! Natural captioning format
|-
| Ogg
|
* Theora (video)
* Vorbis (audio)
|
| CMML
|-
| MP4
| H.264
|
| 3GPP Timed Text
|-
| .flv
|
|
|
|-
| WMV
| WMV9/VC-1
|
|
|}
Note: Subrip is external to the video container and can be used with any format. The main known disadvantage of this is blah, blah. It would make sense to use this if blah.


= Work plan for Captioning =
= Work plan for Captioning =


# Determine which captioning format should be supported in Mozilla for the natively-supported Ogg video. This needs to take into account the extremely complex map of video formats and players today. There is both a container and a codec involved for each video format. For a given container, you need to have a defined (and supported by tools and other video players for network effects) mapping for muxing a giving captioning format into the container. So the video codec rules. The container depends on what's conventional for the video codec, and the choice of captioning format depends on what's conventional for the container. In theory, given a muxing rule, you can put any video codec and any captiong format in any container, but in practice, video codec tends to have a conventional native container, so the video codec dictates the container and then different containers have different conventional timed text formats and the timed text formats might not have muxing rules for non-native containers. Example: Ogg and MP4 are containers, whereas Theora and H.264 are codecs. Gstreamer and QuickTime are both timed media frameworks, which each can play various other container/codec combinations (?). Ogg, Theora and CMML are a natural match. MP4, H.264 and 3GPP TT are a natural match. While technically, you *could* define a way to put 3GPP TT inside Ogg, the disadvantage to doing this is blah.
# Determine which captioning format should be supported in Mozilla for the natively-supported Ogg video. This needs to take into account the extremely complex map of video formats and players today (see above).
# Determine which subset of that format is the most crucial. This can save the Mozilla developers a good deal of work, because captioning formats are complex. Some of the complexity is necessary and some is not necessary for Mozilla suppoort
# Determine which subset of that format is the most crucial. This can save the Mozilla developers a good deal of work, because captioning formats are complex. Some of the complexity is necessary and some is not necessary for Mozilla suppoort
# Work with HTML 5, web browser development and captioning communities to ensure that the solution will be accepted.  We don't want different solutions in each browser. That would either mean one browser would need to redo their work, or that caption developers would have to deal with incompatible solutions in different browsers.
# Work with HTML 5, web browser development and captioning communities to ensure that the solution will be accepted.  We don't want different solutions in each browser. That would either mean one browser would need to redo their work, or that caption developers would have to deal with incompatible solutions in different browsers.
Line 29: Line 102:


= Success Criteria =
= Success Criteria =
# A complete set of documentation and test cases without unnecessary IP restrictions is available
# A complete set of documentation and test cases for captioning and audio descriotion, without unnecessary IP restrictions, is available
# Mozilla and other browser implementations implement the proposed solution for captioning and audio description
# Mozilla and should implement the proposed solution for both captioning and audio description, in a manner which maximizes usability. For example, there should be a consistent UI for turning captions on or off, no matter what the video format being used is.
# Authoring tools are supported
# Authoring tools are available which support the solutions
# At least one mainstream source of video content on the web (e.g. wikimedia) supports the proposed solution
# At least one mainstream source of video content on the web (e.g. wikimedia) has some content which supports the proposed solution for captioning and audio description
Hsivonen
254

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/102723...102741"