Accessibility/Captioning Work Plan: Difference between revisions

m
→‎Background: Fix summary
m (→‎Background: Fix summary)
 
(5 intermediate revisions by 2 users not shown)
Line 1: Line 1:
To do:
To do:
# Separate into phase 1 and phase 2. I suggest that the work plan say in bold for each item, what the deliverable is, e.g. "Deliverable: test cases"
# Separate into phase 1 and phase 2. How should we split? Captioning vs. audio description? Or, make technical choices in phase 1 and create test cases/docs etc. in phase 2?
# I suggest that the work plan say in bold for each item, what the deliverable is, e.g. "Deliverable: test cases"
# Add technology specifics -- any specifics we need
# Add technology specifics -- any specifics we need
# Add points for audio description
# Add points for audio description
# Describe the true complexity of the problem.  
# Describe the true complexity of the problem.  
# Question: is this really a grant to fix captioning in Ogg? If other formats already have captioning built-in, what else might need to be done? How does this relate to the greater HTML 5 effort?


= Background =
= Background =


There is both a container and a codec involved for each video format. For a given container, you need to have a defined (and supported by tools and other video players for network effects) mapping for muxing a giving captioning format into the container. So the video codec rules. The container depends on what's conventional for the video codec, and the choice of captioning format depends on what's conventional for the container. In theory, given a muxing rule, you can put any video codec and any captiong format in any container, but in practice, video codec tends to have a conventional native container, so the video codec dictates the container and then different containers have different conventional timed text formats and the timed text formats might not have muxing rules for non-native containers. Example: Ogg and MP4 are containers, whereas Theora and H.264 are codecs. Gstreamer and QuickTime are both timed media frameworks, which each can play various other container/codec combinations (?). Ogg, Theora and CMML are a natural match. MP4, H.264 and 3GPP TT are a natural match. While technically, you *could* define a way to put 3GPP TT inside Ogg, the disadvantage to doing this is blah.
There is both a container and a set of codecs involved for each video format. Typically there's a video codec and an audio codec. The captioning format can be thought of as a timed text codec.
 
For a given container, you need to have a defined (and supported by tools and other video players for network effects) mapping for muxing a data stream for a given codec into the container. Typically, such mapping don't exist for all codec/container combinations, and there are established combinations that work.
 
The format choice is driven by the video codec, so the container is then the container typically paired with the chosen video codec.
 
The choice of captioning format then depends on what's conventional for the container.  
 
In theory, given a muxing rule, you can put any video codec and any captiong format in any container, but in practice, video codec tends to have a conventional native container, so the video codec dictates the container and then different containers have different conventional timed text formats and the timed text formats might not have muxing rules for non-native containers.  
 
Gecko will embed an Ogg-specific playback framework called liboggplay. It only supports the Ogg container format.
 
The Flash plug-in effectively provides a specialized playback framework for .flv and .mp4 containers.
 
Typically, desktop environments come with a more general timed media playback framework. These frameworks can load extension libraries that enable support for various containers and codecs.
 
{| summary="Frameworks by desktop" border="1"
! Desktop
! Framework
|-
| Windows
| DirectShow
|-
| Mac OS X
| QuickTime
|-
| Gnome
| GStreamer
|-
| KDE
| Phonon
|}
 
 
 
 
Example: Ogg and MP4 are containers, whereas Theora and H.264 are codecs. Gstreamer and QuickTime are both timed media frameworks, which each can play various container/codec combinations. Ogg, Theora and CMML are a natural match. MP4, H.264 and 3GPP TT are a natural match. While technically, you 'could' define a way to put 3GPP TT inside Ogg, the disadvantage to doing this is that the result might not interoperate well with authoring tools and other player due to the combination being unusual.


{| summary="Map of video technology for the web" border="1"
{| summary="Map of video technology for the web" border="1"
! Framework
! Container
! Container
! Codecs
! Codecs
Line 16: Line 53:
! Natural captioning format
! Natural captioning format
|-
|-
| liboggplay
| Ogg
| Ogg
|
|
Line 24: Line 60:
| CMML
| CMML
|-
|-
|
| MP4
| MP4
| H.264
| H.264
Line 30: Line 65:
| 3GPP Timed Text
| 3GPP Timed Text
|-
|-
| Flash plugin
| .flv
| .flv
|
|
|
|-
| GStreamer
|
|
|
|
|-
| QuickTime
|
|
|
|
|
|
|
|-
|-
| DirectShow
| WMV
|
| WMV9/VC-1
|
|
|
|
|
|}
|}


Note: Subrip is external to the video container and can be used with any format. The main known disadvantage of this is blah, blah. It would make sense to use this if blah.
Note: Subrip is external to the video container and can be used with any format. The main known disadvantage of this is blah, blah. It would make sense to use this if blah.
254

edits