Accessibility/Captioning Work Plan

To do:

1. Separate into phase 1 and phase 2. I suggest that the work plan say in bold for each item, what the deliverable is, e.g. "Deliverable: test cases"
2. Add technology specifics -- any specifics we need
3. Add points for audio description
4. Describe the true complexity of the problem.

Background

There is both a container and a codec involved for each video format. For a given container, you need to have a defined (and supported by tools and other video players for network effects) mapping for muxing a giving captioning format into the container. So the video codec rules. The container depends on what's conventional for the video codec, and the choice of captioning format depends on what's conventional for the container. In theory, given a muxing rule, you can put any video codec and any captiong format in any container, but in practice, video codec tends to have a conventional native container, so the video codec dictates the container and then different containers have different conventional timed text formats and the timed text formats might not have muxing rules for non-native containers. Example: Ogg and MP4 are containers, whereas Theora and H.264 are codecs. Gstreamer and QuickTime are both timed media frameworks, which each can play various other container/codec combinations (?). Ogg, Theora and CMML are a natural match. MP4, H.264 and 3GPP TT are a natural match. While technically, you *could* define a way to put 3GPP TT inside Ogg, the disadvantage to doing this is blah.

Framework	Container	Codecs	Authoring tools	Natural captioning format
liboggplay	Ogg	Theora (video) Vorbis (audio)		CMML
	MP4	H.264		3GPP Timed Text
Flash plugin	.flv
GStreamer
QuickTime
DirectShow

Note: Subrip is external to the video container and can be used with any format. The main known disadvantage of this is blah, blah. It would make sense to use this if blah.

Work plan for Captioning

1. Determine which captioning format should be supported in Mozilla for the natively-supported Ogg video. This needs to take into account the extremely complex map of video formats and players today (see above).
2. Determine which subset of that format is the most crucial. This can save the Mozilla developers a good deal of work, because captioning formats are complex. Some of the complexity is necessary and some is not necessary for Mozilla suppoort
3. Work with HTML 5, web browser development and captioning communities to ensure that the solution will be accepted. We don't want different solutions in each browser. That would either mean one browser would need to redo their work, or that caption developers would have to deal with incompatible solutions in different browsers.
4. Explore the need to support the following features and ensure support when found necessary:
   1. social caption creation (This poses very different requirements than the idea of making video files intrinsically accessible. Hsivonen 09:06, 4 August 2008 (UTC)) aaronlev Henri also mentioned that potential legal issues could affect technical issues, but we aren't sure. It would be good if WGBH had some background to help understand this as well while devising a captioning solution.
   2. metadata indicating changes in captioning language for search and Braille. (Google seems to be doing better by ignoring author-entered language metadata. Is rendering foreign words into Braille strong enough an use case to justify the complexity of supporting this and authoring with this data. Hsivonen 09:06, 4 August 2008 (UTC))
   3. semantics and style, etc. (This seems like a pretty big departure from baseline established by TV captioning. Hsivonen 09:06, 4 August 2008 (UTC)) aaronlev I'm not sure -- there are some higher level things such as embedding of a musical note graphic to indicate music. I believe that captioning is moving toward expressing more complex background information.
5. Ensure captioning solution is compatible with current authoring and if possible, video conversion tools, so that current and future content can easily use the solution
6. Ensure the solution is compatible with both existing media repurposed for the Internet (i.e., originating in broadcast and cable TV environments and physical media like DVDs and theatrical motion pictures) and media originally developed for Internet distribution, including user-generated content.
7. Ensure the solition incorporates the expressed needs and preferences of Internet-based media users with sensory disabilities. (( Aaron: what are these? Can we express these up front? WGBH must already know this info, e.g. why would it change based on internet vs. brodcast? ))
8. Ensure all solutions, documentation and tests developed are friendly to open source contributors and clean of known IP conflicts
9. Participate in the relevant deliberations, meetings, standards development activities and proposed work products of HTML 5 WG.
10. Determine what can be done about supporting captioning when an external back end (gstreamer, QuickTime, DirectShow) is in use. (MP4 containers are the most likely external back end case, so 3GPP Timed Text is a potential format candidate in that case.)
11. Build a complete set of open-licensed documentation and test cases for developers and content creators. In general, reach out to developers implementing captioning solutions for the web and assure that issues of captioning (for deaf and hard-of-hearing people) and description (for blind and visually impaired people) are taken into account and are well-understood.
12. Test solutions and file bugs in databases for each browser to drive the necessary work. Attach relevant test cases and documentation. Make sure the developers know what to fix.

Work Plan for Audio Description

TBD

Success Criteria

1. A complete set of documentation and test cases for captioning and audio descriotion, without unnecessary IP restrictions, is available
2. Mozilla and should implement the proposed solution for both captioning and audio description, in a manner which maximizes usability. For example, there should be a consistent UI for turning captions on or off, no matter what the video format being used is.
3. Authoring tools are available which support the solutions
4. At least one mainstream source of video content on the web (e.g. wikimedia) has some content which supports the proposed solution for captioning and audio description