Accessibility/Caption Formats
< Back to main video accessibility page
From the requirements list we have a need for the following formats ("text codecs"):
- a closed caption format
- a closed subtitle format
- a textual audio description format
- potentially a karaoke format
- a metadata / semantic annotations format
- a transcript / script / lyrics format
These formats should specify:
- in format header: the type of text codec they represent
- in format header: the primary language
- in format header: default display mechanism
- in format header: open/closed by default
- in format body: temporal structure
- in format body: text & text styling
- in format body: allow outgoing hyperlinks
- in format body: allow naming of cue points / sections
Contents
Caption & Subtitle Formats
Icon for closed captions: 
Icon for subtitles (3-letter language): 
This is a (incomplete) list of existing caption and subtitle formats:
- W3C TimedText
- 3GPP Timed Text
- DVB Code-based subtitles
- DTV closed captioning
- QuickTime text tracks
- RealText markup
- MS SAMI .smi
- EBU subtitles
- DVD subtitles
- .sub MicroDVD subtitles
- .srt subrip subtitles
- SubStation Alpha .ssa, .ass
- .usf Universal Subtitle Format - mostly with Matroska
Previous comparative analysis:
- caption format comparison wikipedia
- Subtitle comparison
- Style comparison between different formats
- DOOM has examples of the different formats listed
- Timed Text analysis
Summary:
Many subtitle / caption formats come as just a sequence of triples: start-time, end-time, text (e.g. MicroDVD, SubRip). This is the rawest way of providing subtitles and it lacks further important information such as styling, title and language.
Some are more informative, e.g. Substation Alpha, USF, and SAMI, which introduce styling, metadata, and events with dynamic effects. SAMI and USF are in XML, which has been rejected as too talkative a langauge for creating subtitles.
TimedText has the most complex model, in particular for styling and layout.
Karaoke Formats
Icon for karaoke: http://www.istockphoto.com/file_closeup/object/5034281-karaoke-icon.php?id=5034281
- Kate
- Karaoke midi .KAR files ([http://www.geocities.com/takawadanl/ also]
- CD+G Karaoke CD format (graphics) ([http://www.powerkaraoke.com/help/karaokecdgcreator/source/import_lyrics_from_the_text_fi.htm also]
- MP3+G .CDG karaoke files (ripped graphics)
- Substation Alpha subtitle format supports karaoke through changing colours on words
- HTML+Time can also provide timed update of html elements
- Lyrics .lrc format
Comparison: Different Karaoke formats Different Karaoke formats
Summary:
Many karaoke formats are based around non-text formats, such as midi, or graphics. The simplest karaoke format is the lyrics format, which enables the inclusion of timing towards text. Advanced Substation Alpha provides an extension to captioning for karaoke, but nothing fancy. The only format that can carry text and animated images is Kate, which can even draw arbitrary shapes.
Textual Audio Description Formats
Icon for audio descriptions (sound): 
- SMIL format audio description, als HTML+Time
- SAMI, also on wikipedia
Interestingly, both use the file extension .smi. :-)
Summary:
There are not many formats that are explicitly targeted towards creating textual audio descriptions. These audio descriptions have to be synchronised with quiet sections in the video stream, which makes them somewhat different to captions - i.e. less sparse. SAMI has wide uptake as caption format in Korea.
Metadata / Semantic Annotations Formats
Icon for metadata / annotations: 
Dead Web 2.0 sites:
Analysis:
Summary:
There are no cases of RDF annotations attached to video in the wild yet. MPEG-7 as well as RDF for video are mostly research projects. Existing sites (or formerly existing sites) do user-contributed video annotations in proprietary formats, unexposed. MPEG-7 and AAF are too complex as a generic annotation language for multimedia.
Transcript / Script / Lyrics Formats
Icon for transcript: 
- LRC lyrics files
- example transcript (TBL)
- example transcript (SX)
- a11y transcripts
- example transcription service
Summary:
Transcripts are currently found mostly as web pages or as transcripts with time codes, which makes them caption-like. Where they are caption-like, there are no extra requirements. As for related text files - links are already easily presented on Web pages. The lyrics file consists of caption-like text after some core metadata fields, which are header-like information.
