View source for Accessibility/Caption Formats

< [https://wiki.mozilla.org/Accessibility/Video_Accessibility Back to main video accessibility page]

From the requirements list we have a need for the following formats ("text codecs"):
* a closed caption format
* a closed subtitle format
* a textual audio description format
* potentially a karaoke format
* a metadata / semantic annotations format
* a transcript / script / lyrics format

These formats should specify:
* in format header: the type of text codec they represent
* in format header: the primary language
* in format header: default display mechanism
* in format header: open/closed by default
* in format body: temporal structure
* in format body: text & text styling
* in format body: allow outgoing hyperlinks
* in format body: allow naming of cue points / sections


== Caption & Subtitle Formats ==

Icon for closed captions: http://www.css.unt.edu/images/CC.jpg
Icon for subtitles (3-letter language): http://www.pearsonhighered.com/catalog/uploads/martingay_subtitles.jpg

This is a (incomplete) list of existing caption and subtitle formats:

* [http://www.w3.org/TR/2006/CR-ttaf1-dfxp-20061116/ W3C TimedText]
* [http://www.3gpp.org/FTP/Specs/html-info/26245.htm 3GPP Timed Text]
* [http://www.subtitling.com/downloads/DVB%20subtitling.pdf DVB Code-based subtitles]
* [http://www.avio-systems.com/dtvcc/home.html DTV closed captioning]
* [http://www.apple.com/quicktime/tutorials/texttracks.html QuickTime text tracks]
* [http://service.real.com/help/library/guides/realone/ProductionGuide/HTML/htmfiles/realtext.htm RealText markup]
* [http://msdn.microsoft.com/en-us/library/ms971327.aspx MS SAMI .smi]
* [http://www.ebu.ch/CMSimages/en/tec_doc_t3264_tcm6-10528.pdf EBU subtitles]
* [http://sam.zoy.org/writings/dvd/subtitles/ DVD subtitles]
* [http://en.wikipedia.org/wiki/MicroDVD .sub MicroDVD subtitles]
* [http://en.wikipedia.org/wiki/SubRip .srt subrip subtitles]
* [http://en.wikipedia.org/wiki/SubStation_Alpha SubStation Alpha .ssa, .ass]
* [http://en.wikipedia.org/wiki/Universal_Subtitle_Format .usf Universal Subtitle Format - mostly with Matroska]

Previous comparative analysis:

* [http://en.wikipedia.org/wiki/Subtitle_(captioning) caption format comparison wikipedia]
* [http://trac.annodex.net/wiki/CmmlSubtitles Subtitle comparison]
* [http://www.w3.org/AudioVideo/TT/Group/2003/drafts/Basic-Style-Att.html Style comparison between different formats]
* [http://www.doom9.org/index.html?/sub.htm DOOM has examples of the different formats listed]
* [https://wiki.mozilla.org/Accessibility/TimedText Timed Text analysis]

Summary:

Many subtitle / caption formats come as just a sequence of triples: start-time, end-time, text (e.g. MicroDVD, SubRip). This is the rawest way of providing subtitles and it lacks further important information such as styling, title and language.

Some are more informative, e.g. Substation Alpha, USF, and SAMI, which introduce styling, metadata, and events with dynamic effects. SAMI and USF are in XML, which has been rejected as too talkative a langauge for creating subtitles.

TimedText has the most complex model, in particular for styling and layout.

== Karaoke Formats ==

Icon for karaoke: http://www.istockphoto.com/file_closeup/object/5034281-karaoke-icon.php?id=5034281

* [http://wiki.xiph.org/index.php/OggKate Kate]
* [http://en.wikipedia.org/wiki/MIDI#MIDI_Karaoke_File_.28.KAR.29_Format Karaoke midi .KAR files ([http://www.geocities.com/takawadanl/ also]]
* [http://en.wikipedia.org/wiki/CD%2BG CD+G Karaoke CD format (graphics) ([http://www.powerkaraoke.com/help/karaokecdgcreator/source/import_lyrics_from_the_text_fi.htm also]]
* [http://en.wikipedia.org/wiki/MP3%2BG MP3+G .CDG karaoke files (ripped graphics)]
* [http://en.wikipedia.org/wiki/SubStation_Alpha Substation Alpha subtitle format supports karaoke through changing colours on words]
* [http://msdn.microsoft.com/en-us/library/ms533099(VS.85).aspx HTML+Time can also provide timed update of html elements]
* [http://en.wikipedia.org/wiki/Time_tag Lyrics .lrc format]

Comparison:
[http://www.tricerasoft.com/formats.html Different Karaoke formats]
[http://www.dbkaraoke.com/help/formats.php Different Karaoke formats]

Summary:

Many karaoke formats are based around non-text formats, such as midi, or graphics. The simplest karaoke format is the lyrics format, which enables the inclusion of timing towards text. Advanced Substation Alpha provides an extension to captioning for karaoke, but nothing fancy. The only format that can carry text and animated images is Kate, which can even draw arbitrary shapes.

== Textual Audio Description Formats ==

Icon for audio descriptions (sound): http://www.audiodescription.com.au/images/stories/audiodes.gif

* [http://ncam.wgbh.org/webaccess/magpie/magpie_help/audiodescription.html SMIL format audio description], als [http://msdn.microsoft.com/en-us/library/ms533099(VS.85).aspx HTML+Time]
* [http://msdn.microsoft.com/en-us/library/ms971327.aspx SAMI], also [http://en.wikipedia.org/wiki/SAMI on wikipedia]

Interestingly, both use the file extension .smi. :-)

Summary:

There are not many formats that are explicitly targeted towards creating textual audio descriptions. These audio descriptions have to be synchronised with quiet sections in the video stream, which makes them somewhat different to captions - i.e. less sparse. SAMI has wide uptake as caption format in Korea.

== Metadata / Semantic Annotations Formats ==

Icon for metadata / annotations: http://personal.optus.com.au/web/ShowBinary/SCSRepository/assets/images/personal/customerCentre/productHelp/tvHelp/DigitalTVHelp/icon_Subtitles.gif

* [http://en.wikipedia.org/wiki/Advanced_Authoring_Format AAF - Advanced Authoring Fomat, embedded in MXF - extensive media ontology]

* [http://www.joanneum.at/en/fb2/iis/products-solutions-services/semantic-video-annotation.html SVAT uses MPEG-7 annotation schemes for video - shot boundaries etc.]

* [http://au.youtube.com/t/annotations_about YouTube video annotations - speech bubbles, notes, spotlights with text] ([http://blog.makezine.com/archive/2008/06/youtube_has_onvideo_annot.html demo])

* [http://www.veotag.com/whatis/ veotag - clickable text]

* [http://sw.joanneum.at/rammx/ ramm.x - rdf and video]

Dead Web 2.0 sites:

* [http://www.omnisio.com/ Omnisio - speech bubbles and notes over the video plus slides synchronised]

* [http://www.masternewmedia.org/video_internet_television/video-annotation/online-video-annotation-takes-a-giant-leap-20070405.htm Mojito - speech bubbles, thought bubbles, text boxes, subtitles, etc.]

* [http://www.bubbleply.com/Default.aspx Bubbleply - speech bubbles in layers]


Analysis:

[http://www.w3.org/2005/Incubator/mmsem/XGR-vocabularies/ W3C format analysis]


Summary:

There are no cases of RDF annotations attached to video in the wild yet. MPEG-7 as well as RDF for video are mostly research projects. Existing sites (or formerly existing sites) do user-contributed video annotations in proprietary formats, unexposed. MPEG-7 and AAF are too complex as a generic annotation language for multimedia.

== Transcript / Script / Lyrics Formats ==

Icon for transcript: [[Image:Icon_transcript.jpg]]


* [http://en.wikipedia.org/wiki/Time_tag LRC lyrics files]
* [http://www.w3.org/2007/09/21-timbl-egov example transcript (TBL)]
* [http://h71028.www7.hp.com/enterprise/cache/110594-0-0-0-121.html example transcript (SX)]
* [http://www.allerdale.gov.uk/website-guide-and-help/accessibility/online-video-transcripts.aspx a11y transcripts]
* [http://www.trutranscripts.com/transcription-formats.htm example transcription service]

Summary:

Transcripts are currently found mostly as web pages or as transcripts with time codes, which makes them caption-like. Where they are caption-like, there are no extra requirements. As for related text files - links are already easily presented on Web pages. The lyrics file consists of caption-like text after some core metadata fields, which are header-like information.



< [https://wiki.mozilla.org/Accessibility/Video_Accessibility Back to main video accessibility page]