Accessibility/IA2 1.3

Summary

This page describes proposed changes to next IAccessible2 specification.

Proposal

IAccessible2_2 interface

Relations

Propose: a way to get relations by type

interface IAccessible2_2: public IAccessible
{
  /**
   * Return IEnumVariant of IAccessible2 objects for the given type.
   */
  [propget] HRESULT relationTargets
  (
    [in] BSTR type,
    [out, retval] IEnumVariant* relations
  );
};

See IA2 thread for discussion.

Object attribute

Propose: performant way to get object attribute value by name.

interface IAccessible2_2: public IAccessible
{
  [propget] HRESULT attribute
  (
    [in] BSTR name,
    [out, retval] BSTR* value
  );
};

See IA2 thread for discussion.

Caret offset and accessible containing the caret

Propose: performant way to get caret offset within direct hypertext accessible containing the caret.

interface IAccessible2_2 : public IAccessible2
{
  /**
   * Return hypertext accessible in subtree of the accessible and caret
   * offset within it.
   */
  [propget] HRESULT accessibleWithCaret
  (
    [out, retval] IUnknown **accessible,
    [out] long* caretOffset
  );
};

Note: AT should call the method on document accessible for performance reason.

See IA2 thread for discussion.

Document accessible

Propose: performant way to get a document accessible (IAccessibleDocument) the given accessible resides in.

interface IAccessible2_2 : public IAccessible2
{
  /**
   * Return document accessible (IAccessibleDocument) containing the accessible.
   */
  [propget] HRESULT document
  (
    [out, retval] IUnknown **accessible
  );
};

Reorder event details

Propose: performant way to update virtual buffers.

typedef struct IA2ChildrenChange
{
  bool isInserted;
  long startIndex;
  long endIndex;
};

interface IAccessible2_2: public IAccessible2
{
  [propget] HRESULT childrenChange
  (
    [in] long maxChanges, 
    [out, size_is(maxChanges), length_is(*nChanges)]
      IA2ChildrenChange **relations, 
    [out, retval] long *nChanges
  )
};

Explicit name

Propose: let AT to ignore fallback names, i.e. names calculated from subtree of the object

interface IAccessible2_2 : public IAccessible2
{
  /**
   * Return accessible name pointed by author.
   */
  [propget] HRESULT explicitName
  (
    [out, retval] BSTR* name
  );
};

See bug 637578 for original idea.

IAccessibleText2 interface

Text attributes

Propose: way to get range for list of text attributes.

interface IAccessibleText2 : public IAccessibleText
{
  /**
   * Return range (start and end offsets) and text attributes that corresponds
   * the given attributes filter (filter format: "attr1, attr2") at the given
   * offset.
   */
  [propget] HRESULT attributeRange
    (
     [in] long offset,
     [in] BSTR attributesFilter,
     [out] long *startOffset,
     [out] long *endOffset,	
     [out, retval] BSTR *textAttributes
    );
};

IAccessibleEditableText2 interface

Propose: clean up clipboard methods

Special offsets for selection

Introduce new kind of special offset to deal with selection. If both start and end offsets are equal to -3 then they point to start and end offset of active selection (i.e. start or end of selection has a caret). Make them applicable to all methods of the interface except insertText method.

Paste text

interface IAccessibleEditableText2: public IAccessibleEditableText
{
  /**
   * Replace the text between start and end offsets by data from clipboard of the given mime type. If mime type is missed then
   * application choose more appropriate one depending on insertion context.
   */
  HRESULT pasteText
    (
     [in] long startOffset,
     [in] long endOffset,
     [in] BSTR mimeType
    );
};

See 1st IA2 thread and 2nd IA2 thread for discussion.

IAccessibleHypertext2 interface

Hyperlinks from hypertext

Propose: performant way to iterate through hyper links

interface IAccessibleHypertext2 : public IAccessibleText
{
  /**
   * Return IEnumVariant of IAccessibleHyperlink objects.
   */
  [propget] HRESULT hyperlinks
  (
    [out, retval] IEnumVariant* hyperlinks
  );
};

See IA2 thread for discussion.

IAccessibleDocument interface

Anchor target

Propose: a way to get anchor target (#anchor URL) if any since AT may miss scrolling_start event due to various reasons.

interface IAccessibleDocument : public IAccessible2
{
  /**
   * Return anchor target if any and still valid.
   */
  [propget] HRESULT anchorTarget
  (
    [out, retval] IUnknown **accessible
  );
};

See bug 617544 for original propose.

New relation types

Propose: be compatible with ATK and for relation types symmetry.

/** This object is a parent of a target object. */
const WCHAR *const IA2_RELATION_NODE_PARENT_OF = L"nodeParentOf";

/** opposite to IA2_RELATION_MEMBER_OF (Pete's possible addition) */
const WCHAR *const IA2_RELATION_GROUPING_OBJECT_FOR  = L"groupingObjectFor";

/** opposite to IA2_RELATION_POPUP_INITIATOR_FOR (Pete's possible addition) */
const WCHAR *const IA2_RELATION_POPUP_INITIATOR_FOR  = L"popupInitiatorFor";

See IA2 thread for discussion.

IAccessibleMedia interface

Control video/audio

Propose: provide API to control video/audio.

interface IAccessibleMedia : public IUnknown
{
  /**
   * Allows to control the video/audio
   */
  HRESULT play();
  HRESULT pause();
  HRESULT stop();
  HRESULT un/mute();
  HRESULT get/setVolume();
  HRESULT get/setPosition();
  HRESULT get/setPlaybackRate();

  /**
   * Video/audio characteristics.
   */
  [propget] HRESULT duration(
     [out, retval] number);

  IAccessibleTextTrack[] getTextTracks();
  IAccessibleMediaTrack[] getAudioTracks();
  IAccessibleMediaTrack[] getVideoTracks();
}

interface IAccessibleTextTrack : public IUnknown
{
  HRESULT kind;
  HRESULT label;
  HRESULT language;
  HRESULT enabled;
  IAccessibleTextTrackCue[] getCues();
}

interface IAccessibleMediaTrack : public IUnknown
{
  HRESULT kind;
  HRESULT label;
  HRESULT language;
  HRESULT enabled;
}

Text cues

Propose: provide a way to handle text descriptions and other text cues.

AT server fires event (MEDIA_TEXT_CUE_CHANGE) when text cue of a track changes while video/audio is played. AT client handles it and gets current cue text by

interface IAccessibleTextTrackCue : public IUnknown
{
  /**
   * Return the current text.
   */
  [propget] HRESULT sourceText(
     [out, retval] BSTR text);
  [propget] HRESULT htmlText(
     [out, retval] BSTR text);

  HRESULT id;
  HRESULT startTime;
  HRESULT endTime;
  HRESULT pauseOnExit;
}

If next event is fired before screen reader finished reading then AT server stops media until AT client gets the current text description, after that AT server makes sure media is playing.

Text Descriptions and Subtitles for Video and Audio [Silvia description]

The <track> element in HTML5 exposes captions, subtitles and text descriptions to a HTML5 video player.

See the HTML5 track element for the specification.

The captions have a visual presentation, but are probably not relevant to be exposed to screen readers, because they repeat what is already available in the audio.

The subtitles have a visual presentation. They could be made available as an alternative to the video's native audio track.

The text descriptions have no visual presentation, but are supposed to be exposed to screen readers.

Text description cues (i.e. a single instance of a text description that is to be rendered from a certain start time to a certain stop time) exist in HTML5: http://www.whatwg.org/specs/web-apps/current-work/multipage/the-iframe-element.html#texttrackcue . They need to be exposed through IA2 to screen readers during video playback whenever they become active. Screen readers should, upon receiving a TextTrackCue start reading out the text. However, often the time that is available during the video playback for reading out the description text is not sufficient for the screenreader.

We therefore need a means for the screenreader to reach back to the browser and pause the video playback until it finishes reading.

HTML5 has JavaScript interfaces for this to happen, in particular the "pauseOnExit" attribute and the play() function, which will allow the kind of interaction that is required. This needs to be worked out for IA2 so we can have a fully automated way of pausing/resuming of a text description experience.

Other ideas [NOT a part of proposal]

Registry API

Note: IsWinEventHookInstalled may do all we need. See Mick's suggestion.

Propose: a way to enable set of features.

interface IAccessibleApplication2 : public IAccessibleApplication
{
  /**
   * Enable set of features. Return set of features that were enabled.
   */
  HRESULT requestFeature(
    [in] long features,
    [out] long supportedFeatures]
};

Initially proposed features:

1. Events
   1. ShowHideEvents - fire show/hide events
   2. ReorderEvent - fire reorder event
   3. TextEvents - fire text inserted/deleted/modified events
   4. TableEvents - fire table events
   5. HyperlinkEvents - fire hyper link events
   6. HypertextEvents - fire hyper text events
2. Interfaces (?)
   1. davidb: Alexander do you mean something like IAccessible2 or more granular? - alex: more granular
3. Accessible Tree (?)
   1. davidb: Alexander, what did you mean here?

Server implementation: it's up to the server to define set of base features that's always enabled (e.g. focus events); others are considered as optional and should be enabled on request. In order not to break existing AT, the server should maintain a whitelist of ATs for which the current support is preserved.

See IA2 list for discussion.