Accessibility/IA2 1.3

From MozillaWiki
Jump to: navigation, search

Summary

This page describes proposed changes to next IAccessible2 specification.

Proposal

IAccessible2_2 interface

Relations

Propose: a way to get relations by type

Status: green

interface IAccessible2_2: public IAccessible
{
  /**
   * Return IEnumVariant of IAccessible2 objects for the given type.
   */
  [propget] HRESULT relationTargets
  (
    [in] BSTR type,
    [out, retval] IEnumVariant* relations
  );
};

See IA2 thread for discussion.

Object attribute

Propose: performant way to get object attribute value by name.

Status: green

interface IAccessible2_2: public IAccessible
{
  [propget] HRESULT attribute
  (
    [in] BSTR name,
    [out, retval] BSTR* value
  );
};

See IA2 thread for discussion.

Caret offset and accessible containing the caret

Propose: performant way to get caret offset within direct hypertext accessible containing the caret.

Status: green

interface IAccessible2_2 : public IAccessible2
{
  /**
   * Return hypertext accessible in subtree of the accessible and caret
   * offset within it.
   */
  [propget] HRESULT accessibleWithCaret
  (
    [out, retval] IUnknown **accessible,
    [out] long* caretOffset
  );
};

Note: AT should call the method on document accessible for performance reason.

See IA2 thread for discussion.

Document accessible

Propose: performant way to get a document accessible (IAccessibleDocument) the given accessible resides in.

Status: green

interface IAccessible2_2 : public IAccessible2
{
  /**
   * Return document accessible (IAccessibleDocument) containing the accessible.
   */
  [propget] HRESULT document
  (
    [out, retval] IUnknown **accessible
  );
};

Reorder event details [YELLOW]

Propose: performant way to update virtual buffers.

Status: yellow

Concerns: Jamie has concerns, wait for Jamie's feedback

typedef struct IA2ChildrenChange
{
  bool isInserted;
  long startIndex;
  long endIndex;
};

interface IAccessible2_2: public IAccessible2
{
  [propget] HRESULT childrenChange
  (
    [in] long maxChanges, 
    [out, size_is(maxChanges), length_is(*nChanges)]
      IA2ChildrenChange **relations, 
    [out, retval] long *nChanges
  )
};

Explicit name [YELLOW]

Propose: let AT to ignore fallback names, i.e. names are calculated from subtree of the object

Status: yellow

Concerns: Peter and Jamie would prefer flag approach, wait for final decision, see discussion

interface IAccessible2_2 : public IAccessible2
{
  /**
   * Return accessible name pointed by author.
   */
  [propget] HRESULT explicitName
  (
    [out, retval] BSTR* name
  );
};

See bug 637578 for original idea.

IAccessibleText2 interface

Text attributes

Propose: way to get range for list of text attributes.

Status: green

interface IAccessibleText2 : public IAccessibleText
{
  /**
   * Return range (start and end offsets) and text attributes that corresponds
   * the given attributes filter (filter format: "attr1, attr2") at the given
   * offset.
   */
  [propget] HRESULT attributeRange
    (
     [in] long offset,
     [in] BSTR attributesFilter,
     [out] long *startOffset,
     [out] long *endOffset,	
     [out, retval] BSTR *textAttributes
    );
};

IAccessibleEditableText2 interface [YELLOW]

Propose: clean up clipboard methods

Status: yellow

Concerns: everybody should read

Special offsets for selection

Introduce new kind of special offset to deal with selection. If both start and end offsets are equal to IA2_TEXT_OFFSET_SELECTION (value -3) then they point to start and end offset of active selection (i.e. start or end of selection has a caret). Make them applicable to all methods of the interface except insertText method.

Paste text

interface IAccessibleEditableText2: public IAccessibleEditableText
{
  /**
   * Replace the text between start and end offsets by data from clipboard of the given mime type. If mime type is missed then
   * application choose more appropriate one depending on insertion context.
   */
  HRESULT pasteText
    (
     [in] long startOffset,
     [in] long endOffset,
     [in] BSTR mimeType
    );
};

See 1st IA2 thread and 2nd IA2 thread for discussion.

IAccessibleHypertext2 interface

Hyperlinks from hypertext

Propose: performant way to iterate through hyper links

Status: green

interface IAccessibleHypertext2 : public IAccessibleText
{
  /**
   * Return IEnumVariant of IAccessibleHyperlink objects.
   */
  [propget] HRESULT hyperlinks
  (
    [out, retval] IEnumVariant* hyperlinks
  );
};

See IA2 thread for discussion.

IAccessibleDocument interface

Anchor target

Propose: a way to get anchor target (#anchor URL) if any since AT may miss scrolling_start event due to various reasons.

Status: green

interface IAccessibleDocument : public IAccessible2
{
  /**
   * Return anchor target if any and still valid.
   */
  [propget] HRESULT anchorTarget
  (
    [out, retval] IUnknown **accessible
  );
};

See bug 617544 for original propose.

New relation types

Propose: be compatible with ATK and for relation types symmetry.

Status: green

/** This object is a parent of a target object. */
const WCHAR *const IA2_RELATION_NODE_PARENT_OF = L"nodeParentOf";

/** opposite to IA2_RELATION_MEMBER_OF (Pete's possible addition) */
const WCHAR *const IA2_RELATION_GROUPING_OBJECT_FOR  = L"groupingObjectFor";

/** opposite to IA2_RELATION_POPUP_INITIATOR_FOR (Pete's possible addition) */
const WCHAR *const IA2_RELATION_POPUP_INITIATOR_FOR  = L"popupInitiatorFor";

See IA2 thread for discussion.

IAccessibleMedia interface [YELLOW]

Control video/audio

Propose: provide API to control video/audio.

Status: yellow

Concerns: Jamie has concerns. Get an agreement.

interface IAccessibleMedia : public IUnknown
{
  /**
   * Allows to control the video/audio
   */
  HRESULT play();
  HRESULT pause();
  HRESULT un/mute();
  HRESULT get/setVolume();
  HRESULT get/setPosition();
  HRESULT get/setPlaybackRate();

  /**
   * Video/audio characteristics.
   */
  [propget] HRESULT duration(
     [out, retval] number);

  IAccessibleTrack[] getTextTracks();
  IAccessibleTrack[] getAudioTracks();
  IAccessibleTrack[] getVideoTracks();
}

interface IAccessibleMediaTrack : public IUnknown
{
  HRESULT kind;
  HRESULT label;
  HRESULT language;
  HRESULT enabled;
}


Text cues

Propose: provide a way to handle text descriptions and other text cues.

Status: yellow

Concerns: Jamie has concerns. Get an agreement.

interface IAccessibleMedia : public IUnknown
{
  /**
   * Return the current cue text.
   */
  [propget] HRESULT cueText(
     [out, retval] BSTR text);
}

Algorithm:

  1. When the browser has a new cue available for a text track, then the browser fires a "MEDIA_TEXT_CUE_CHANGE" event at AT
  2. AT picks up the cue, which causes the browser to set a "pauseOnExit" flag to true on this cue; from here on, the AT and the video play asynchronously
    1. AT reads out cue and calls "cueText" property when it is finished, which causes the browser to set the "pauseOnExit" flag to false and unpauses the video if it is paused
    2. in parallel the browser plays video until the end of the cue; if the "pauseOnExit" flag is true when it reaches the end time of the cue, the video pauses, otherwise it continues playing
  3. repeat from 1)

[Track Cue interface]

interface IAccessibleTextTrackCue : public IUnknown
{
  /**
   * Return the current text.
   */
  [propget] HRESULT sourceText(
     [out, retval] BSTR text);
  [propget] HRESULT htmlText(
     [out, retval] BSTR text);

  HRESULT id;
  HRESULT startTime;
  HRESULT endTime;
}

Text Descriptions and Subtitles for Video and Audio [Silvia description]

The <track> element in HTML5 exposes captions, subtitles, text descriptions, and chapters for navigation to a HTML5 video player.

See the HTML5 track element for the specification.

The subtitles and captions have a visual presentation, but are probably not relevant to be exposed to screen readers, because they repeat what is already available in the audio. They may still be relevant to screen magnifiers though. They could be made available as an alternative to the video's native audio track.

The chapters represent navigation markers and are supposed to have a visual presentation, but it's not quite solved yet.

The text descriptions have no visual presentation, but are supposed to be exposed to screen readers.

Text description cues (i.e. a single instance of a text description that is to be rendered from a certain start time to a certain stop time) exist in HTML5: http://www.whatwg.org/specs/web-apps/current-work/multipage/the-iframe-element.html#texttrackcue . They need to be exposed through IA2 to screen readers during video playback whenever they become active. Screen readers should, upon receiving a TextTrackCue start reading out the text. However, often the time that is available during the video playback for reading out the description text is not sufficient for the screenreader.

We therefore need a means for the screenreader to reach back to the browser and pause the video playback until it finishes reading.

HTML5 has JavaScript interfaces for this to happen, in particular the "pauseOnExit" attribute and the play() function, which will allow the kind of interaction that is required. This needs to be worked out for IA2 so we can have a fully automated way of pausing/resuming of a text description experience.

Other ideas [NOT a part of proposal]

Registry API

Note: IsWinEventHookInstalled may do all we need. See Mick's suggestion.

Propose: a way to enable set of features.

interface IAccessibleApplication2 : public IAccessibleApplication
{
  /**
   * Enable set of features. Return set of features that were enabled.
   */
  HRESULT requestFeature(
    [in] long features,
    [out] long supportedFeatures]
};

Initially proposed features:

  1. Events
    1. ShowHideEvents - fire show/hide events
    2. ReorderEvent - fire reorder event
    3. TextEvents - fire text inserted/deleted/modified events
    4. TableEvents - fire table events
    5. HyperlinkEvents - fire hyper link events
    6. HypertextEvents - fire hyper text events
  2. Interfaces (?)
    1. davidb: Alexander do you mean something like IAccessible2 or more granular? - alex: more granular
  3. Accessible Tree (?)
    1. davidb: Alexander, what did you mean here?

Server implementation: it's up to the server to define set of base features that's always enabled (e.g. focus events); others are considered as optional and should be enabled on request. In order not to break existing AT, the server should maintain a whitelist of ATs for which the current support is preserved.

See IA2 list for discussion.