MozillaWiki

Accessibility/IA2 1.3: Difference between revisions

Page Discussion

Accessibility/IA2 1.3 (view source)

Revision as of 17:31, 21 June 2011

7,983 bytes added ,  21 June 2011
→‎Special offsets for selection
 
(33 intermediate revisions by 3 users not shown)
Line 4: Line 4:
=Proposal=
=Proposal=


==Registry API==
==IAccessible2_2 interface==


Propose: a way to enable set of features.
===Relations===


<pre>
''Propose'': a way to get relations by type
interface IAccessibleApplication2 : public IAccessibleApplication
{
  /**
  * Enable set of features. Return set of features that were enabled.
  */
  HRESULT requestFeature(
    [in] long features,
    [out] long supportedFeatures]
};
</pre>


Initially proposed features:
''Status'': <span style="background-color: Chartreuse;">green</span>
# Events
## ShowHideEvents - fire show/hide events
## ReorderEvent - fire reorder event
## TextEvents - fire text inserted/deleted/modified events
## TableEvents - fire table events
## HyperlinkEvents - fire hyper link events
## HypertextEvents - fire hyper text events
# Interfaces?
# Accessible Tree?
 
Server implementation: it's up to the server to define set of base features that's always enabled; others are considered as optional and should be enabled on request. The server is allowed to whitelist ATs the server is officially compatible with to keep backward compatibility.
 
See [https://lists.linux-foundation.org/pipermail/accessibility-ia2/2010-June/001186.html IA2 list] for discussion.
 
==Relations==
 
Propose: a way to get relations by type


<pre>
<pre>
Line 55: Line 28:
See [https://lists.linux-foundation.org/pipermail/accessibility-ia2/2010-April/001071.html IA2 thread] for discussion.
See [https://lists.linux-foundation.org/pipermail/accessibility-ia2/2010-April/001071.html IA2 thread] for discussion.


==Object attribute==
===Object attribute===
 
''Propose'': performant way to get object attribute value by name.


Propose: performant way to get object attribute value by name.
''Status'': <span style="background-color: Chartreuse;">green</span>


<pre>
<pre>
Line 72: Line 47:
See [https://lists.linux-foundation.org/pipermail/accessibility-ia2/2009-October/000980.html IA2 thread] for discussion.
See [https://lists.linux-foundation.org/pipermail/accessibility-ia2/2009-October/000980.html IA2 thread] for discussion.


==Text attributes==
===Caret offset and accessible containing the caret===
 
''Propose'': performant way to get caret offset within direct hypertext accessible containing the caret.
 
''Status'': <span style="background-color: Chartreuse;">green</span>
 
<pre>
interface IAccessible2_2 : public IAccessible2
{
  /**
  * Return hypertext accessible in subtree of the accessible and caret
  * offset within it.
  */
  [propget] HRESULT accessibleWithCaret
  (
    [out, retval] IUnknown **accessible,
    [out] long* caretOffset
  );
};
</pre>
 
Note: AT should call the method on document accessible for performance reason.
 
See [https://lists.linux-foundation.org/pipermail/accessibility-ia2/2010-December/001299.html IA2 thread] for discussion.
 
===Document accessible===
 
''Propose'': performant way to get a document accessible (IAccessibleDocument) the given accessible resides in.
 
''Status'': <span style="background-color: Chartreuse;">green</span>
 
<pre>
interface IAccessible2_2 : public IAccessible2
{
  /**
  * Return document accessible (IAccessibleDocument) containing the accessible.
  */
  [propget] HRESULT document
  (
    [out, retval] IUnknown **accessible
  );
};
</pre>
 
===Reorder event details [YELLOW]===
 
''Propose'': performant way to update virtual buffers.
 
''Status'': <span style="background-color: yellow;">yellow</span>
 
''Concerns'': Jamie has [https://lists.linux-foundation.org/pipermail/accessibility-ia2/2011-June/001340.html concerns], wait for Jamie's feedback
 
<pre>
typedef struct IA2ChildrenChange
{
  bool isInserted;
  long startIndex;
  long endIndex;
};
 
interface IAccessible2_2: public IAccessible2
{
  [propget] HRESULT childrenChange
  (
    [in] long maxChanges,
    [out, size_is(maxChanges), length_is(*nChanges)]
      IA2ChildrenChange **relations,
    [out, retval] long *nChanges
  )
};
</pre>
 
===Explicit name [YELLOW]===
 
''Propose'': let AT to ignore fallback names, i.e. names are calculated from subtree of the object
 
''Status'': <span style="background-color: yellow;">yellow</span>
 
''Concerns'': Peter and Jamie would prefer flag approach, wait for final decision, see [https://lists.linux-foundation.org/pipermail/accessibility-ia2/2011-June/001338.html discussion]
 
<pre>
interface IAccessible2_2 : public IAccessible2
{
  /**
  * Return accessible name pointed by author.
  */
  [propget] HRESULT explicitName
  (
    [out, retval] BSTR* name
  );
};
</pre>
 
See [https://bugzilla.mozilla.org/show_bug.cgi?id=637578 bug 637578] for original idea.


Propose: way to get range for list of text attributes.
==IAccessibleText2 interface==
 
===Text attributes===
 
''Propose'': way to get range for list of text attributes.
 
''Status'': <span style="background-color: Chartreuse;">green</span>


<pre>
<pre>
Line 95: Line 169:
</pre>
</pre>


==IAccessibleEditableText2 interface [YELLOW]==


==Hyperlinks from hypertext==
''Propose'': clean up clipboard methods


Propose: performant way to iterate through hyper links
''Status'': <span style="background-color: yellow;">yellow</span>
 
''Concerns'': everybody should read
 
===Special offsets for selection===
 
Introduce new kind of [http://accessibility.linuxfoundation.org/a11yspecs/ia2/docs/html/_generalinfo.html#_specialOffsets special offset] to deal with selection. If both start and end offsets are equal to IA2_TEXT_OFFSET_SELECTION (value -3) then they point to start and end offset of active selection (i.e. start or end of selection has a caret). Make them applicable to all methods of the interface except insertText method.
 
===Paste text===
 
<pre>
interface IAccessibleEditableText2: public IAccessibleEditableText
{
  /**
  * Replace the text between start and end offsets by data from clipboard of the given mime type. If mime type is missed then
  * application choose more appropriate one depending on insertion context.
  */
  HRESULT pasteText
    (
    [in] long startOffset,
    [in] long endOffset,
    [in] BSTR mimeType
    );
};
</pre>
 
See [https://lists.linux-foundation.org/pipermail/accessibility-ia2/2010-September/001219.html 1st IA2 thread] and [https://lists.linux-foundation.org/pipermail/accessibility-ia2/2010-October/001227.html 2nd IA2 thread] for discussion.
 
==IAccessibleHypertext2 interface==
 
===Hyperlinks from hypertext===
 
''Propose'': performant way to iterate through hyper links
 
''Status'': <span style="background-color: Chartreuse;">green</span>


<pre>
<pre>
Line 116: Line 225:
See [https://lists.linux-foundation.org/pipermail/accessibility-ia2/2010-April/001106.html IA2 thread] for discussion.
See [https://lists.linux-foundation.org/pipermail/accessibility-ia2/2010-April/001106.html IA2 thread] for discussion.


==Caret offset and accessible containing the caret==
==IAccessibleDocument interface==
 
===Anchor target===
 
''Propose'': a way to get anchor target (#anchor URL) if any since AT may miss scrolling_start event due to various reasons.


Propose: performant way to get caret offset within direct hypertext accessible containing the caret.
''Status'': <span style="background-color: Chartreuse;">green</span>


<pre>
<pre>
interface IAccessible2_2 : public IAccessible2
interface IAccessibleDocument : public IAccessible2
{
{
   /**
   /**
   * Return hypertext accessible in subtree of the accessible and caret
   * Return anchor target if any and still valid.
  * offset within it.
   */
   */
   [propget] HRESULT accessibleWithCaret
   [propget] HRESULT anchorTarget
   (
   (
     [out, retval] IUnknown **accessible,
     [out, retval] IUnknown **accessible
    [out] long* caretOffset
   );
   );
};
};
</pre>
</pre>


Note: AT should call the method on document accessible for performance reason.
See [https://bugzilla.mozilla.org/show_bug.cgi?id=617544 bug 617544] for original propose.
 
==New relation types==
 
''Propose'': be compatible with ATK and for relation types symmetry.
 
''Status'': <span style="background-color: Chartreuse;">green</span>
 
<pre>
/** This object is a parent of a target object. */
const WCHAR *const IA2_RELATION_NODE_PARENT_OF = L"nodeParentOf";
 
/** opposite to IA2_RELATION_MEMBER_OF (Pete's possible addition) */
const WCHAR *const IA2_RELATION_GROUPING_OBJECT_FOR  = L"groupingObjectFor";
 
/** opposite to IA2_RELATION_POPUP_INITIATOR_FOR (Pete's possible addition) */
const WCHAR *const IA2_RELATION_POPUP_INITIATOR_FOR  = L"popupInitiatorFor";
</pre>
 
See [https://lists.linux-foundation.org/pipermail/accessibility-ia2/2010-February/001016.html IA2 thread] for discussion.
 
==IAccessibleMedia interface [YELLOW]==
 
===Control video/audio===


See [https://lists.linux-foundation.org/pipermail/accessibility-ia2/2010-December/001299.html IA2 thread] for discussion.
''Propose'': provide API to control video/audio.


==Anchor target==
''Status'': <span style="background-color: yellow;">yellow</span>


Propose: a way to get anchor target (#anchor URL) if any since AT miss scrolling_end event due to various reasons.
''Concerns'': Jamie has [https://lists.linux-foundation.org/pipermail/accessibility-ia2/2011-June/001353.html concerns]. Get an agreement.


<pre>
<pre>
interface IAccessibleDocument : public IAccessible2
interface IAccessibleMedia : public IUnknown
{
{
   /**
   /**
   * Return anchor target if any and still valid.
   * Allows to control the video/audio
  */
  HRESULT play();
  HRESULT pause();
  HRESULT un/mute();
  HRESULT get/setVolume();
  HRESULT get/setPosition();
  HRESULT get/setPlaybackRate();
 
  /**
  * Video/audio characteristics.
   */
   */
   [propget] HRESULT anchorTarget
   [propget] HRESULT duration(
  (
    [out, retval] number);
    [out, retval] IUnknown **accessible
 
   );
  IAccessibleTrack[] getTextTracks();
};
  IAccessibleTrack[] getAudioTracks();
   IAccessibleTrack[] getVideoTracks();
}
 
interface IAccessibleMediaTrack : public IUnknown
{
  HRESULT kind;
  HRESULT label;
  HRESULT language;
  HRESULT enabled;
}
</pre>
</pre>


See [https://bugzilla.mozilla.org/show_bug.cgi?id=617544 bug 617544] for original propose.


==Document accessible==
===Text cues===
''Propose'': provide a way to handle text descriptions and other text cues.
 
''Status'': <span style="background-color: yellow;">yellow</span>


Propose: performant way to get a document accessible (IAccessibleDocument) the given accessible contained in.
''Concerns'': Jamie has [https://lists.linux-foundation.org/pipermail/accessibility-ia2/2011-June/001353.html concerns]. Get an agreement.


<pre>
<pre>
interface IAccessible2_2 : public IAccessible2
interface IAccessibleMedia : public IUnknown
{
{
   /**
   /**
   * Return document accessible (IAccessibleDocument) containing the accessible.
   * Return the current cue text.
   */
   */
   [propget] HRESULT document
   [propget] HRESULT cueText(
  (
    [out, retval] BSTR text);
    [out, retval] IUnknown **accessible
}
  );
};
</pre>
</pre>


==New relation types==
Algorithm:


Propose: be compatible with ATK and for relation types symmetry.
<ol>
<li> When the browser has a new cue available for a text track, then the browser fires a "MEDIA_TEXT_CUE_CHANGE" event at AT</li>
<li> AT picks up the cue, which causes the browser to set a "pauseOnExit" flag to true on this cue; from here on, the AT and the video play asynchronously</li>
<ol>
<li> AT reads out cue and calls "cueText" property when it is finished, which causes the browser to set the "pauseOnExit" flag to false and unpauses the video if it is paused</li>
<li> in parallel the browser plays video until the end of the cue; if the "pauseOnExit" flag is true when it reaches the end time of the cue, the video pauses, otherwise it continues playing</li>
</ol>
<li>repeat from 1)</li>
</ol>


[Track Cue interface]
<pre>
<pre>
/** This object is a parent of a target object. */
interface IAccessibleTextTrackCue : public IUnknown
const WCHAR *const IA2_RELATION_NODE_PARENT_OF = L"nodeParentOf";
{
  /**
  * Return the current text.
  */
  [propget] HRESULT sourceText(
    [out, retval] BSTR text);
  [propget] HRESULT htmlText(
    [out, retval] BSTR text);
 
  HRESULT id;
  HRESULT startTime;
  HRESULT endTime;
}
</pre>
 
===Text Descriptions and Subtitles for Video and Audio [Silvia description]===
 
The &lt;track> element in HTML5 exposes captions, subtitles, text descriptions, and chapters for navigation to a HTML5 video player.
 
See the [http://www.w3.org/TR/html5/the-iframe-element.html#the-track-element HTML5 track element] for the specification.
 
The subtitles and  captions have a visual presentation, but are probably not relevant to be exposed to screen readers, because they repeat what is already available in the audio. They may still be relevant to screen magnifiers though. They could be made available as an alternative to the video's native audio track.
 
The chapters represent navigation markers and are supposed to have a visual presentation, but it's not quite solved yet.
 
The text descriptions have no visual presentation, but are supposed to be exposed to screen readers.
 
Text description cues (i.e. a single instance of a text description that is to be rendered from a certain start time to a certain stop time) exist in HTML5: http://www.whatwg.org/specs/web-apps/current-work/multipage/the-iframe-element.html#texttrackcue . They need to be exposed through IA2 to screen readers during video playback whenever they become active. Screen readers should, upon receiving a TextTrackCue start reading out the text. However, often the time that is available during the video playback for reading out the description text is not sufficient for the screenreader.
 
We therefore need a means for the screenreader to reach back to the browser and pause the video playback until it finishes reading.
 
HTML5 has JavaScript interfaces for this to happen, in particular the "pauseOnExit" attribute and the play() function, which will allow the kind of interaction that is required. This needs to be worked out for IA2 so we can have a fully automated way of pausing/resuming of a text description experience.
 
=Other ideas [NOT a part of proposal]=
 
==Registry API==
 
Note: IsWinEventHookInstalled may do all we need. See [https://lists.linux-foundation.org/pipermail/accessibility-ia2/2010-July/001209.html Mick's suggestion].


/** opposite to IA2_RELATION_MEMBER_OF (Pete's possible addition) */
Propose: a way to enable set of features.
const WCHAR *const IA2_RELATION_GROUPING_OBJECT_FOR  = L"groupingObjectFor";


/** opposite to IA2_RELATION_POPUP_INITIATOR_FOR (Pete's possible addition) */
<pre>
const WCHAR *const IA2_RELATION_POPUP_INITIATOR_FOR  = L"popupInitiatorFor";
interface IAccessibleApplication2 : public IAccessibleApplication
{
  /**
  * Enable set of features. Return set of features that were enabled.
  */
  HRESULT requestFeature(
    [in] long features,
    [out] long supportedFeatures]
};
</pre>
</pre>


See [https://lists.linux-foundation.org/pipermail/accessibility-ia2/2010-February/001016.html IA2 thread] for discussion.
Initially proposed features:
# Events
## ShowHideEvents - fire show/hide events
## ReorderEvent - fire reorder event
## TextEvents - fire text inserted/deleted/modified events
## TableEvents - fire table events
## HyperlinkEvents - fire hyper link events
## HypertextEvents - fire hyper text events
# Interfaces (?)
## davidb: Alexander do you mean something like IAccessible2 or more granular? - alex: more granular
# Accessible Tree (?)
## davidb: Alexander, what did you mean here?


==name==
Server implementation: it's up to the server to define set of base features that's always enabled (e.g. focus events); others are considered as optional and should be enabled on request. In order not to break existing AT, the server should maintain a whitelist of ATs for which the current support is preserved.


expose how name was determined, i.e. if it was provided by author or is it fallback name https://bugzilla.mozilla.org/show_bug.cgi?id=637578.
See [https://lists.linux-foundation.org/pipermail/accessibility-ia2/2010-June/001186.html IA2 list] for discussion.
Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/290507...321266"