IA2/ATK Text Interface. Next

Problems of existing approach

• (#1) Embedded objects concept adds complexity which potentially affects performance
• (#2) Hard to figure out what the word/line is. Example

  <p>a<a>b</a>c</p>

  p is one word 'abc'
• (#3) Offsets concept can be confusing because same offset can belong to two lines (in case of soft line breaks). So when AT handles caret move event and get text at caret offset (no magic value) then returned line may be wrong
• (#4) offsets can be difficult/expensive to calculate for the server

Use cases

• Move by unit and get the text (including attributes and enclosing accessibles)
• Get selection (including attributes and enclosing accessibles)
• Set selection
• Get/set caret
• Check whether the caret has moved; e.g. deletion or cursoring where you get the caret, perform some operation, get the caret again, compare
• Check whether something is at the start/end of the document/paragraph/line; e.g. to inform the user, to display something only at the start of the line (such as a bullet)
• Review of text without moving the caret/selection

Text Range API

Advantages:

• solves all problems
• doesn't have text offsets restrictions like one text offset may describe two text points (for example, caret right before a link and inside the link
• close to UIA which is good in terms of portability
• more steady to bugs since wrong move in text is better than wrong offsets because wrong move allows to traverse the whole text while wrong offsets usually don't

Disadvantages:

• can be complex for implementation on the server side
• servers have to keep track of more objects
• servers may actually have to deal with at least some offsets internally, so they have to keep track of the range if text mutates
• a lot of work to transition
• both clients and servers will probably have to implement both the old and new interfaces because of older servers/clients which don't support the newer interface

Types:
Accessible
// Specifies the start or end of a text range.
TextEndPoint: EP_START, EP_END
// Specifies a significant position in the text.
TextPosition: POS_FIRST, POS_LAST, POS_ALL, POS_CARET
TextAttributes
Rect

TextUnit:
UNIT_PARAGRAPH, UNIT_SENTENCE, UNIT_LINE, UNIT_WORD, UNIT_CHARACTER
// Covers text that has the same accessible and attributes.
// Used to gather text while also communicating all changes.
UNIT_CHANGE

/* Represents a range of text.
 * Ranges are bounded by two endpoints:
 * the start (EP_START) and the end (EP_END).
 */
TextRange:
// Get the text in this range.
string getText()
/* Compare this range with another range.
 * @param other: The range to compare against.
 * @return: 0 if the ranges are the same, -1 if this range is before the other range,
 *  1 if this range is after the other range.
 */
long compare(TextRange other)
/* Compare an endpoint in this range with an endpoint in another range.
 * @param thisEndPoint: The endpoint in this range.
 * @param other: The other range.
 * @param otherEndPoint: The endpoint in the other range.
 * @return: 0 if the endpoints are the same, -1 if this endpoint is before the other endpoint,
 *  1 if this endpoint is after the other range.
 */
long compareEndPoints(TextEndPoint thisEndPoint, TextRange other,  TextEndPoint otherEndPoint)
/* Set an endpoint in this range to an endpoint in another range.
 * @param thisEndPoint: The endpoint in this range.
 * @param other: The other range.
 * @param otherEndPoint: The endpoint in the other range.
 */
void setEndPoint(TextEndPoint thisEndPoint, TextRange other,  TextEndPoint otherEndPoint)
// Expand this range to cover the entire unit of text.
void expand(TextUnit unit)
/* Move endpoint in this range to a unit of text.
 * @param unit: The unit of interest.
 * @param direction: How many units to move forward or backward;
 *  positive means forward, negative means backward.
 * @param endPoint: The endpoint to move.
 */
void move(TextUnit unit, long direction, TextEndPoint endPoint)
// Make an independent copy of this range.
TextRange copy()
// Set the caret to this range.
// If the range is not collapsed, the start will be used.
void setCaret()
// Select this range.
void select()
// Unselect this range.
void unselect()
/* Split this range into units and/or at another range.
 * @param unit: The unit to split by.
 * @param range: A range to split at.
 *  For example, this is useful when retrieving the line at the caret
 *  and you want to know where the caret is positioned in that line.
 * @return: An array of ranges and the start and end indexes into that array
 *  covering the requested split range.
 */
TextRange[], long, long split(TextUnit unit, TextRange range)
// Retrieve the text attributes for text in this range.
TextAttributes attributes()
// Retrieve the accessible enclosing this range.
Accessible enclosingAccessible()
Rect screenLocation()

TextProvider:
// Retrieve the range of text covered by a given accessible.
TextRange rangeForAccessible(Accessible accessible)
// Retrieve a significant position in the document.
TextRange rangeForPosition(TextPosition position)
// Retrieve the range(s) that are selected.
TextRange[] getSelection()
TextRange rangeForScreenLocation(Rect location)

Accessible:
There needs to be a way to retrieve the TextProvider from any accessible. In Windows, we'd probably do this with QueryService. For ATK, it might require another interface or method on Accessible.

Usage Example

Pythonic pseudo-code to get the line at the caret:

# This could just as well be the document if that's easier.
acc = getFocus()
tp = acc.getTextProvider()
caret = tp.rangeForPosition(POS_CARET)
line = caret.copy()
line.expand(UNIT_LINE)
ranges, caretStart, caretEnd = line.split(UNIT_CHANGE, caret)
lastAcc = None
lastAttrs = None
index = 0
for r in ranges:
    acc = r.enclosingAccessible()
    attrs = r.attributes()
    if acc != lastAcc:
        presentInfoAboutAccessible(acc)
    if attrs != lastAttrs:
        presentChangedAttributes(lastAttrs, attrs)
    if index == caretStart:
        presentBrailleCaret()
    presentText(r.text)
    index += 1

Questions

• How do we handle detection of caret movement for backspacing? Ranges should stick with the text when the underlying text moves, but in the case of backspace, the caret range will be in the same place, so it will always seem as if the caret hasn't moved.
• Should TextRange have collapse and isCollapsed methods? These can be achieved using setEndPoint and compareEndPoints respectively, but this isn't necessarily intuitive.
• Do we want EP_BOTH to specify both endpoints for TextRange.move (which would result in a collapsed range)? Note that it would only be used by one method.
• Should TextRange.split for UNIT_CHANGE somehow specify what the change is (accessible, attributes or both)? This would avoid some needless calls. However, it'd need to be an array of pairs or something and might need to be a method specific to UNIT_CHANGE.

No embed objects for inlines

Advantages:

• relatively small changes, totally goes with IA2 and ATK specifications
• solves words navigation problem (#2)
• reduces some complexity (#1); e.g. getting lines and words

Disadvantages:

• not backward compatible since AT might have dependencies on existing implementation, AT sniffing can be a requirement
• doesn't solve complexity problem for selection (#1)
• doesn't solve (#3), though this can be solved (if inelegantly) with IA2_TEXT_OFFSET_CARET or equivalent
• doesn't solve (#4)

Text Interface Extension

Advantages:

• relatively small change
• supposed to solve navigation problem (#2)
• solves problem (#1) for the client since it allows navigation with no embedded characters

Disadvantages:

• doesn't solve (#1) for the server
• doesn't solve (#3), though this can be solved (if inelegantly) with IA2_TEXT_OFFSET_CARET or equivalent
• doesn't solve (#4)

// Get the start and end of a unit.
getUnitBoundaries(Accessible accessible, long offset, TextUnit unit, Accessible* startAccessible, long* startOffset, Accessible* endAccessible, long* endOffset)
// Get the boundaries of the selection.
GetSelection(TextUnit unit, Accessible* startAccessible, long* startOffset, Accessible* endAccessible, long* endOffset)
// Walk through a range, splitting it at accessibles and attribute changes.
splitRangeAtChanges(Accessible startAccessible, long startOffset, Accessible endAccessible, long endOffset, pair<Accessible, Attributes>[]* segments)