MozillaWiki

Accessibility/MathML: Difference between revisions

Page Discussion

Accessibility/MathML (view source)

Revision as of 20:25, 24 April 2014

4,798 bytes added ,  24 April 2014
Added role for the 'math' tag.
m (Updated Gecko MathML table-related element support.)
(Added role for the 'math' tag.)
 
(10 intermediate revisions by 2 users not shown)
Line 12: Line 12:


The following includes descriptions and examples for the various types of MathML elements for the purpose of this document.
The following includes descriptions and examples for the various types of MathML elements for the purpose of this document.
Some examples are pulled from MDN.
Some examples are pulled from MDN.  Information about potential roles and relations will be included alongside the elements they cover.


===Tokens===
===Tokens===
Line 33: Line 33:


<pre>
<pre>
<math>  
<math>
   <mi>y</mi>
   <mi>y</mi>
   <mo>+</mo>
   <mo>+</mo>
Line 50: Line 50:
[[File:MathML-tokens.png|frame|left|The above describes y + 10, arbitrary text, an interpreted string literal, and a glyph (an image of x).  Rendered using MathJax due to lack of glyph support in Gecko.]]
[[File:MathML-tokens.png|frame|left|The above describes y + 10, arbitrary text, an interpreted string literal, and a glyph (an image of x).  Rendered using MathJax due to lack of glyph support in Gecko.]]
<br clear=all>
<br clear=all>
====Potential role mappings====
* mi = MATHML_IDENTIFIER
* mn = MATHML_NUMBER
* mo = MATHML_OPERATOR
* mtext = MATHML_TEXT
* mspace = MATHML_SPACE
* ms = MATHML_STRING_LITERAL
* mglyph = MATHML_GLYPH


===Layout===
===Layout===
These have been split into three groups: layout elements that describe semantic meaning,
These have been split into three groups: layout elements that describe semantic meaning,
layout elements only for formatting and styling, and layout elements for describing elementary math.
layout elements only for formatting and styling, and layout elements for describing elementary math.


Line 117: Line 126:
[[File:MathML-layout-semantics.png|frame|left|The above describes (x+y) + 2, x/2 as a fraction, the square root of 2, the 5th root of x, a 3D vector (x, y, z), and (a + b) circled.  Rendered using Gecko.]]
[[File:MathML-layout-semantics.png|frame|left|The above describes (x+y) + 2, x/2 as a fraction, the square root of 2, the 5th root of x, a 3D vector (x, y, z), and (a + b) circled.  Rendered using Gecko.]]
<br clear=all>
<br clear=all>
=====Potential role mappings=====
* mrow = MATHML_ROW
* mfrac = MATHML_FRACTION
* msqrt = MATHML_SQUARE_ROOT
* mroot = MATHML_ROOT
* mfenced = MATHML_FENCED
* menclose = MATHML_ENCLOSED
=====Potential relations=====
* row to children: MATHML_ROW_FOR
* children to row: MATHML_IN_ROW
* numerator to fraction: MATHML_NUMERATOR_OF
* fraction to numerator: MATHML_NUMERATOR
* denominator to fraction: MATHML_DENOMINATOR_OF
* fraction to denominator: MATHML_DENOMINATOR
* radicand to root: MATHML_RADICAND_OF
* root to radicand: MATHML_RADICAND
* root index to root: MATHML_ROOT_INDEX_OF
* root to root index: MATHML_ROOT_INDEX


====Formatting====
====Formatting====
Line 124: Line 153:
* mpadded - adds padding for child elements, containing any number of elements
* mpadded - adds padding for child elements, containing any number of elements
* mphantom - renders child elements invisibly, containing any number of elements
* mphantom - renders child elements invisibly, containing any number of elements
As an mphantom element is visually rendered as an equivalent mrow containing the same elements, but using blank space, it's possible that these sections can be called out explicitly by ATs based on context (e.g. a worksheet for math problems).


Example markup is listed below.
Example markup is listed below.
Line 143: Line 174:
[[File:MathML-layout-formatting.png|frame|left|The above describes the expression "x + y" rendered in blue from right to left, with extra padding, and with the "+ y" not displayed.  Rendered using Gecko.]]
[[File:MathML-layout-formatting.png|frame|left|The above describes the expression "x + y" rendered in blue from right to left, with extra padding, and with the "+ y" not displayed.  Rendered using Gecko.]]
<br clear=all>
<br clear=all>
=====Potential role mappings=====
* mstyle = MATHML_STYLE
* mpadded = MATHML_PADDED
* mphantom = MATHML_PHANTOM


====Elementary math====
====Elementary math====
Line 192: Line 228:
[[File:MathML-layout-elementary.png|frame|left|300px|The above describes the subtraction of 15 from 523, with a borrow from the ten's column to the one's column, as well as the division of 5 by 5 resulting in 1.  Rendered using the MathML-mml3ff addon with Gecko as elementary math support is not natively available in Gecko.]]
[[File:MathML-layout-elementary.png|frame|left|300px|The above describes the subtraction of 15 from 523, with a borrow from the ten's column to the one's column, as well as the division of 5 by 5 resulting in 1.  Rendered using the MathML-mml3ff addon with Gecko as elementary math support is not natively available in Gecko.]]
<br clear=all>
<br clear=all>
=====Potential role mappings=====
* mstack = MATHML_STACK
* mlongdiv = MATHML_LONG_DIVISION
* msgroup = MATHML_STACK_GROUP
* msrow = MATHML_STACK_ROW
* mscarries = MATHML_STACK_CARRIES
* mscarry = MATHML_STACK_CARRY
* msline = MATHML_STACK_LINE


===Scripts and limits===
===Scripts and limits===
Line 224: Line 269:
       <mn>1</mn>
       <mn>1</mn>
     </mrow>
     </mrow>
     <mi>5</mi>
     <mn>5</mn>
   </munderover>
   </munderover>
   <mi>n</mi>
   <mi>n</mi>
Line 243: Line 288:
[[File:MathML-scripts-limits.png|frame|left|The above describes the base b with subscript 1 and superscript 2, the summation of n from 1 to 5 being 15, and x with post-superscript a and pre-subscript b.  Rendered using Gecko.]]
[[File:MathML-scripts-limits.png|frame|left|The above describes the base b with subscript 1 and superscript 2, the summation of n from 1 to 5 being 15, and x with post-superscript a and pre-subscript b.  Rendered using Gecko.]]
<br clear=all>
<br clear=all>
====Potential role mappings====
* msub = MATHML_SUB
* msup = MATHML_SUP
* msubsup = MATHML_SUB_SUP
* munder = MATHML_UNDER
* mover = MATHML_OVER
* munderover = MATHML_UNDER_OVER
* mmultiscripts = MATHML_MULTISCRIPTS
====Potential relations====
* underscript to parent structure (munder/munderover): MATHML_UNDER_OF
* parent structure to underscript: MATHML_UNDER
* overscript to parent structure (mover/munderover): MATHML_OVER_OF
* parent structure to overscript: MATHML_OVER
* base to parent structure (munder, mover, munderover, msub, msup, msubsup, mmultiscripts): MATHML_BASE_OF
* parent structure to base: MATHML_BASE
* subscript to parent structure (msub, msubsup): MATHML_SUBSCRIPT_OF
* parent structure to subscript: MATHML_SUBSCRIPT
* superscript to parent structure (msup, msubsup): MATHML_SUPERSCRIPT_OF
* parent structure to superscript: MATHML_SUPERSCRIPT
* pre-subscript to parent structure (mmultiscripts): MATHML_PRE_SUBSCRIPT_OF
* parent structure to pre-subscript: MATHML_PRE_SUBSCRIPT
* pre-superscript to parent structure (mmultiscripts): MATHML_PRE_SUPERSCRIPT_OF
* parent structure to pre-superscript: MATHML_PRE_SUPERSCRIPT
* post-subscript to parent structure (mmultiscripts): MATHML_POST_SUBSCRIPT_OF
* parent structure to post-subscript: MATHML_POST_SUBSCRIPT
* post-superscript to parent structure (mmultiscripts): MATHML_POST_SUPERSCRIPT_OF
* parent structure to post-superscript: MATHML_POST_SUPERSCRIPT


===Tables/matrices===
===Tables/matrices===
Line 350: Line 424:
[[File:MathML-tables-matrices.png|frame|left|The above describes the two-by-two identity matrix, the equation to determine the volume of a sphere, and two aligned equations.  Rendered using MathJax due to lack of support of mlabeledtr, maligngroup, and malignmark in Gecko.]]
[[File:MathML-tables-matrices.png|frame|left|The above describes the two-by-two identity matrix, the equation to determine the volume of a sphere, and two aligned equations.  Rendered using MathJax due to lack of support of mlabeledtr, maligngroup, and malignmark in Gecko.]]
<br clear=all>
<br clear=all>
====Potential role mappings====
* mtable = MATHML_TABLE
* mtr = MATHML_LABELED_ROW
* mlabeledtr = MATHML_TABLE_ROW
* mtd = MATHML_CELL
* maligngroup = MATHML_ALIGNMENT_GROUP
* malignmark = MATHML_ALIGNMENT_MARK


===Semantics annotation===
===Semantics annotation===
Line 389: Line 471:
     <!-- annotate TeX -->
     <!-- annotate TeX -->
     <annotation encoding="application/x-tex">
     <annotation encoding="application/x-tex">
       x^{2} + y  
       x^{2} + y
     </annotation>  
     </annotation>
   </semantics>
   </semantics>
</math>
</math>
Line 397: Line 479:
[[File:MathML-semantics-annotation.png|frame|left|The above is the square of x added to y, with the content also annotated (invisibly) as Content MathML and TeX markup.  Rendered using Gecko.]]
[[File:MathML-semantics-annotation.png|frame|left|The above is the square of x added to y, with the content also annotated (invisibly) as Content MathML and TeX markup.  Rendered using Gecko.]]
<br clear=all>
<br clear=all>
====Potential role mappings====
* semantics = MATHML_SEMANTICS
* annotation = MATHML_ANNOTATION
* annotation-xml = MATHML_XML_ANNOTATION


===Other===
===Other===
Line 440: Line 527:
         <mo>&sdot;</mo>
         <mo>&sdot;</mo>
         <mn>2</mn>
         <mn>2</mn>
       </mrow>  
       </mrow>
     </mfrac>
     </mfrac>
     <mfrac>
     <mfrac>
Line 452: Line 539:
[[File:MathML-other-action.gif|frame|left|The above is a togglable expression displaying 6/8, (3*2)/(4*2), and 3/4 upon each mouse press.  Rendered using Gecko.]]
[[File:MathML-other-action.gif|frame|left|The above is a togglable expression displaying 6/8, (3*2)/(4*2), and 3/4 upon each mouse press.  Rendered using Gecko.]]
<br clear=all>
<br clear=all>
====Potential role mappings====
* maction = MATHML_ACTION
* merror = MATHML_ERROR


=Interface=
=Interface=


==nsIAccessibleMathML==
==MathMLAccessible==
Inherit from nsISupports.
Inherit from HyperTextAccessibleWrap.


<pre>
* NativeAttributes - Override Accessible, and append MathML specific attributes if the element accepts them.
readonly attribute AString tokenValue;
* RelationByType - Override Accessible, process MathML specific relations.
// Get token value for token MathML elements (mi, mo, mn, mtext, mspace, ms,
 
// mglyph).  Returns empty string if not a token element.
Attributes of interest might include the following:
</pre>
 
* accent - Boolean value determining if the operator is an accent over the content.
* accentunder - Boolean value determining if the operator is an accent under the content.
* actiontype - String value specifying what an maction should do.
* align - String value specifying the type of alignment to be used for an mtable, munder, mover, munderover, or mstack.  May be useful for determining logical ordering of an equation.
* bevelled - Boolean value determining rendering format of the fraction.  May be useful for word choice (e.g. "one over two" vs. "one divided by two").
* close - String value for the closing delimiter of an mfenced element.
* columnlines - String value for the type of line between table columns.  May be useful for determining table purpose.
* crossout - String value specifying the type of crossout to be used in an mscarry, changing the crossout's meaning.
* dir - String value (ltr, rtl) specifying text direction of the content.
* fence - Boolean value specifying whether an mo is a fence element.
* href - String value for hyperlinking to a URI.
* largeop - Boolean value for determining if an mo should be drawn larger.  May be useful in determining semantics.
* linethickness - String or length value determining thickness of an mfrac line.  May be useful in determining fraction's purpose (e.g. displaying binomial coefficients for a linethickness of 0).
* location - String (cardinal direction) value determining location of an mscarry relative to its anchoring element.  May be useful in determining the carry's purpose.
* longdivstyle - String value controlling the appearance of an mlongdiv.  May be useful in determining its usage.
* mathvariant - String value specifying the logical class of a token element.  May be useful in determining a token's usage.
* notation - String value specifying the style to apply to the contents of an menclose.
* open - String value for the opening delimiter of an mfenced element.
* position - Integer value for mstack and mlongdiv child elements determining horizontal positioning.
* rowlines - String value for the type of line between table rows.
* selection - Integer value for determining which child element of an maction is to be used.
* separator - Boolean value specifying whether an mo is a separator.
* separators - String value containing separator values for an mfenced element.
* shift - Integer value for mstack and mlongdiv child elements determining vertical positioning.
* stretchy - Boolean value determining if an mo stretches to its adjacent elements' size.  May be useful in determining operator usage.


There's several different options for describing relations between MathML elements:
MathML attributes will be added to/from this tentative list to be displayed under the accessible object attributes based on their usefulness to ATs.


Option 1:  Relations linking accessibles based on role, so we won't have attributes/functions in the interface.  An example would be for a fraction structure, as follows.
Relations will link accessibles based on role.  An example would be for a fraction structure, as follows.


<pre>
<pre>
Line 477: Line 593:
</pre>
</pre>


mfrac would have RELATION_NUMERATOR_IS -> mi and RELATION_DENOMINATOR_IS -> mn or similar.
mfrac would have RELATION_NUMERATOR -> mi and RELATION_DENOMINATOR -> mn.
mi would have RELATION_NUMERATOR_FOR -> mfrac and mn would have RELATION_DENOMINATOR_FOR -> mfrac.
mi would have RELATION_NUMERATOR_OF -> mfrac and mn would have RELATION_DENOMINATOR_OF -> mfrac.


A list of potential relations is as follows:
A list of relations is as follows:


* MATHML_NUMERATOR_FOR
* MATHML_NUMERATOR_OF
* MATHML_NUMERATOR_IS
* MATHML_NUMERATOR
* MATHML_DENOMINATOR_FOR
* MATHML_DENOMINATOR_OF
* MATHML_DENOMINATOR_IS
* MATHML_DENOMINATOR
* MATHML_RADICAND_FOR
* MATHML_RADICAND_OF
* MATHML_RADICAND_IS
* MATHML_RADICAND
* MATHML_ROOT_INDEX_FOR
* MATHML_ROOT_INDEX_OF
* MATHML_ROOT_INDEX_IS
* MATHML_ROOT_INDEX
* MATHML_UNDER_FOR
* MATHML_UNDER_OF
* MATHML_UNDER_IS
* MATHML_UNDER
* MATHML_OVER_FOR
* MATHML_OVER_OF
* MATHML_OVER_IS
* MATHML_OVER
* MATHML_BASE_FOR
* MATHML_BASE_OF
* MATHML_BASE_IS
* MATHML_BASE
* MATHML_SUBSCRIPT_FOR
* MATHML_SUBSCRIPT_OF
* MATHML_SUBSCRIPT_IS
* MATHML_SUBSCRIPT
* MATHML_SUPERSCRIPT_FOR
* MATHML_SUPERSCRIPT_OF
* MATHML_SUPERSCRIPT_IS
* MATHML_SUPERSCRIPT
* MATHML_PRE_SUBSCRIPT_FOR
* MATHML_PRE_SUBSCRIPT_OF
* MATHML_PRE_SUBSCRIPT_IS
* MATHML_PRE_SUBSCRIPT
* MATHML_PRE_SUPERSCRIPT_FOR
* MATHML_PRE_SUPERSCRIPT_OF
* MATHML_PRE_SUPERSCRIPT_IS
* MATHML_PRE_SUPERSCRIPT
* MATHML_POST_SUBSCRIPT_FOR
* MATHML_POST_SUBSCRIPT_OF
* MATHML_POST_SUBSCRIPT_IS
* MATHML_POST_SUBSCRIPT
* MATHML_POST_SUPERSCRIPT_FOR
* MATHML_POST_SUPERSCRIPT_OF
* MATHML_POST_SUPERSCRIPT_IS
* MATHML_POST_SUPERSCRIPT
* MATHML_ROW_FOR
* MATHML_ROW_FOR
* MATHML_IN_ROW
* MATHML_IN_ROW


Option 2:  Explicit readonly attributes (accessors) for going from parent elements to children and vice versa.  e.g.
==MathMLTableAccessible, MathMLTableRowAccessible, and MathMLTableCellAccessible==
<pre>
Inherit from HTMLTableAccessible, HTMLTableRowAccessible, and HTMLTableCellAccessible respectively.
readonly attribute nsIAccessibleMathML radicand;
readonly attribute nsIAccessibleMathML rootIndex;
readonly attribute nsIAccessibleMathML under;
...
readonly attribute nsIAccessibleMathML parent;
</pre>
 
Returns nullptr if the related accessible doesn't exist.
Note that specialised elements such as mrow or mmultiscripts would return arrays of elements.


==nsIAccessibleMathMLTable==
Override NativeAttributes and RelationByType as with MathMLAccessible.
Inherit from nsISupports.
The table elements can be traversed using the same interface as HTMLTableAccessible (nsIAccessibleTable and nsIAccessibleTableCell).
 
<pre>
/**
* Return columns count in the table.
*/
readonly attribute long columnCount;
 
/**
* Return rows count in the table.
*/
readonly attribute long rowCount;
 
/**
* Return the MathML accessible object at the given row and column in the
* table.
*
* @param  rowIndex    [in] the row index to retrieve the cell at
* @param  columnIndex  [in] the column index to retrieve the cell at
*/
nsIAccessibleMathML getCellAt(in long rowIndex, in long columnIndex);
 
/**
* Return the number of columns occupied by the MathML accessible cell at
* the specified row and column in the table. The result differs from 1 if
* the specified cell spans multiple columns.
*
* @param  row    [in] row index of the cell to return the column extent for
* @param  column  [in] column index of the cell to return the column extent
*                  for
*/
long getColumnExtentAt(in long row, in long column);
 
/**
* Return the number of rows occupied by the MathML accessible cell at the
* specified row and column in the table. The result differs from 1 if the
* specified cell spans multiple rows.
*
* @param  row    [in] row index of the cell to return the column extent for
* @param  column  [in] column index of the cell to return the column extent
*                  for
*/
long getRowExtentAt(in long row, in long column);
 
/**
* Return the label MathML accessible of the specified row in the table if
* available (mlabeledtr), or nullptr otherwise.
*
* @param  rowIndex  [in] the row index to retrieve label mtd for
*/
nsIAccessibleMathML getRowLabel(in long rowIndex);
</pre>
 
==nsIAccessibleMathMLTableCell==
Inherit from nsISupports.
 
<pre>
/**
* Return container MathML table accessible.
*/
readonly attribute nsIAccessibleMathMLTable table;
 
/**
* Return column index of this cell.
*/
readonly attribute long columnIndex;
 
/**
* Return row index of this cell.
*/
readonly attribute long rowIndex;
 
/**
* Return the number of columns occupied by this cell. The result differs
* from 1 if the specified cell spans multiple columns.
*/
readonly attribute long columnExtent;
 
/**
* Return the number of rows occupied by this accessible cell. The result
* differs from 1 if the specified cell spans multiple rows.
*/
readonly attribute long rowExtent;
</pre>
 
==MathMLAccessible==
Inherit from AccessibleWrap, and implements nsIAccessibleMathML.
 
* NativeAttributes - Override Accessible, and append MathML specific attributes if the element accepts them.
* RelationByType - Override Accessible, process MathML specific relations if we go that route.
* GetTokenValue - Sets token value string if applicable.
* GetIsToken - Returns true if element is a token element.


==Roles==
==Roles==
One role for each of the following constructs (41 of them):
One role for each of the following constructs (41 of them), plus the <math> tag itself:


====Tokens====
====Tokens====
Line 678: Line 694:
* maction
* maction
* merror
* merror
* math


===Sample Gecko roles===
===Sample Gecko roles===
* MATHML_MATH
* MATHML_IDENTIFIER
* MATHML_IDENTIFIER
* MATHML_NUMBER
* MATHML_NUMBER
Line 721: Line 739:
* MATHML_STACK_CARRY
* MATHML_STACK_CARRY
* MATHML_STACK_LINE
* MATHML_STACK_LINE
==Comparison with WebKit==
There's a decent mapping between the proposed interface and how WebKit currently implements MathML accessibility.
As an example, the <msup> element will be used.
<pre>
<math>
  <msup>
    <mi>x</mi>
    <mn>2</mn>
  </msup>
</math>
</pre>
MathML elements are of AXRole AXGroup.  In order to determine the type of MathML element, the AXSubrole is checked.  The <msup> element has an AXSubrole of AXMathSubscriptSuperscript.
In order to determine the type of MathML element it is (<msup>, <msub>, or <msubsup>), there are the following three attributes that are added:
* AXMathBase
* AXMathSubscript
* AXMathSuperscript
These attributes point to the element fulfilling each particular role, and have a nil value otherwise.
In this case, since it's a superscript element, AXMathBase points to the <mi> and AXMathSuperscript points to the <mn>.  AXMathSubscript has a nil value.
On the <mi> and <mn> elements, no additional information regarding MathML structure is given, apart from knowing that the sub-roles are AXMathIdentifier and AXMathNumber respectively.
This is an advantage of the proposed interface, as it provides a way for child elements to know what part in their parent structure they are.
<mi> would know it's the base, and <mn> would know it's the superscript in this case.
With regards to mapping the proposed interface back to a WebKit compatible format, the existing AccessibleWrap and RoleMap can be modified so that the same interface is provided on OS X.
Additional information will also be available in the proposed interface (AXMathBaseOf or similar for the above example).
Jwei
29

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/945250...968540"