Accessibility/B2G
Boot2Gecko Accessibility, AKA B2AW (Boot 2 Accessible Web)
This is a collection of ideas and requirements for the B2G project from the accessibility standpoint.
Output via Speech
Speech output can be done through either the Android accessibility/TTS framework or
huh, why would we use the android stuff??? its just some java goo around espeak svox pico and whatever other tts's they support now. Since we'll only care about the freely redistributable ones anyway, it seems to make a lot more sense to just directly use the respective libraries api and expose some reasonable abstraction to js through ctypes / xpcom
through writing an engine ourselves in JavaScript. The latter has the advantage that we have it under our control what we include and don't have to rely on possibly closed-sourced parts of the Android infrastructure. The things to consider are:
- Multilingual support. We don't want something that speaks only English.
- Speech quality: We don't want something that sounds like it was built on a Votrax chip in the early 1980s.
Speech recognition
A solution that, like Nuance does it, sends stuff to a server to be recognized, since the speech rec process requires processing power that is not really available on mobile devices today. Speech recognition consumes considerably more CPU power than TTS.
First, have Legal check whether we can sort of copy Apple's gesture-based interaction model or if they have patented it. It is a sophisticated gesturing model that allows blind users to do virtually everything without requiring a physical keyboard:
- Touch something: Speak what's under the finger.
- Double tap the last touched item: Activate it if possible (e. g. activate a link, set focus to an input etc.)
- Swipe left and right: Move from item to item. Not just tabbable ones, but everything, so the page can be explored element by element.
- Two finger dial gesture: Set the rotor (see next gesture) to a certain element. Elements can be links, headers, landmarks, form elements, lists, graphics, tables, and others we think are useful. Turn the dialer counterclock wise and the selection moves in the opposite direction.
- Swipe up and down: Move to the previous resp. next element of a given type.
- Two finger swipe up: Read from beginning of page to currently touched element.
- Two finger Swipe down: Read from current location to end of page continuously.
- Three finger swipe down/up: Move to next/previous visible part of page (like what fits on the touchscreen)
- Tap with three fingers: Say which page, of how many, one is on. Gives the user an idea on how long the page actually is.
Physical Keyboard input
This could be tailored to what NVDA is doing: In browse situations, allow to jump to headings, landmarks, links, form fields, graphics, and others. When inside an editable field, type into the field. Make it transparent to users whether they're browsing or filling out a form (the Orca/VoiceOver transparency model).
Zoom
Provide Zoom features.