Mobile/IME
Introduction
Here we will be collecting information about keyboard/IME handling and issues in Fennec.
IME architecture
Components
Android IME-handling code consists of three major parts: Java, widget, and content (XUL version also had IPC layer as the content process was separate).
Java part is mostly in GeckoInputConnection.java, which deals with Android API, and translates Java interface calls made by Android system to the corresponding Gecko events.
The IME events are the objects of GeckoEvent class with one of the IME_XXX types. They get sent using GeckoAppShell.sendEventToGecko() across JNI to nsWindow.cpp in widget, wrapped in AndroidGeckoEvent C++ class.
nsWindow does minimum packaging and dispatches the events further to content, where they go all over the place. Many of those events are handled in content/events/src/nsContentEventHandler.cpp, for example NS_QUERY_SELECTED_TEXT is dispatched to nsContentEventHandler::OnQuerySelectedText.
The IME events go only in one direction: from front-end to back-end, the reverse communication is done using notifications going from back-end to front-end. They pass through nsWindow::OnIMExxx in widget, across JNI to GeckoAppShell.NotifyIMEChange, and handled in GeckoInputConnection.notifyXXXChange.
After the major Java part redesign made in bug 595008 most of the input handling is done by the base Android class BaseInputConnection. We override only some of its methods to send the actual editable object changes to Gecko. The majority of the IME events from Java code are sent to Gecko from GeckoInputConnection.onTextChanged() method.
Ranges
Some IMEs are marking a part of the text that you're working on. For example, here a part of the text being entered is underlined:
Information about those parts is passed to Gecko as an array of ranges. Each range has several properties, including a type, which defines the nature of the range, styles like underlining, and text colors. The most noticeable and important range types are the composing text, and the text selection.
The ranges are stored as spans in the internal mEditable class member, and handled by the BaseInputConnection class.
Composition
A composing text is a range of text that you are currently working on. While it is in the composition state, it can be freely changed, these changes are considered as a batch operation, and are submitted as one update at the end of the composition, so the whole text modification is stored as one action in the edit history.
GeckoInputConnection tracks the composing state using mComposing flag. When it receives a text modification request from Android it sets this flag and sends IME_COMPOSITION_BEGIN event to Gecko. While in that state the mComposingText variable contains the current composing text, and mCompositionStart points to a start index of the composition within the text body. mCompositionSelStart/mCompositionSelLen track the text selection during the composition, as Gecko selection is not supposed to change while in the composing state.