The Async Pan/Zoom module (APZ)1 is a platform component that allows panning and zooming to be performed asynchronously (on the compositor thread rather than the main thead).
For zooming, this means that the APZ reacts to a pinch gesture immediately and instructs the compositor to scale the already-rendered layers at whatever resolution they have been rendered (so e.g. text becomes more blurry as you zoom in), and meanwhile sends a request to Gecko to re-render the content at a new resolution (with sharp text and all).
For panning, this means that the APZ asks Gecko to render a portion of a scrollable layer, called the "display port", that's larger than the visible portion. It then reacts to a pan gesture immediately, asking the compositor to render a different portion of the displayport (or, if the displayport is not large enough to cover the new visible region, then nothing in the portions it doesn't cover - this is called checkerboarding), and meanwhile sends a request to Gecko to render a new displayport. (The displayport can also be used when zooming out causes more content of a scrollable layer to be shown than before.)
1. This module used to be called Async Pan/Zoom Controller (APZC), but this was changed to avoid confusion because there is a class called AsyncPanZoomController of which there are now multiple instances.
The APZ module is currently enabled on B2G and Metro. Fennec has a Java implementation of asynchronous panning and zooming, but the plan is to port Fennec to use the APZ module as well.
The APZ module relies on OMTC, so a prerequisite for porting it to a platform is for OMTC to work on that platform.
Inside the APZ module, every scrollable layer has an AsyncPanZoomController (APZC). Scrollable layers roughly correspond to the root document, documents in iframes, and overflow:scroll elements. See OMTC for more information about layers.
The APZCs are arranged in a tree whose structure reflects the structure of the layer tree containing the layers that they correspond to, except that the APZC tree only has nodes for scrollable layers, while the layer tree has nodes for all layers. The APZC tree is managed by a class called APZCTreeManager. The APZCTreeManager is the interface through which the outside world interacts with the APZ module - there is no access to the APZCs directly, but the APZCTreeManager provides methods that operate on a specific APZC in the tree. These methods identify an APZC using three integer identifiers, grouped in a structure called ScrollableLayerGuid. The three identifiers are:
- A layers id, which identifies the layer tree that the scrollable layer belongs to. Layers ids are maintained by the compositor, with CompositorParent::RootLayerTreeId() returning the layers id for the root layer tree, and CompositorParent::AllocateLayerTreeId() allocating a layers id for a new layer tree.
- A pres shell id, which is a temporal identifier that correponds to the document that contains the scrollable layer, which may change over time. The pres shell id can be obtained using nsIDOMWindowUtils::GetPresShellId().
- A scroll id, which identifies the actual scroll frame. The scroll id is called a view id in Gecko code, and can be obtained using nsIDOMWindowUtils::GetViewId().
Interactions with other components
The APZ interacts with the following other platform components:
The compositor creates and stores the APZCTreeManager, and interacts with it in the following ways:
- When the compositor receives an update to the layer tree, it propagates this update to the APZCTreeManager by calling APZCTreeManager::UpdatePanZoomControllerTree(). This function updates the tree of APZCs to reflect the tree of scrollable layers in the updated layer tree.
- Every time the compositor composites a frame, it queries each APZC for its current async transform (see AsyncPanZoomController::SampleContentTransformForFrame()). The APZC modifies this transform as the user pans and zooms.
- APZCs can schedule a composite by calling CompositorParent::ScheduleRenderOnCompositorThread().
The widget code is a platform-specific component of a browser implementation that interfaces with the native widget implementation. It corresponds roughly to the following pieces of code:
- On B2G: RenderFrameParent/RenderFrameChild and TabParent/TabChild
- On Metro: MetroWidget and MetroInput
- On Fennec (which doesn't use the APZ yet): GeckoAppShell and GeckoLayerClient, which communicate with native code via AndroidJNI.
The widget code interacts with the APZ in the following ways:
- forwards relevant input events to the APZ (APZCTreeManager::ReceiveInputEvent())
- notifies the APZ about Gecko events that are relevant to it, such as:
- a reflow that causes a change in the dimensions of a scrollable layer (APZCTreeManager::UpdateCompositionBounds())
- a scrollTo performed by content (APZCTreeManager::UpdateScrollOffset())
- a request to zoom in to a rectangle (APZCTreeManager::ZoomToRect())
In addition, the widget code provides APZ with an interface for interacting with Gecko, called GeckoContentController. The implementation of this interface is different for each platform. It may even be different for different APZCs within one platform (for example, in the B2G browser, APZCs representing scrollable elements in content processes use the RemoteContentController implementation, while those representing scrollable elements in the parent process will use a different implementation (currently being developed in bug 912657)).
GeckoContentController allows the APZ to do the following:
- request a repaint (GeckoContentController::RequestContentRepaint())
- request handling of various gestures, such as single taps, double taps, and long taps (GeckoContentController::HandleSingleTap() etc.)
- fire async scroll events (GeckoContentController::SendAsyncScrollDOMEvent())
- schedule arbitrary actions to be performed on the Gecko thread (GeckoContentController::PostDelayedTask())
Threads and processes
When OMTC is enabled on a platform (which is a requirement for using the APZC), the compositor runs on its own thread, called the compositor thread. APZCs and the APZCTreeManager can be thought of as living on the compositor thread, although they can be used in certain ways by other threads (usually the Gecko thread, or the platform UI thread if there is one). APZCTreeManager.h documents which APZCTreeManager methods can be called on which threads.
On B2G, there are not only multiple threads but also multiple processes, as each app (or in the case of the browser, each tab) has its own process. In this setup, only the parent process has a compositor thread, and all APZCs live in the parent process, even ones corresponding to layers from a child thread.
APZ code needs to deal with quite a variety of coordinate systems. Here are some resources for understanding the various coordinate systems involved. They build on each other, so looking at them in order is recommended:
- Unraveling coordinate systems
- Unraveling coordinate systems, part 2
- The comment above APZCTreeManager::GetInputTransforms()
- Diagram illustrating the coordinate confusion that gave rise to bug 935219
For debugging APZ-related issues, bug 958596 added logging of the APZC tree. When enabled, on every layers update, the APZC tree manager prints out the structure of the layer tree, showing which layers have an APZC, and for those layers showing some APZ-related metrics.
This logging can be turned on by setting the "apz.printtree" pref to "true".
(In no particular order)
Turn on APZ in all Gaia apps - bug 909877
- Fix up issues in Gaia with APZ enabled - bug 949585
Make APZ work well for Metro - bug 886321
Make APZ work even better in Metro - bug 963116(Metro project cancelled)
Implement support for Fennec-like dynamic toolbar on B2G - bug 860812
- Improve hit detection for the different touch-sensitive regions - bug 928833
Implement support for the CSS touch-action property - bug 795567
- Implement touch-action bindings for B2G and enable it - bug 960316, bug 960209
- Switch Fennec over from the Java version to the C++ APZ - bug 776030
- Reduce input latency by changing the threading model - bug 930939
- Get APZ turned on for desktop platforms - bug 944938 for OS X
- Smooth scrolling API for content/script - bug 928839
If you have questions about anything APZ-related, please add them here, and somebody watching this page will try to answer it.
- What platforms is APZ currently enabled on? [2014-01-14]
- As of January 8, 2014, APZ is currently enabled in all B2G processes that are not the root process (i.e. this includes all browser content and Gaia apps). It is also enabled on Firefox Metro (when running in metro mode).
- How does APZ impact the onload event? Will this impact user perceived performance? [2014-01-28]
- In general APZ shouldn't really impact the onload event. We might be painting a larger area now but if the app is well-behaved and doesn't trigger a lot of paints the impact from this should not be very large. If there are reports of delayed onload because of APZ, it would be useful to get profiles with and without APZ enabled to compare.
- If the onload event is delayed due to a long page, should we expect regressions in constructing long pages/elements after load? [2014-01-28]
- By this I assume you mean dynamically generating DOM nodes and inserting them after the load event has fired. As this would also trigger repaints, and again since we are painting a larger area it will be slower. The more repaints that are triggered, the slower it will be. I don't think it matters much whether the nodes are inserted before or after the load event, except that if you're doing it via script rather than in the HTML itself it's more likely to accidentally trigger reflow by querying some layout property.
- I've heard we're expecting scroll performance to generally be better everywhere. Granted there are some exceptions with things like sticky headers which we will be resolving soon. Are there any other times we would see a performance regression when it comes to scrolling and FPS? [2014-01-28]
- Average FPS should be better with APZ, but because we don't have tiling yet we might see some jank when the painted content is uploaded to the compositor, because it is a large buffer that needs to be copied over. This may show up as a regression depending on how you measure it. Note that currently input events still need to be processed on the Gecko thread in the root process before they are used by APZ, so scrolling isn't 100% asynchronous. This might be a source of poor scrolling performance, if the root process' Gecko thread is busy - however this should be no worse than without APZ so I wouldn't call it a regression.
- Do you have any suggestions on ways to architect our apps/tests to take advantage of APZ? [2014-01-28]
- I think I've run into a few apps on B2G where the body element is not scrollable but all the scrollable content lives inside another overflow:scroll div. (I think this might be the case in the settings app, actually). This is undesirable because while we ensure the body's content is always layerized and ready to scroll right away, other scrolling subdocuments may not be. This is required to prevent allocating massive amounts of memory. What happens is that as soon as you start scrolling the subdocument it will be layerized and you can scroll it async, but there is still a small latency involved. I think Chris Lord filed a bug about this for the settings app. Other than that, just minimizing the number of paints would help - try to use animations and such that can be done in the compositor as much as possible (BenWa can provide more info on this if you need it).