Gecko:CrossProcessLayers

Proposal

• Have a dedicated "GPU Process" responsible for all access to the GPU via D2D/D3D/OpenGL
  • This process can be killed and restarted to fix leaks or cover for other driver bugs
  • This process would be privileged, allowing content processes to be sandboxed but still use the GPU remotely
  • In some configurations the GPU process could be a set of threads in a regular browser process (the master process)
• Browser processes (content and chrome) maintain a layer tree on their main threads
  • Layer tree is maintained by layout code
• Each transaction that updates a layer tree pushes a set of changes over to a "shadow layer tree"
  • This shadow layer tree is what we use for rendering off the main thread
  • This is necessary for layer-based animation to work without being blocked by the main thread
• Since we're pushing changes across threads anyway, we might as well push them across process boundaries at the same time, so push them all the way to the GPU process
• Therefore, the GPU process maintains a set of shadow layer trees and is responsible for compositing them
• Compositing a shadow layer tree either results in a buffer that's rendered into a window, or a buffer that is later composited into some other layer tree
  • We can reduce VRAM usage at the expense of increased recomposition work by recompositing a content process layer tree every time we composite its parent layer tree and not having a persistent intermediate buffer
  • We can control the scheduling of content layer tree composition

This proposal lets us use a generic remoting layer backend. Hardware/platform specific backends are isolated to the GPU process and do not need to do their own remoting.

Implementation Steps

The immediate need is to get something working for Fennec. Proposal:

• Initially, let the GPU process be a thread in the master process
• Build the remoting layers backend
• Publish changes from the content process layer trees and the master process chrome layer tree to shadow trees managed by the GPU thread

Implementation Details for Fennec

• Key question: what cairo backend do we use to draw into ThebesLayers?
  • Image backend?
    • Allocate shared system memory or bc-cat buffers for the regions of ThebesLayers to update
    • Gecko processes draw into those areas using cairo image backend
    • GL backend uploads textures from system memory or acquires texture handle for bc-cat buffer across processes (e.g. using texture_to_pixmap); composites those changes into its ThebesLayer buffers
    • Windowless plugins suffer, except for Flash where we have NPP_DrawImage and can use layers to composite those images together
    • GTK theme rendering suffers
    • do we care?
  • Xlib backend?
    • Allocate textures for changed ThebesLayer areas and map to pixmap using texture_to_pixmap
    • Gecko processes draw into those pixmaps using cairo Xlib backend
    • GL backend acquires texture handle across processes; composites those changes into its ThebesLayer buffers
  • Maybe we can have some XShm or bc-cat hack that lets us do it all ... an X pixmap backed that we can also poke directly through shared memory that's also a texture!

Future Details

• How to handle D2D?
  • Direct access
    • Allocate D3D buffer for changed ThebesLayer areas
    • Gecko processes draw into it using cairo D2D backend
  • Indirect access (for sandboxed content processes etc)
    • Remote cairo calls across to the GPU process, creating a command queue that gets posted instead of a new buffer