Platform/GFX/textures
TextureClient & TextureHost
TextureClient and TextureHost are the way to share texture memory accross threads or processes.
New vs Deprecated textures
Depreacted texture clients and host suffer from a mix of imperfect design decisions (design decisions that did not work well in the specific case of B2G's memory model coupled with some of the funky sharing in Gecko) and the pile of quick hacks that had to be done to keep them around. As a result, deprecated texture clients and host are hard to reason about and bug prone on B2G.
In order to incrementally fix the badness of TextureClient and TextureHost, the classes have been marked deprecated and new texture clients and host were designed, using a more strict and defined memory model. The goal is now to replace all the usage of DeprecatedTexture* by the new classes. When writing new code, if possible please use the new classes.
The biggest difference between deprecated and new textures is that deprecated did not own any shared data. Look at them as channels through which SurfaceDescriptors were sent from one side to the other. This caused ownership problems because Surface descriptor doesn't have any notion of ownership (it is just an IPDL generated structure for serialization). New textures on the other hand fully own their shared data. there should be no object referring to data shared with the compositor that is not doing so through a TextureClient (or TextureHost). This is important because TextureClient and TextureHost define a strict ownership protocol that is designed to cover all the (numerous) use cases in gecko. So there is always one and only one TextureClient/Host pair per shared buffer. If there is need to send a new buffer, then a new TextureClient/Host pair is created along with it. This way it is possible to track the lifetime of buffers that are shared between several threads on the content side, plus the compositor thread (which lives in a separate process), which was not possible with the derpecated texture (the latter supposed that we could get away with loosing ownership of the buffers when sending them accross IPC, which turned out to be incompatible with the way Gecko works for some things like video and buffer rotation).
Migration to the new textures (Meta bug 893300)
The idea is to add new compositable classes beside the deprecated ones and to remove the deprecated ones when the new ones considered stable enough on all platforms. This is an improvement for:
- Code maintainability (we can remove deprecated code and use the new one that is much easier to maintain)
- In some cases it opens the door for performance improvement
Port ContentClient/ContentHost compositable classes (Bug 893301)
- ContentClientSingleBuffered / ContentHostSingleBuffered
Probably the most important followup, the logic should not change so it should not take "too long" (maybe a week or two).
- ContentClientDoubleBuffered / ContentHostDoubleBuffered
Should be easy once SingleBuffered version is implemented (a day or two)
- ContentClientIncremental / ContentHostIncremental
The logic here needs to be modified a bit, would probably take a week.
Port TiledContentClient / TiledContentHost (Bug 893303)
Current implementation does it's own thing that does not fit even in the Deprecated textures model, so porting it to the new textures may require some investigation. The bright side is that then it would work on B2G.
Port CanvasClientWebGL / CanvasHostWebGL (Bug 893304)
Just like the tiled classes, right now it does it's own thing, needs some investigation. So porting it to new textures depends on whether we want to do it "properly" or do it like we did for the Deprecated textures.
Rebase and land the basic and D3D11 backends
New textures have been ported to these backends and even r+'d, but with the rush to get the new textures on B2G I haven't taken the timeto land them yet. (See Bug 858914)
Port the D3D9 backend
D3D9 compositing will land soon and has been made for the deprecated texture model I expect it to be similar to D3D11 so I would be surprised that this take more than a week, but I haven't read the code so I don't know for sure.
Remove all the IPDL messages that are specific to deprecated classes
This is just cleaning up unsued code, will have to wait until all the compositable classes are ported to the new textures and enabled everywhere. This is an imporvement code maintainability, a lot of bad code will be removed.
The items below are things we want to do with new textures, but if we don't do them we will still be on par with the deprecated textures. So I expect them to be lower priority, at this point we are mostly interested in the items above.
Remove GrallocBufferActor (Bug 879681)
Gralloc buffer actor is the source of a lot our gralloc related problems. The main problem is that it does not have any memory management (it is not ref counted and there is no ownership defined even though a lot of objects on different thread and processes are holding pointer to it). android::GraphicBuffer on the other hand, has builtin cross-process reference counting which is great except that by wrapping the safely manageed GraphicBuffer in an unsafe non-managed GrallocBufferActor we completely loose the benefits of GraphicBuffer.
GrallocBufferActor was not just a gratuitous evil decision, it serves one purpose that we should keep in mind when we remove it: when we send a gralloc buffer accross IPC we need to do some file descriptor serialization/decerialization that has an overhead, so instead of sending the GraphicBuffer every time, we create the it at the same time as the GrallocBufferActor and we use the GrallocBufferActor as a way to pass the buffer though IPC (matching IPDL actors is faster than doing the FD serialization dance).
The most important aspect of this work is to ensure that objects that need to hold references to gralloc buffers hold references to a reference counted object, even if this object wraps the GrallocBufferActor as a first step. This reference counted object would be GrallocTextureClient, because it has a well defined life-time protocol that is flexible enough to cover all the edge cases that we have met so far.
Once all user of gralloc are doing so through GrallocTextureClientOGL, most of the ownership bugs should be fixed, and the hardest part is done because we then just have to replace GrallocBufferActor by a more genric PTexture protocol that would serve the same purpose also with all the other texture types.
Implement partial updates for DataTextureSource implementations
This will let us use IncrementalContentHost for non-OpenGL backends. This doesn't have any dependency but will not be useful before IncrementalContentHost is ported to the new textures.
- DataTextureSourceD3D11
- DataTextureSourceBasic
Implement cross-process locks for TextureClient/TextureHost (Bug 902169)
Having them would let us have more choice in the treade-off speed vs memory usage. They can be implemented in parallel, should be really low priority. This doesn't depend on other items. (windows already has a CrossProcessLock)
- for Linux (for Shmem, shared textures)
- for OSX (for Shmem, shared textures)
This is an improvement for:
- flexibility/memory usage: we will be able to use single buffering in places where we can only double buffer if we don't have a lock
Implement TextureClient pools
This is a performance improvement that is important at least for video: rather than allocating/deallocating shared memory every time, recycle TextureClients (along with their shared buffers and reuse them). We had this feature only with async-video before the all mighty layers refactoring, and lost it during the refactoring. If we implement it now it should work also for non-omtc (thanks to the refactoring). This is an improvement for:
- performances in places where we tend to allocate and deallocate a lot of buffers (like video)
Make it possible for several compositables to use the same Texture.
We need to do this by replacing the pair {PCOmpositable, ID} by a proper PTexture IPDL actor to refer to texture clients and host accros IPC. This will be a big performance win in certain specific cases, and let us implement the "Texture atlas" optimization for mobile platforms where we want to reduce the number of draw calls. This doesn't have dependencies, although we'll probably see clearer when more compositables are ported? I expect ImageClient/Host to be the only compositable that would use it at least at first, but I may be wrong. See bug 897452
This is an improvement for:
- performances
- memory usage