MozillaWiki

Platform/GFX/textures: Difference between revisions

Page Discussion

Platform/GFX/textures (view source)

Revision as of 15:30, 23 October 2013

541 bytes added ,  23 October 2013
no edit summary
No edit summary
No edit summary
Line 74: Line 74:
To give an analogy with android's SurfaceTexture, CompositableClient should be the equivalent of ANativeWindow, and a given implementation of TextureClient would be the equivalent of SurfaceTexture and implement the same producer/consumer model, while TextureClient/Host operate at a lower level to just abstract out the type memory (since the multiple backends require us to support more than just EGLSurface).
To give an analogy with android's SurfaceTexture, CompositableClient should be the equivalent of ANativeWindow, and a given implementation of TextureClient would be the equivalent of SurfaceTexture and implement the same producer/consumer model, while TextureClient/Host operate at a lower level to just abstract out the type memory (since the multiple backends require us to support more than just EGLSurface).


== Migration to the new textures (Meta bug 893300)==
== Planned work ==
 
The idea is to add new compositable classes beside the deprecated ones and to remove the deprecated ones when the new ones considered stable enough on all platforms.
This is an improvement for:
* Code maintainability (we can remove deprecated code and use the new one that is much easier to maintain)
* In some cases it opens the door for performance improvement


=== Port ContentClient/ContentHost compositable classes (Bug 893301) ===
=== Port ContentClient/ContentHost compositable classes (Bug 893301) ===
Line 93: Line 88:


Current implementation does it's own thing that does not fit even in the Deprecated textures model, so porting it to the new textures may require some investigation. The bright side is that then it would work on B2G.
Current implementation does it's own thing that does not fit even in the Deprecated textures model, so porting it to the new textures may require some investigation. The bright side is that then it would work on B2G.
Bas seems to have plans to do that.


=== Port CanvasClientWebGL / CanvasHostWebGL (Bug 893304) ===
=== Port CanvasClientWebGL / CanvasHostWebGL (Bug 893304) ===
Line 98: Line 94:
Just like the tiled classes, right now it does it's own thing, needs some investigation.
Just like the tiled classes, right now it does it's own thing, needs some investigation.
So porting it to new textures depends on whether we want to do it "properly" or do it like we did for the Deprecated textures.
So porting it to new textures depends on whether we want to do it "properly" or do it like we did for the Deprecated textures.
Sotaro has a plan to get WebGL to use ImageBridge + ImageContainer (so this would use ImageClient which already is ported to the new textures.


=== Rebase and land the basic and D3D11 backends ===
=== Rebase and land the basic and D3D11 backends ===
Line 108: Line 105:
D3D9 compositing will land soon and has been made for the deprecated texture model
D3D9 compositing will land soon and has been made for the deprecated texture model
I expect it to be similar to D3D11 so I would be surprised that this take more than a week, but I haven't read the code so I don't know for sure.
I expect it to be similar to D3D11 so I would be surprised that this take more than a week, but I haven't read the code so I don't know for sure.
=== Remove all the IPDL messages that are specific to deprecated classes ===
This is just cleaning up unsued code, will have to wait until all the compositable classes are ported to the new textures and enabled everywhere.
This is an imporvement code maintainability, a lot of bad code will be removed.
== Other texture-related plans ==
The items below are things we want to do with new textures, but if we don't do them we will still be on par with the deprecated textures. So I expect them to be lower priority, at this point we are mostly interested in the items above.
=== Remove GrallocBufferActor (Bug 879681) ===
Gralloc buffer actor is the source of a lot our gralloc related problems. The main problem is that it does not have any memory management (it is not ref counted and there is no ownership defined even though a lot of objects on different thread and processes are holding pointer to it). android::GraphicBuffer on the other hand, has builtin cross-process reference counting which is great except that by wrapping the safely manageed GraphicBuffer in an unsafe non-managed GrallocBufferActor we completely loose the benefits of GraphicBuffer.
GrallocBufferActor was not just a gratuitous evil decision, it serves one purpose that we should keep in mind when we remove it: when we send a gralloc buffer accross IPC we need to do some file descriptor serialization/decerialization that has an overhead, so instead of sending the GraphicBuffer every time, we create the it at the same time as the GrallocBufferActor and we use the GrallocBufferActor as a way to pass the buffer though IPC (matching IPDL actors is faster than doing the FD serialization dance).
The most important aspect of this work is to ensure that objects that need to hold references to gralloc buffers hold references to a reference counted object, even if this object wraps the GrallocBufferActor as a first step. This reference counted object would be GrallocTextureClient, because it has a well defined life-time protocol that is flexible enough to cover all the edge cases that we have met so far.
Once all user of gralloc are doing so through GrallocTextureClientOGL, most of the ownership bugs should be fixed, and the hardest part is done because we then just have to replace GrallocBufferActor by a more genric PTexture protocol that would serve the same purpose also with all the other texture types.
=== Implement partial updates for DataTextureSource implementations ===
This will let us use IncrementalContentHost for non-OpenGL backends.
This doesn't have any dependency but will not be useful before IncrementalContentHost is ported to the new textures.
* DataTextureSourceD3D11
* DataTextureSourceBasic


=== Implement cross-process locks for TextureClient/TextureHost (Bug 902169) ===
=== Implement cross-process locks for TextureClient/TextureHost (Bug 902169) ===


Having them would let us have more choice in the  treade-off speed vs memory usage.
Having them would let us have more choice in the  trade-off speed vs memory usage.
They can be implemented in parallel, should be really low priority.
They can be implemented in parallel, should be really low priority.
This doesn't depend on other items. (windows already has a CrossProcessLock)
This doesn't depend on other items. (windows already has a CrossProcessLock)
Line 155: Line 125:
=== Make it possible for several compositables to use the same Texture. ===
=== Make it possible for several compositables to use the same Texture. ===


We need to do this by replacing the pair {PCOmpositable, ID} by a proper PTexture IPDL actor to refer to texture clients and host accros IPC.
We need to do this by replacing the pair {PCompositable, ID} by a proper PTexture IPDL actor to refer to texture clients and host accros IPC.
This will be a big performance win in certain specific cases, and let us implement the "Texture atlas" optimization for mobile platforms where we want to reduce the number of draw calls.
This will be a big performance win in certain specific cases, and let us implement the "Texture atlas" optimization for mobile platforms where we want to reduce the number of draw calls.
This doesn't have dependencies, although we'll probably see clearer when more compositables are ported? I expect ImageClient/Host to be the only compositable that would use it at least at first, but I may be wrong. See bug 897452
This doesn't have dependencies, although we'll probably see clearer when more compositables are ported? I expect ImageClient/Host to be the only compositable that would use it at least at first, but I may be wrong. See bug 897452
Line 162: Line 132:
* performances
* performances
* memory usage
* memory usage
* Fix a broken feature (same video in several layers).
=== Remove GrallocBufferActor (Bug 879681) ===
Gralloc buffer actor is the source of a lot our gralloc related problems. The main problem is that it does not have any memory management (it is not ref counted and there is no ownership defined even though a lot of objects on different thread and processes are holding pointer to it). android::GraphicBuffer on the other hand, has builtin cross-process reference counting which is great except that by wrapping the safely manageed GraphicBuffer in an unsafe non-managed GrallocBufferActor we completely loose the benefits of GraphicBuffer.
GrallocBufferActor was not just a gratuitous evil decision, it serves one purpose that we should keep in mind when we remove it: when we send a gralloc buffer accross IPC we need to do some file descriptor serialization/decerialization that has an overhead, so instead of sending the GraphicBuffer every time, we create the it at the same time as the GrallocBufferActor and we use the GrallocBufferActor as a way to pass the buffer though IPC (matching IPDL actors is faster than doing the FD serialization dance).
The most important aspect of this work is to ensure that objects that need to hold references to gralloc buffers hold references to a reference counted object, even if this object wraps the GrallocBufferActor as a first step. This reference counted object should be GrallocTextureClient, because it has a well defined life-time protocol that is flexible enough to cover all the edge cases that we have met so far.
Once all user of gralloc are doing so through GrallocTextureClientOGL, most of the ownership bugs should be fixed, and the hardest part is done because we then just have to replace GrallocBufferActor by a more genric PTexture protocol that would serve the same purpose also with all the other texture types.
=== Make shared surface allocation get its own IPDL protocol ===
The allocation protocol would be independent from layers (probably live in gfx/ipc) and just use the same ipc channel as layers.
This would give us a cleaner design and make the allocator's lifetime not dependent on the ShadowLayerForwarder.
=== Move TextureClient out of gfx/layers ===
TextureClient is all about managing the lifetime of shared texture data. This is not specific to layers even though layers happens to be the only user at the moment.
This could involve adding a ShareableSurface interface that would expose reference counting, Lock and Unlock. TextureClient would inherit from it and implement the ipc stuff. This is really just low priority sugar to that aims at getting people comfortable with the idea that using TextureClient does not mean depending on layers, and that TextureClient is not an implementation detail. Maybe what we need is to rename texture client into something else.
=== Implement partial updates for DataTextureSource implementations ===
This will let us use IncrementalContentHost for non-OpenGL backends.
This doesn't have any dependency but will not be useful before IncrementalContentHost is ported to the new textures.
* DataTextureSourceD3D11
* DataTextureSourceBasic
=== Remove all the IPDL messages that are specific to deprecated classes ===
This is just cleaning up unsued code, will have to wait until all the compositable classes are ported to the new textures and enabled everywhere.
This is an imporvement code maintainability, a lot of bad code will be removed.
Nical
Confirmed users
138

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/735377"