Confirmed users
138
edits
Platform/GFX/textures: Difference between revisions
Jump to navigation
Jump to search
no edit summary
No edit summary |
No edit summary |
||
| Line 2: | Line 2: | ||
== TextureClient & TextureHost == | == TextureClient & TextureHost == | ||
TextureClient and TextureHost are the way to share texture memory | TextureClient and TextureHost are the way to share texture memory across threads or processes. | ||
For more information read the inline documentation | For more information read the inline documentation | ||
| Line 9: | Line 9: | ||
* http://dxr.mozilla.org/mozilla-central/source/gfx/layers/composite/TextureHost.h | * http://dxr.mozilla.org/mozilla-central/source/gfx/layers/composite/TextureHost.h | ||
In order to efficiently render web content we need to be able to paint content in a surface that can be shared and composited directly. It is important to avoid copies, so we often face situations where the surface has to be shared with several threads living in different processes at the same time. As a result managing the lifetime of the shared data can become tricky. Our shared surfaces need cross-process reference counting and locking semantics. | |||
The TextureClient and TextureHost abstraction offer exactly that., where TextureClient is the view on the surface for the client process and TextureHost the view on the surface for the compositor process. There is currently no common interface between TextureClient and TextureHost (They are both reference counted and offer locking semantics, but they don't share a common "SharedTextureObject" base class), because their is currently no code that is run both on the client and host sides. This tends to confuse some people so by popular demand we'll add a common base class (this could also help us implement main-thread compositing using the Compositor API). | |||
Very important: TextureClient and TextureHost are *NOT* the equivalent of SurfaceTexture. they don't have a swap chain, they don't encapsulate several buffers, only one. TextureClient and TextureHost are the equivalent of android::GraphicBuffer. Except that Gecko needs to work with GraphicBuffer, and EGLImage, and shmem, and D3D textures, etc. So TextureClient and TextureHost provide an abstraction that unifies textures backed by gralloc, shmem, etc. If you need to implement a swap chain, the swap chain should manipulate several TextureClient/Host, and not be implemented insede TextureClient. | |||
Also very important: TextureClient is *NOT* an implementation detail of gfx/layers/ that should be hidden from the rest of Gecko. Just like android::GraphicBuffer is not an implementation detail of SurfaceFlinger. If you are producing content that needs to be shared with the compositor process without copy, then you should be using one or several TextureClients. | |||
Almost as important: The two important items above are really important, so read them several times. | |||
=== New vs Deprecated textures === | === New vs Deprecated textures === | ||
Deprecated texture clients and host suffer from a mix of imperfect design decisions (design decisions that did not work well in the specific case of B2G's memory model coupled with some of the funky sharing in Gecko) and the pile of quick hacks that had to be done to keep them around. As a result, deprecated texture clients and host are hard to reason about and bug prone (especially on B2G). | |||
In order to incrementally fix the badness of TextureClient and TextureHost, the classes have been marked deprecated and new texture clients and host were designed, using a more strict and defined memory model. The goal is now to replace all the usage of DeprecatedTexture* by the new classes. When writing new code, if possible please use the new classes. | In order to incrementally fix the badness of TextureClient and TextureHost, the classes have been marked deprecated and new texture clients and host were designed, using a more strict and defined memory model. The goal is now to replace all the usage of DeprecatedTexture* by the new classes. When writing new code, if possible please use the new classes. | ||
| Line 23: | Line 30: | ||
That is a valid question. If you read code from other open-source compositors and compositor protocols such as SurfaceFlinger and Wayland, you will notice that they have simpler memory ownership models. In the case of SurfaceFlinger for instance, you have a simple producer/consumer model where ownership is transfered from one side to the other along with the data. Why not do the same in Gecko? Well, we tried and that's what the deprecated textures should have been. It turned out that the way Gecko works, we need to shared textures with the compositor while still being able to read it from *any* thread on the content side. | That is a valid question. If you read code from other open-source compositors and compositor protocols such as SurfaceFlinger and Wayland, you will notice that they have simpler memory ownership models. In the case of SurfaceFlinger for instance, you have a simple producer/consumer model where ownership is transfered from one side to the other along with the data. Why not do the same in Gecko? Well, we tried and that's what the deprecated textures should have been. It turned out that the way Gecko works, we need to shared textures with the compositor while still being able to read it from *any* thread on the content side. | ||
Example 1: the video pipeline. When a video frame is produced we try to put it in shared memory as early as possible to avoid making too many copies of the frame before sending it to the compositor. The video video frame is then passed to the compositor but the content side will keep a reference to it, because at any moment we can do screenshot of the page | Example 1: the video pipeline. When a video frame is produced we try to put it in shared memory as early as possible to avoid making too many copies of the frame before sending it to the compositor. The video video frame is then passed to the compositor but the content side will keep a reference to it, because at any moment we can do for example a screenshot of the page, or decide to synchonously pick the current frame and place it in a canvas. These operations need to access the shared texture data and are done synchronously on the content process while the data is being used by the compositor process. Also a given video frame (in a shmem) can be set in any order, at any time, to any number of ImageContainers and be, as a result, used by any number of layers. Tricky business. | ||
Example 2: Buffer rotation. We do an optimization with (non tiled) thebes layers in order to redraw as few pixels as possible. It is called buffer rotation and I won't explain it here but it needs to do a synchronization between the front and the back buffer in which we need to copy data from one to the other. So we send the front buffer to the compositor in a transaction and we read from it to write in the back buffer in the main thread, and swap. Again, here the texture data is read by both the content and the compositor side at the same time. | Example 2: Buffer rotation. We do an optimization with (non tiled) thebes layers in order to redraw as few pixels as possible. It is called buffer rotation and I won't explain it here but it needs to do a synchronization between the front and the back buffer in which we need to copy data from one to the other. So we send the front buffer to the compositor in a transaction and we read from it to write in the back buffer in the main thread, and swap. Again, here the texture data is read by both the content and the compositor side at the same time. | ||
There are other examples of this, but the bottom line is that in order to reduce the number of copies and the amount of memory we consume, we need to share things between a lot of threads simultaneously and this introduces more complicated ownership problems. It is not possible to fit a clean and simple memory model like SurfaceFlinger's without | There are other examples of this, but the bottom line is that in order to reduce the number of copies and the amount of memory we consume, we need to share things between a lot of threads simultaneously and this introduces more complicated ownership problems. It is not possible to fit a clean and simple memory model like SurfaceFlinger's without abandoning some optimization that we do and without changing some of the things Gecko does that have been designed before we thought of compositing on a separate thread. | ||
A notable difference between Gecko's compositing architecture and, say, android's, is that Gecko's texture sharing model is not taken into account in the way content is produced. | A notable difference between Gecko's compositing architecture and, say, android's, is that Gecko's texture sharing model is not taken into account in the way content is produced. Optimization like producing content directly in shared memory before giving it to the compositing system does not involve the compositing system itself. As a result it is very hard for the compositing system to be smart and take fast paths. Managing the texture memory outside of the compositing system, for instance, causes massive headaches to get things working without race conditions. Sharing the same texture data to several layers becomes hard, ect. | ||
=== who owns the shared data with the new textures? === | === who owns the shared data with the new textures? === | ||