Project Silk

From MozillaWiki
Jump to navigation Jump to search

Overview

The objective of Project Silk is to provide smoother scrolling and animation on B2G. To achieve the goal, it aligns HW Vsync signals among input, content painting and composition modules.

Architecture

Object diagram

Project Silk Object relation

Class diagram

Project Silk Object relation

VsyncDispatcher

VsyncDispatcherHost/Client stands at the key position of Project Silk. The main role of this object is

  1. Receive HW vsync-event from vsync driver.
  2. Real time dispatching vsync-event to registered-event listeners.

There is exactly one VsyncDispatcherHost object in chrome process and one VsyncDispatcherClient in each content process.

In chrome process

  • VsyncDispatcherHost creates a dedicated thread, named as vsync dispatcher thread, to dispatch vsync-event. Since vsync-event dispatching is very performance sensitive, we create a dedicate thread which only works on event dispatching to prevent transmission flow from being blocked by irrelevant tasks.
  • Receive HW vsync events from HWComposer. On platforms which have no HW vsync supported, it's fallback to SW vsync driver automatically.
  • Export registry functions to clients who want to receive vsync-event notification.
  • Sent vsync notifications to registered listeners, on the vsync dispatcher thread.

In Content process, what VsyncDispatcherClient does is relatively simpler:

  • Been created and initialized on main thread. The reason that we did not create a dedicated thread here is base on the fact: there is only one vsync listener in content side, which is nsRefreshDriver, and it can be kicked on main thread only.
  • Export registry/unregistry function to user clients who want to receive vsync-event notification.
  • In main thread, sent vsync notification to registered listeners.

GonkVsyncDispatcher(Gonk implementation of VsyncDispatcherHost)

GonkVsyncDispatcher which is derived from VsyncDispatcherHost contains the gonk specific implementation. Basically, it's created immediately in the first VsyncDispatcherHost::GetInstance calling.

The following lists the platform-dependency interfaces that are used in chrome 

  virtual void StartUpVsyncEvent() MOZ_OVERRIDE;
  virtual void ShutDownVsyncEvent() MOZ_OVERRIDE;
  virtual void EnableVsyncEvent(bool aEnable) MOZ_OVERRIDE;

Order of vsync-event dispatching

Threading Modal

Project Silk Threading modal

Our Current Design: 

  VsyncDispatcherHost:
    1. notify the main thread to handle input event.
    2. wait the input handling task done, then notify the compositor parent to do compose.
    3. send vsync event to content process.
    4. tick Chrome refresh driver
  VsyncDispatcherClient:
    1. tick Content refresh driver


Why execute a sync InputDispatcher task on main thread, instead of vsync-dispatcher thread, in step 1?

The APZ event processing code all currently runs on the main thread. APZ can't process the input at non-main thread until we complete the Bug 930939. It will do the APZ off the main thread(it will update the APZ at VsyncDispatcher thread).

We also need to wait the APZ to update the frame matrix and forward any required input events to the child processes before we notify the compositor and refresh driver. Thus, we will have the most up to date touch coordinates rather than have a racy condition where we might process the previous touch event versus the latest touch events when we composite.

Note1: We need to wait the APZC processed the input event in step 1, then do the remaining task. We might have a large delay for context switch. Current Thread Model

Note2: If we have Bug 930939, the order will be: 

  VsyncDispatcherHost:
    1. APZ processs input event at VsyncThread.
    2. compositor parent to do compose.
    3. send vsync event to content process.
    4. tick Chrome refresh driver
  VsyncDispatcherClient:
    1. tick Content refresh driver

Idel Thread Model

Object life cycle

VsyncDispatcherHost/Client is created as a singleton object in each process.

  • VsyncDispatcherHost's life cycle in chrome process
    • Create in gfxPlatform::Init
    • Release in gfxPlatform::Shutdown
  • VsyncDispatcherClient's life cycle in content process
    • Create in PVsyncEventChild::Create(VsyncEvent protocol creation). Since we define PVsyncEvent as a top level protocol, VsyncDispatcherClient is constructed in the early stage of content process initialization.
    • Release in PVsyncEventChild::ActorDestroy(VsyncEvent protocol destroy)

Vsync event protocol

PVsyncEvent, a new top level protocol, is used to pass Vsync events from chrome to content process, and also provides the register/unregister interfaces for content process to receive Vsync events. In PVsyncEventChild side, it is running at the main thread of content process. In PVsyncEventParent side, it is running at the VsyncDispatcher thread of chrome process. Inside chrome, we dispatch the Vsync event to chrome's refresh driver directly without using PVysncEvent protocol(step 5 in above architecture diagram). 

  IPC interfaces
    child:
      NotifyVsyncEvent(VsyncData aVsyncData);

    parent:
      RegisterVsyncEvent();
      UnregisterVsyncEvent();

HWComposer2D

Here HWComposer2D is responsible to notify Vsync events to VsyncDispatcher. Android HWC only supports HW Vsync after android JB. Therefore, we also port SW Vsync into HWComposer2D. If there is no HW Vsync support, the SW Vsync will be used. During initialization phase, VsyncDispatcher registers a Vsync notification callback into HWComposer2D and this callback will be called for every Vsync events. If there is no Vsync observer, we can disable the Vsync notifications inside HWComposer2D.

HW Vsync versus SW Vsync

SW Vsync is a periodic timer which calls it's callback function with 60Hz frequency. Currently, according to the WIP patch in bug 1043822, we use nanosleep() to implement the timer loop and the most parts of this timer are very similar with those in Android's SW Vsync timer. Our SW Vsync provides some APIs, such as 

 void StartTimer();
 void StopTimerAsync();
 void StopTimerSync();
 double GetRate() const;

In the normal cases, EnableVsyncEvent() will use StartTimer() and StopTimerAsync() to control this timer. StopTimerSync() is called while we want to shut down VsyncDispatcher because VsyncDispatcher should wait until SW Vsync finishes all it's job. Enabling/Disabling the Vsync module is controlled by HwComposer2D right now, and we think users shouldn't worry about which one (HW or SW Vsync) is using, so we have to use a generic API for HW Vsync and SW Vsync. For example, EnableVsync() in HwcComposer2D is the generic function which handles the enable/disable of Vsync event. We can hide the logic of the decision between HW and SW Vsync in this function. In other words, if we are using HW Vsync, EnableVsync() will call eventControl() to handle it; if we are using SW Vsync, EnableVsync() will call the APIs of SW Vsync timer. By this design, GonkDisplayJB() can also use HwcComposer2D::EnableVsync() to turn on/off the Vsync event according to the status of the display.

Power on/off screen

We can't control the HW Vsync event on/off during power off. We will always turn off the Vsync event at power off and force to turn on at power on. That's ok if VsyncDispatcher receive an unexpected Vsync event at power on. After receiving the Vsync event, we can just disable the HW Vsync again.

Testing Plan

Unit test

  • VsyncDispatcher API
    • Registry - vsync listener should be able to receive sync-notification steadily after registry.
    • Unregistry - vsync listener should not receive any vync notification after synchronize unregistry from VsyncDispatcher

Integration test

    • First paint - have a test case to make sure we do not break first-paint.

Benchmark

Type:

  • requestAnimationFrame, WebGL
  • OMTA test
  • scrollable layer

Project Silk Discussion at Taipei Workweek

Project Silk

Bugs

New/Open

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);


Resolved

Full Query
ID Priority Summary Status Assigned to Target milestone Last change time
930939 P1 Separate the threads that run gecko and that receive input events from the OS RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com) mozilla38 2018-10-31T13:12:35Z
980241 P2 Vsync-triggered RefreshDriver RESOLVED Jerry Shih[:jerry] (UTC+8) (inactive) --- 2014-06-12T18:07:26Z
982972 P2 investigate using RT priority in compositor on gonk RESOLVED Mason Chang [Inactive] [:mchang] --- 2014-08-15T00:11:44Z
987523 P2 Vsync-triggered CompositorParent RESOLVED Jerry Shih[:jerry] (UTC+8) (inactive) --- 2015-01-29T14:26:00Z
987527 P1 Register Vsync monitor to HWComposer RESOLVED Boris Chiou [:boris] 2.1 S3 (29aug) 2018-10-31T13:12:35Z
987529 P1 Integrate vsync dispatcher on B2G RESOLVED Jerry Shih[:jerry] (UTC+8) (inactive) 2.1 S4 (12sep) 2018-10-31T13:12:35Z
990832 P1 Build a tool to Visualize Frame Uniformity with ScrollGraph Output RESOLVED Jinho Hwang [:jeffhwang] mozilla33 2014-08-20T17:42:53Z
991420 P1 (silk) IPC Shows 1-2ms latency sending message from b2g to content process and b2g main thread to composite thread RESOLVED Mason Chang [Inactive] [:mchang] 2.0 S4 (20june) 2015-01-29T01:23:23Z
991483 -- Vsync-triggered GeckoInputDispatcher RESOLVED Jinho Hwang [:jeffhwang] --- 2018-10-31T13:12:35Z
1035076 P2 Measure vsync event latency between VsyncDispatcher and other modules in Project Silk RESOLVED Jerry Shih[:jerry] (UTC+8) (inactive) 2.1 S5 (26sep) 2015-01-29T14:49:24Z
1042017 P3 Re-Examine the APZ kinetic Scroll Friction Animation RESOLVED Mason Chang [Inactive] [:mchang] --- 2014-11-11T17:39:43Z
1043822 P2 Support SW Vsync and combine it with HW Vsync RESOLVED Boris Chiou [:boris] 2.1 S3 (29aug) 2018-10-31T13:12:35Z
1048667 P1 Implement Vsync dispatch framework RESOLVED Jerry Shih[:jerry] (UTC+8) (inactive) 2.1 S2 (15aug) 2018-10-31T13:12:35Z
1062117 -- Support the priority change for VsyncDispatcher RESOLVED Boris Chiou [:boris] --- 2018-10-31T13:12:35Z
1062331 -- Turn on by default touch event smoothing with gfx.frameuniformity.hw-vsync and gfx.touch.resample RESOLVED Mason Chang [Inactive] [:mchang] mozilla38 2018-10-31T13:12:35Z
1072627 P1 Hardware Composer Vsync Events aren't stable RESOLVED Mason Chang [Inactive] [:mchang] --- 2018-10-31T13:12:35Z
1073545 -- Add Vsync Markers to SPS Profiler RESOLVED Mason Chang [Inactive] [:mchang] mozilla35 2014-10-06T13:48:12Z
1073704 -- Input touch resampling is broken RESOLVED Michael Wu [:mwu] mozilla35 2018-10-31T13:12:35Z
1078136 -- Tune the thread priority for VsyncDispatcher thread RESOLVED --- 2018-10-31T13:12:35Z
1078152 P2 Provide performance label for frame uniformity RESOLVED Jerry Shih[:jerry] (UTC+8) (inactive) --- 2018-03-02T21:34:10Z
1078160 -- Provide an automatic testing environment for QA to check the project silk performance RESOLVED --- 2015-03-25T02:43:27Z
1078168 P3 Measure "where is my water" game overall performance with project silk RESOLVED --- 2015-12-14T06:01:05Z
1078171 P3 Measure "cut the rope" game overall performance with project silk RESOLVED --- 2015-12-14T06:00:56Z
1083530 -- Refactor GeckoTouchDispatcher to use mozilla::TimeStamp instead of nsecs_t for vsync times RESOLVED Mason Chang [Inactive] [:mchang] mozilla36 2018-10-31T13:12:35Z
1092245 -- Create a benchmark to measure refresh driver alignment to vsync and detect regressions RESOLVED Mason Chang [Inactive] [:mchang] --- 2017-01-30T16:53:10Z
1092978 P1 Integrate refresh driver for silk RESOLVED Jerry Shih[:jerry] (UTC+8) (inactive) mozilla38 2015-01-29T01:22:03Z
1094760 -- Compositor will sleep for a long time during composing if we align composing and refresh driver tick with vsync RESOLVED Jerry Shih[:jerry] (UTC+8) (inactive) --- 2015-04-08T01:46:04Z
1095242 -- Disable Vsync When Idle RESOLVED Mason Chang [Inactive] [:mchang] mozilla38 2015-02-22T14:59:55Z
1098701 -- [Silk] Gtest for project Silk RESOLVED Mason Chang [Inactive] [:mchang] mozilla38 2015-01-25T05:32:44Z
1101974 P1 Create a single VsyncDisptacher per nsBaseWidget RESOLVED Mason Chang [Inactive] [:mchang] mozilla37 2014-12-19T15:30:03Z
1102200 -- Composing time becomes much longer randomly when we unplug the usb cable on flame-kk device RESOLVED --- 2022-05-30T19:26:41Z
1102631 -- Create a software vsync timer RESOLVED Mason Chang [Inactive] [:mchang] --- 2016-07-26T21:48:15Z
1117870 -- Let the CompositorVsyncObserver unobserve vsync after a configurable number of vsyncs have passed RESOLVED Mason Chang [Inactive] [:mchang] mozilla37 2022-01-03T17:02:05Z
1118530 -- [meta] Turn on by default hardware vsync and the Vsync Aligned Compositor on b2g RESOLVED Mason Chang [Inactive] [:mchang] mozilla38 2015-07-18T14:29:51Z
1123734 -- Notify global VsyncSource in hwc RESOLVED Jerry Shih[:jerry] (UTC+8) (inactive) mozilla38 2018-10-31T13:12:35Z
1123762 -- [META] Turn on vsync-aligned refresh driver by default on b2g RESOLVED Mason Chang [Inactive] [:mchang] mozilla39 2022-06-01T22:16:00Z
1130678 -- dom/broadcastchannel/tests/test_broadcastchannel_sharedWorker.html intermittent with vsync refresh driver RESOLVED Mason Chang [Inactive] [:mchang] mozilla38 2022-06-01T22:14:38Z
1139253 -- Reuse the same thread when we start/stop vsync event for software vsync timer RESOLVED Jerry Shih[:jerry] (UTC+8) (inactive) 2.2 S8 (20mar) 2015-03-10T02:04:26Z

38 Total; 0 Open (0%); 38 Resolved (100%); 0 Verified (0%);


FAQ

Platform supporting plan
We are going to support Firefox OS first and extend this framework to other platforms after.
Can Silk improve rendering performance?
No
Retrieved from "https://wiki.mozilla.org/index.php?title=Project_Silk&oldid=1008795"