Mobile/AsyncSubframePanning
From MozillaWiki
< Mobile
Contents
Goals
Phase 1
- land in 23 (not likely)
- B2G subframe scrolling
- Performance is
- as good as main frame
- subframes can't zoom
- seamed scrolling
Phase 2a
- seamless scrolling (needed for B2G dynamic toolbar)
Phase 2b
- Fennec subframe scrolling
Not doing
- scroll indicators are a non-goal
- not fixing page drift
- concurrently scrolling parents and ancestors
- subframe zooming
Plan
- To dos
- merge B2G APZC into fennec
- multiple display ports
- bind multiple display ports
- hit detection
- transform equation cleanup
- unit tests
- tuning/profiling
- concurrently scrolling subframes
Design
Step 1
Performed in Layout module, Main Gecko thread
- (Bug 866232) Layout (FrameLayerBuilder et al.) builds an initial simple layer tree with the position of the subframes tagged to the best fitting layer
- (See diagram for sample page & layer tree)
Step 2
Performed in the compositor's main thread
- Layer tree update/transaction is received by the compositor.
- Touch event is delivered at (6,51).
- (Bug 775452) Hit detection is performed on a layer tree in reverse depth-first order and matches 1st subframe.
- APZC decides of the appropriate buffer/pre-rendering and sends an IPC message to the main thread with an updated FrameMetrics/display port request.
- (See diagram for sample page & layer tree)
Step 3
Performed in Layout module, Main Gecko thread
- Display port is requested on the subframe in Layout
- (Bug 864447) Layer tree is grown to make room for a layer of the 1st subframe.
- Subframe is painted with a display port allowing Async scrolling. FrameMetrics are updated.
- (See diagram for sample page & layer tree)
Step 4
Performed in the compositor's main thread
- (Bug 866265) APZC can now perform Async panning on new sublayer.
Step 5
- After timeout APZC resets FrameMetrics/display port on the subframe removing the display port, the layer and releasing the memory usage.
Notes and diagram
Bugs
Tracking bug for async subframe scrolling: bug 775452
24 Total; 0 Open (0%); 24 Resolved (100%); 0 Verified (0%);
Tracking bug for allowing Fennec to use APZC: bug 776030
75 Total; 2 Open (2.67%); 71 Resolved (94.67%); 2 Verified (2.67%);
Meetings
2013-04-26
- initial meeting, created phases, goals and overall plan
2013-05-21
- Chris will take the display port work
2013-05-28
- Kats put up patch for disambiguated point classes
- will continue propegating the use of the classes through the
- should be up for review this week
- Chris should start on display ports per scroll frame this week
2013-06-04
- Kats
- propagating type changes to frame metrics and APZC code
- BenWa
- unit tests
- hit testing
- ajones
- multiple display ports
- code already largely supports it
- does metro need subframe scrolling?
- next items:
- hit testing (in progress, owned by BenWa)
- creating and managing APZCs from the frame loader builder (need owner)
2013-06-11
- Getting Jim and Brian up to speed, explaining how things work:
- Going through some of our "older" bugs, figuring out the history of how this came to be.
- C++ version that is now being used in Android, almost working completely
- In depth discussion on what the pieces are, how they work together
- Jim & Brian will work on getting single APZC without subframes on Metro. Primarily Brian.
- This will be based off C++ code which B2G uses, and there may be remaining issues in there for a desktop platform. We have to make sure this is tested as soon as it starts working so that we can flush out any remaining bugs.
- The work that Kats, BenWa, AJones are doing should then just click in place and give us subframe APZC on Metro as well.
2013-06-18
- Kats:
- There is another coordinate system that we have to worry about, and B2G seems to be very broken on high dpi devices. This is what the code looks like as well.
- We don't need high dpi for 1.1, so we probably don't need to uplift this. Need to test these changes, and then we should be OK.
- Benoit:
- On hit testing during the last week. More complicated test cases, with transforms, etc.
- Multiple display ports support in the layout, but it probably already supports it. We just need to use those assumptions and proceed.
2013-06-25
- BenWa:
- Pushed the hit testing on try, failed, sorting it out now.
- will finish hit testing, need to sort out gtest
- Kats:
- Sorted out regressions. Tests for bad transformation equations.
- Ran into CSS transforms problem (rotated iFrame, vertical moves end up being diagonal).
- http://people.mozilla.com/~kgupta/tmp/rotated.html
- This is not blocking shipping, Kats will spend no more that another day to fix it
- Blog post of current coordinate system: https://staktrace.com/spout/entry.php?id=800
- will be finishing up the tests
- then move on to managing display ports
2013-07-02
- done
- BenWa hit testing
- layout assumed that root frames were container frames, which isn't correct
- kats fixed high dpi event delivery
- two code paths (with root APZC and w/o APZC)
- seems like enforcing that there is always a root APZC would be the best route
- next
- BenWa will finish hit testing before going on PTO
- when he gets back from PTO, hook up hit testing
- Kats will make sure that there is no code assumeing that there is only one APZC
- will require making APZC owned by layer tree not compositor parent, compositor parent will hold a reference to layer tree
- Kats will defer continuing to work on input event transformation
2013-07-09
- done
- Kats
- We can have multiple PZCs now
- How do we (uniquely) identify scrollable layers? Need to use the layer ID and the scroll/frame ID, and presshell ID to guard against page navigation changes.
- Threading problem is under control by creating a separate tree of APZC instances that shadows the layer tree that is walkable on the UI thread
- This will tie in with dzbarsky's work
- BenWa
- Creating the scroll info layers is done, will be landed today
- Not much else to do at this point, blocked on kats
2013-07-16
- Need to discuss
- Requirements for dynamic toolbar that roc/Cwiiis planned out - this requires controlling which nested frame gets the scroll events
- roc/Cwiiis claim they depend on the APZC work. Unclear if they're blocked, and on which part.
- Controlling which nested frame gets scroll events seems to be the item that is blocking the above work. This is what we called "Phase 2, seamless scrolling". Looks like this should be the first thing in the second phase.
- What about Metro?
- In good shape, mostly working
- Scrolling properly
- Waiting for us to finish
- Brian was looking at axis locking
- What's next?
- Dynamic scrollbar
- Seamless scrolling
- B2G E-mail app (two frames)
- Kats
- Tree manager issues, timing of APZC being created. Even if message gets dropped, we seem to get the right information later, and will process it there.
- Still not clean, as we're dropping messages, so we may be looking for a better solution in the future.
- Porting hit testing patches from before, in progress. This is next. Hoping the input events aren't going to complicate things and that it should be as we had it before. The focus model needs to be sorted out as well.
- BenWa
- Transform shadow tree test, then sidetracked by a mobile bug. Will focus on the tests today.
2013-07-23
- status
- kats
- has patch for APZC per layer
- patch was expecting a layer tree, was actually getting indicator layers
- plan is to on-demand convert layers to scrollable layers
- currently working on making code in RenderFrameParent/TabParent/TabChild do the right thing in terms of scrolling the subframes when APZC asks it to
- BenWa is at SIGRAPH
- Botond
- patch to make APZC not break progressive tile painting
- on inbound for Android
- next steps
- Kats
- wrap up patch
- write more tests
- Botond
- making b2g APZC work with progressive tiles
- notes
- aiming to land everything next week. It'll be tight
2013-07-30
- status
- kats
- got review from BenWa, waiting for mattwoodrow
- landed a few follow up bugs
- most important is focus
- overscroll DIVs are broken, they're not being made scrollable layers
- bug 890290 function to convert widget space in input events needs to be modified to take async into account
- BenWa
- away at sigraph
- did reviews
- next
- get multi APZC landed
- figure out focus model
- figure out overscroll DIVs
2013-08-06
- status
- kats
- landed multiple APZC patch
- regression bug 900742 - input events don't always end up at the right place
- regression bug 899810 - on B2G and Metro where the screen goes blank
- landed multiple APZC patch
- next
- BenWa: bug 898443
- kats: bug 890280 & bug 901339
2013-08-13
- status
- kats
- input event targeting
- discussed a central gesture detector
- setting resolution on top level pres shell
- benwa
- (unrelated) b2g leo blocker
- botond
- metro apzc
- next steps
- kats
- several patches up for review, need to get them landed
- some should have some more testing before landing
- continue working on input events
- several patches up for review, need to get them landed
- benwa
- finish leo+ blockers
- do reviews
2013-08-20
- status
- kats
- landed a bunch of stuff for event targeting (bug 890280, bug 906109)
- filed a bug for disabling pinch zooming on subframes bug 906427
- have patches up for review for more event targeting bug 898443
- benwa
- camera crash bugs
- has his head around the crashes, should have patches soon
- camera crash bugs
- botond
- getting feedback on gaia patch
- looking at seemless panning transition
- next
- kats
- turn on async panning for all apps on b2g
- investigate overflow div panning
- need to look at
- display port tuning
- input events go through gecko thread
- would be better if input events didn't
2013-08-27
- status
- kats
- landed some focus model stuff (bug 898443, bug 906427, 907977)
- have a patch for bug 907754 (incorrect scaling in subframes) which needs cleanup. Also landed bug 885023 which this fix required
- looked at what things are like when APZC is turned on for all of gaia. Some weird issues but I will work through the big ones one at a time to see if they are hiding any other issues underneath
- next
- kats
- going to work on tracking down the overflow:scroll div problem
- work through the gaia issues I found (mostly using the contacts app as a test)
2013-09-03
- status
- kats
- landed 907754 (zoom, overflow documents issues)
- landing 910322 today after addressing some comments
- overflow scroll sub-documents problems - probably bug in flattening code, waiting for mwoodrow and tn
- BenWa
- not working on this last week
- botond
- handing overscroll to the parent layer; have the first implementation going
- will be looking at display port bugs
- composition bounds of iframes, same as what jmathies was running at - have an r+ from tn
- next steps
- kats
- may run out of stuff to look at - one bug to file, hit testing a bit off
- blocked on waiting for info and reviews
- notes
- kats - issues with tiling on fennec? wrong resolution, wrong sizes?
- benwa - you see progressive tile painting in the cases where you should not (e.g, updating existing content)
2013-09-10
- status
- kats
- implementing parent APZC
- talked to tn about overflow scroll problem
- frames construction isn't correct (there's a nested scroll frame)
- tn will think about it
- milan will follow up with tn tomorrow
- botond
- patch for scroll hand off posted
- issue with zoomed subframes
- moving on to annotating the browser iframe
- patch for scroll hand off posted
- BenWa did nothing of relevance
- next
- botond to annotate the browser iframe
2013-09-24
- status
- kats
- last week was mostly mobile work week
- continued working on bug 912657. input events are currently not being sent to the APZC in the root process so that will need fixing. currently they are delivered from gonk/nsWindow.cpp directly to gecko, gecko does some hit testing, and if they are in the area of the content process, *then* they go to the APZC. instead nsWindow should send it directly to the APZC first. i will work on this, building off the work jimm has done in bug 915213.
- botond
- at a C++ standardization conference this week
- last week he worked on bug 912666 and made good progress on special handling for the toolbar. the individual pieces seem to work but it depends on bug 912657 to tie it all together.
