DevTools/Memory/Roadmap

Overview

We want to ship memory tools built-in to the Firefox Developer Tools for web and firefox developers to identify, locate, and resolve memory issues related to memory bloat and leaks. If the memory tool can inform, educate and empower developers in each of these 3 stages, then we believe that this tool will successfully solve memory bloat and leak.

Originally, this tool was going to handle all issues involving memory — in 2014, it was agreed upon that this memory tool will focus on memory bloat and leaks, while the Performance tool handles performance issues, including issues that arise from GC pauses from allocations. The performance tool will be used to *identify* memory issues in the first place, as a springboard into using the memory tools for further debugging.

The platform work for the memory tool was started in 2014, with initial requirements gathering and work on the memory APIs. At the time of writing (April 2015), we now flesh out the use-cases and requirements on the front end experience.

Tool Stages

Description of the debugging stages, with which features can solve different issues along the way. View Feature Descriptions sections for definitions of the components of memory tools used in this document.

Stage 1: Identify

It should be easy to discover when there is a memory problem in the current target. This can be as obvious as an OOM crash, or a more subtle leak. For identifying leaks, the performance tool’s timeline can be used to observe memory growth over time, and observing GC events not reducing memory consumption. For bloat issues, the memory tool itself should be used to identify types of objects that are unreasonably large.

Stage 2: Locate

A Heap View can be used to identify areas that are taking up more than expected amounts of memory by viewing the heap, sorting by size or allocations, and grouping by type/class. With similar sorting and grouping capabilities, a heap diff view can illustrate uncollected objects after a cycle, highlighting suspiciously large areas of memory that have not been collected. For finding accumulation points, the dominator view can also be used.

Stage 3: Resolve

Dominator tree views and path (from GC root) views are two ways to provide enough context of a memory issue to fix it.

Allocations by frame (currently in performance) and allocations by type/time view can also be used for additional information on where certain allocations are occurring if needed.

Users/Use Cases

• Web developers wanting to identify what’s using memory on their web app, as well as in real time, by type
• FxOS developers targeting low-memory devices wanting to use as little memory as possible, monitoring memory consumption with a “high-water mark” in mind.
• FxOS developers wanting to debug memory issues on device/simulator

• Firefox engineers wanting to solve memory issues in Firefox with browser toolbox
• Web developers, Firefox engineers wanting to solve short-term memory leak issues
• compare memory between two points in time
• Should provide ways via console/gcli to show paths to objects that are expected to be collected

Need to turn into actionable issues

• Finding longterm memory leaks via just snapshots. — Chrome has their “recording” and snapshots completely separated, IE11/VS2015 have them rolled together. Maybe we can get creative on how to do this.
• When a GC event occurs, I should be able to see the objects being collected and where they were created and where they were released. (current performance allocations view kind of does this; we can show were objects are created, but cannot show what got collected AFAIK)
• show when there are orphaned DOM fragments
• show memory/count of event handlers

Requirements

meta: Memory Tools v1, Fx44

Memory Tools Component in Bugzilla

QUESTIONS

• Can we get an allocations by time/type view via census timeline, and using heap diffs on censuses?
• Should we show allocation views in memory, even if redundant with performance tool?
• What “types” of objects can we classify in a high-level timeline? (Chrome has heap, document, nodes, event listeners)
• can we get info about when a specific object got collected? IIRC, Fitzgen said once an object is GC’d we can’t get info on it.
• If we have a “timeline”, frequently taking a census of the heap, we can either store just the first and most recent state, or store each census. How large are these censuses?
• Ideas on “leak hunter” tool? Eclipse MAT is interesting. Definitely v2.

---

• What “type” of information do we get from censuses? constructor name, allocation site (if enabled), primitives, etc
• What “types” of objects can we classify in a high-level timeline? (Chrome has heap, document, nodes, event listeners)
• How large is a census? For gmail?
• How large is a snapshot? For gmail?
• can we get info about when a specific object got collected? IIRC, Fitzgen said once an object is GC’d we can’t get info on it. Can we however, track the absence of an object? So, IDing objects, and having 1+ states to compare to. Census? Snapshot?
• Memory Profiler - What’s the overlap here? What goals are redundant with this, and/or the performance allocation view? (bug)

Performance Tools

• GC/CC Markers in timeline
• Allocation pressure graph

Memory Tools

• Heap View
• Heap Diff View
• Dominator Tree View
• Force GC
• Paths from GC View
• Allocations by frame ?
• Allocations by time/type ?
• Memory Recording Timeline ?
• Memory Timeline: Show Total Process Consumption (like current performance tool)
• Memory Timeline: Ability to add a high-water mark
• Memory Timeline: inspect heap state along timeline
• Memory Timeline: diff heap state via timeline selection

Additional Tools

• console method to find leaked objects’ paths
• console method to report retained size of object

Additional Info

• Fitzgen on the platform memory API (2014)
• Debugger.Memory API

Design

Platform APIs

Platform APIs are being tracked in (Meta) DevTools platform APIs for heap / memory analysis. The platform APIs generally fall into one of two categories: (1) GC related APIs for debugging pauses originating from the collector, or (2) analyses of the heap graph (live heap or serialized snapshot).

GC APIs

The various GC APIs are exposed on SpiderMonkey's Debugger API, under the Debugger.Memory qualifier. Documentation for Debugger.Memory APIs.

These APIs are concerned with

• Why a GC was triggered (hint: allocating too much stuff)

• What are the allocations that applied pressure on the collector (or, which allocations collectively triggered a GC)

• Monitoring how long GCs are taking, whether they are incremental or not, etc

Each of these APIs is centered around helping developers minimize GC pauses in their application.

Heap Graph Analyses

Given a heap graph, we can run various analyses on it that give us different bits of information. Note that the heap graph can be either the live heap graph or a serialized heap snapshot. We can run our analyses on either heap graph. Depending on the analysis, and how expensive computing it is, it may make sense to prefer the live heap graph or a serialized heap snapshot for any given analysis. Heavier, more expensive computations may benefit from being run exclusively on heap snapshots because the heap snapshot can be deserialized in a worker thread and the analysis run there so that it doesn't jank the main thread. As a rule of thumb, if the analysis is cheaper than taking a heap snapshot, run it on the live heap graph. Otherwise, when the analysis is more expensive than taking a heap snapshot, take the heap snapshot and then run the analysis in a worker thread.

Dominator Trees

If x dominates y, then any path from the global window to y must pass through x. We can use this information in two practical ways:

1. If you nullify all references to x, every y such that x dominates y will also become unreachable and will eventually be garbage collected.

1. We can calculate the retained size of x. That is, the amount of memory that will be reclaimed if x (and therefore also every y such that x dominates y) were to be garbage collected.

It also lets us sort by retained size and present the dominator tree directly. This lets developers easily navigate the most expensive (in terms of retained size) objects in their heap.

Breadth First Search

By doing a BFS in the heap graph from the GC roots to any given object, we find the shortest retaining path(s) for that object.

In the frontend, we can use these paths to construct a developer-friendly label for that object. Often the label we provide will be a snippet of JavaScript that can be evaluated in the console. For example: "window.MyApp.WidgetView.element". Other times, we will be forced to display labels that cannot be evaluated in the console: "window.requestAnimationFrame renderLoop.closure environment .player.sprite".

Mockups

Latest Sketches

PDF (sorry!): File:Memory-tool-mockup-sketch.pdf

Each section below describes one facet of those sketches.

Overview

This is primarily driven by the Debugger.Memory.prototype.takeCencus API. Gives an overview of the contents of the heap over time. Aggregate counts and aggregate shallow (not retained) sizes, along with percentages, and a + or - indicating whether it is growing or shrinking.

You will be able to group by allocation site, type (Array, DOMException, js::Shape, etc), constructor, prototype, etc.

Feedback given by njn:

One thing I've learned over the years is that everybody always thinks they want a graph showing memory usage vs. time, but in practice such graphs are pretty but largely useless. Because all you ever do with them is find the peak usage point and then look at that point in detail. Because what's the point of optimizing anything other than the peak?

This implies two things (IMO):

• The graph stuff is much less important than the snapshot stuff.
• The ability to automatically get a snapshot at peak usage (or close to peak, because getting the exact peak is tricky) is useful.

Response to feedback by fitzgen:

Agreed that peak is most important, but the graphs help you do two things:

• Identify the peak.
• Correlate that peak with interactions or events on the page.

Perhaps we don't need so many categories and the overview, but just a simple size-of-tab graph.

Automatic heap snapshots when near the peak sounds great.

Allocations and GC

Because this view is primarily about minimizing GC pauses (and jank associated therein), this should be part of the performance panel, rather than the memory panel.

Anyways, this overlays GCs on top of a graph of the rate of allocation. Below is the allocations tree (and we could make it toggle to a flame graph/chart and make the tree invert-able, too). Size and count of allocations are shown as well as the percentage of allocations.

This is using the allocations log and Debugger.Memory's onGarbageCollection hook.

Heap Snapshots: Dominator Tree

Computing dominator trees is expensive enough that we can only really do it when the user explicitly asks for it (unlike taking a census), so that's why it is part of heap snapshots. This is a view of the things in the heap sorted by biggest retained size, and the set of things they are each keeping alive in turn.

Heap Snapshots: Retaining Paths

This view shows you the N shortest paths from a GC root to a given GC thing. Unsure how a thing's dominator is keeping it alive and feel like it should have been collected? Here are all the ways. This is probably a side panel of the dominator tree, rather than its own tab in the tool.

Older Mockups

Pretty, but aren't as clearly solving our user stories / requirements. Not sure what everything here is supposed to be.

• Original mockup
• interactive wireframes

Feature Description

A collection of feature descriptions referenced in the prior art, as well as requirements. These are features common in many other memory tools, to establish terminology used, not necessarily for implementing in the Firefox memory tools.

Heap View

This is different in every tool, but usually a high level look at a heap snapshot, able to be sorted by class/type/allocation site/module, etc., with allocation counts, size, overall memory consumption, and a first step to discovering where a memory issue is hiding.

Dominator View

A view showing which objects dominate other objects. In OptimizeIt, this is known as a reduced reference graph.

Paths View

A view showing, from the GC root, a path of object references leading to the object in question.

Force GC

Utility to force garbage collection.

Leak Detection

This is pretty different in each tool, if it exists. Usually a smart analysis with some heuristics to point the developer to a potentially leaky object.

Heap Diffs

View that compares the difference between two heaps, usually showing difference in size and class counts of objects.

Allocations By Frame

A stack view with allocations performed in each frame — similar to the current Firefox Developer Tools performance allocation view.

Allocations By Time/Type

A slice of allocations that occurred in a timerange, able to view by type (like a heap view), like Chrome’s allocation timeline.

Timeline/Milestones

• Requirements: Mid Q2
• Ship: Mid Q3

Prior Art

There are many other memory profiling tools out there; when researching this, we look at not only other browser developer tools, but also other environments’ tooling, like Java. Many of the tools contain many of the same features, listed below in the overview graph, with unique features for each product explained after.

Comparison

Other Memory Tools

Tool	Heap View	Dominator View	Paths View	Force GC	Leak Detect	Heap Diffs	Allocs by stack	Allocs by time/type
YourKit Java Profiler	✓	✓	✓		✓	✓	✓	✓
Eclipse MAT (tut)	✓	✓	✓		✓	✓
Borland OptimizeIt	✓	✓	✓	✓	✓	✓	✓	✓
Chrome DevTools	✓	✓	✓	?		✓		✓
IE11 DevTools	✓	✓	✓			✓
VS2015	✓	✓	✓			✓

YourKit Java Profiler

• Can snapshot on high memory load
• Has a Merged Paths view, that’s similar to path from GC view, but does not show path of individual objects, but shows paths from multiple objects grouped by class.
• Has “incoming references” view, inverse of references.
• Has automation inspection view, which highlights potential memory issues. Not sure what all this provides, but looks like a good “coach” for memory issues.
• Has GC view showing times of GC, length of GC, and GC in the context of a call tree

Eclipse MAT

• Histogram tool (heap view), dominator view, path to GC, etc.
• leak hunter tool
• Interesting SQL-like query for the heap.
• Seems like a very complicated heap diff mode.

Borland OptimizeIt

• The paths view (reference graph) can be inverse (outgoing reference graph)
• Allocation Backtrace - Can view any class/type of object and list all allocation sites for that type, sorted by number
• Aggregated Graph View - A graph visualization of the call stack, with each node being a frame, with red nodes indicating the frame allocates objects (the more saturated the red, the more allocations in this frame)
• Can disable GC.

Chrome DevTools

• Has summary view (heap view), diff two snapshots in comparison view, path view via containment view, and a dominator view in the heap snapshot profiler.
• Has a heap allocations timeline that displays allocation pressure, color coding whether or not the objects are still alive at the current time (or at the end of the recording). Uses the same views as the memory profiler to compare the heap snapshots during the timeline.
• Chrome Timeline can record memory usage over time (by very high level type).
• Does not have explicit GC — but GC’s on memory snapshot. Timeline tool has a force GC, however.

IE11/VS2015

• Seem to be based on the same infrastructure.
• Caller/callee mode (inverting), like incoming/outgoing reference paths.
• “Just my Code” (VS2015) and “Show built-ins” (IE11), a way to hide platform data.
• Toggleable ability to display circular references
• VS2015 - can set a threshold line in memory timeline — great for a “memory emulator” feature?
• Very beautiful overview page showing diffs in heap snapshots during the timeline recording.

Current Mozilla Memory Tools

• Memory reporting APIs
• about:memory
• Memory Profiler (deprecated addon)
• GC/CC dump analyzers
• Dark Matter Detector