Performance:Leak Tools: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Replaced content with "This documentation has moved to [https://developer.mozilla.org/en-US/docs/Mozilla/Performance MDN].")
 
(93 intermediate revisions by 18 users not shown)
Line 1: Line 1:
== What leak tools do we have? ==
This documentation has moved to [https://developer.mozilla.org/en-US/docs/Mozilla/Performance MDN].
 
When trying to make a particular testcase not leak, I recommend focusing first on the largest object graphs (since these entrain many smaller objects), then on smaller reference-counted object graphs, and then on any remaining individual objects or small object graphs that don't entrain other objects.
 
=== Leak tools for large object graphs ===
 
==== leak-gauge ====
 
Leak gauge is a script (available as a [http://lxr.mozilla.org/mozilla/source/tools/footprint/leak-gauge.pl?raw=1 perl script] or as [http://lxr.mozilla.org/mozilla/source/tools/footprint/leak-gauge.html?raw=1 HTML and JavaScript that must be run from a local file]) that post-processes an log taken by setting environment variables in a release build. It is designed to assist in detecting what leaks of large object graphs occur during normal browsing activity. The logging can be run (as described in the script) during normal browsing without significant overhead. Then the script can be run to provide information about the documents, window objects, and docshells that leaked. See [http://www.squarefree.com/2006/01/13/memory-leak-detection-tool/ Jesse Ruderman's blog entry] for more details including a list of other links.
 
==== leak-monitor ====
 
[http://dbaron.org/mozilla/leak-monitor/ Leak Monitor] is an extension that detects (when closing windows) the primary ways in which JavaScript code can cause leaks that are not bugs in the native code core. (An example of such a bug would be registering (in each window) a JavaScript-implemented observer with the global observer service and never removing the observers, leaving JavaScript code running in the context of each window registered as an observer until the user quits the browser.) When a leak is detected, the extension presents the user (or application/extension developer!) with a dialog with information about the leaked objects. (The alerts can also be triggered by bugs in the core code. See the [http://dbaron.org/mozilla/leak-monitor/ extension's homepage] for more details.)
 
==== Cycle Collector Debugging (DEBUG_CC) ====
 
For leaks involving windows, documents, or content nodes (the first two of which show up with leak-gauge), and for many other large graphs of objects that are cycle collected, the cycle collector debugging code can be useful.
 
The cycle collector debugging code can be enabled by uncommenting the <code>#define DEBUG_CC</code> lines in '''both''' <code>xpcom/base/nsCycleCollector.h</code> and <code>xpcom/glue/nsCycleCollectionParticipant.h</code>.
 
The cycle collector debugging code relies on being told that cycle-collected objects are expected to be garbage.  This is done through the functions <code>nsCycleCollector_DEBUG_shouldBeFreed</code> and <code>nsCycleCollector_DEBUG_wasFreed</code>.  When using the cycle collector debugging, it may be helpful to add additional calls to these functions (or maybe even remove some of the existing ones if they add noise).  It may also be useful to look only at the cycle collection that happens at shutdown (which can be done by surrounding the call to <code>ExplainLiveExpectedGarbage</code> in <code>nsCycleCollector::Collect</code> with <code>if (origTryCollections > 1)</code>.
 
When this code is enabled, it prints an explanation of what is keeping alive any objects that are expected to be garbage but are not collected.  This means it will print information about what objects from which they are reachable have reference counts that the cycle collector does not know about; you can then debug the leak starting from these objects.  It also prints the path from the object keeping this alive to the closest object that it is keeping alive, and if the object keeping things alive is a JS object, it prints all the objects from which that object is reachable.
 
==== JavaScript heap dump ====
 
Setting the <code>XPC_SHUTDOWN_HEAP_DUMP</code> environment variable to a file name will cause XPConnect to, at shutdown, dump a log to that file explaining why all remaining JS objects are still alive.  This may be sufficient to debug some JS-related leaks without the full rebuild required by DEBUG_CC.  It also shows more information about reachability, since it shows property names for all connections.
 
=== Leak tools for medium-size object graphs ===
 
==== [http://mozilla.org/performance/refcnt-balancer.html Refcount balancer] ====
 
The refcount balancer consists of the [http://lxr.mozilla.org/seamonkey/source/xpcom/base/nsTraceRefcntImpl.cpp nsTraceRefcnt] code in [http://lxr.mozilla.org/seamonkey/source/xpcom/base/ mozilla/xpcom/base/] and the perl scripts in [http://lxr.mozilla.org/mozilla/source/tools/rb/ mozilla/tools/rb/]. It works by instrumenting refcounting, so every AddRef and Release method created using NS_IMPL_ISUPPORTSn and variants, NS_IMPL_ADDREF, or NS_IMPL_RELEASE is automatically instrumented. It also logs information for non-refcounted objects instrumented using the MOZ_COUNT_CTOR and MOZ_COUNT_DTOR macros.
 
Because it is based on instrumentation, it is not reliable for gathering aggregate statistics. (In spite of this, it is currently used for the leak stats on tinderbox, although a trace-malloc based replacement is in development.) However, it is by far the best tool we have for debugging leaks of reference counted objects, which are the leaks in Mozilla that can entrain the largest object graphs.
 
These tools work on Windows, Mac (PPC and Intel), and Linux (x86), although  nsCOMPtr logging doesn't work on Windows (?) and Mac and Linux stack traces require some post-processing (see [[#Post-processing of stack traces|below]]). In optimized builds they can be built with --enable-logrefcnt.
 
See below for a detailed description of how to use the refcount balancer effectively.
 
==== leaksoup ====
 
leaksoup is a trace-malloc tool that analyzes the log from trace-malloc's secondary feature, the ability to dump allocations.  (See below for more information on using trace-malloc.)  trace-malloc's main log is a log that contains information about all allocations and the stacks at which they were allocated and freed, but it also has the ability to dump, at a given time (including shutdown), all the currently live allocations, the stacks at which they were allocated, ''and the contents of the memory''. This last feature allows leaksoup to analyze the graph of live objects and determine which allocations are roots (within that graph, of course -- stack allocations and global variables don't count). Leaksoup also finds sets of objects that are rooted by a cycle (i.e., a set of reference counted objects that own references to each other in a cycle). However, it cannot distinguish between owning and non-owning pointers, which means that non-owning pointers that are nulled out by destructors may show up in leaksoup as cycles. However, despite that, it is probably the easiest way to determine what leak roots are present.
 
To create a memory dump using trace-malloc, run a build using ''both'' the --trace-malloc and --shutdown-leaks options, for example "./mozilla -P default --trace-malloc=malloc.log --shutdown-leaks=sdleak.log". Ignore the malloc.log file (unless you're interested in other trace-malloc tools, such as [http://mozilla.org/projects/footprint/spaceTrace.html SpaceTrace]). Then run leaksoup over the memory dump (which is a dump of all allocations still live at shutdown) with a command such as ./run-mozilla.sh ./leaksoup sdleak.log > sdleak.html. This generates a ''large'' HTML file as output.
 
The output of leaksoup begins with all the leak roots, and then lists all the non-root allocations. The roots are either listed as single objects or as strongly connected components (minimal sets of nodes in the graph in which any node is reachable from all other nodes). (A strongly connected component with only one node is listed as a single object.) Any single object listed as a root is really a leak root, and any component listed as a root either (a) contains an object that is a root or (b) contains objects that form an ownership cycle that is a root.
 
=== Leak tools for simple objects and summary statistics ===
 
==== Trace-malloc ====
 
The trace-malloc code consists of the nsTraceMalloc code in [http://lxr.mozilla.org/mozilla/source/tools/trace-malloc/lib/ mozilla/tools/trace-malloc/lib/] and the perl scripts and trace-malloc readers in [http://lxr.mozilla.org/mozilla/source/tools/trace-malloc/ mozilla/tools/trace-malloc/]. It works by overriding / hooking into malloc and free (and related functions) and logging all calls, with stacks. It is probably the best tool we currently have for gathering aggregate memory usage statistics. It is generally more useful for bloat measurement than leak measurement, but it can be used to find leaks. It works on Windows, Linux (x86), and Mac (PPC and Intel).  It can be built by checking out mozilla/tools/trace-malloc/ and building with --enable-trace-malloc.
 
==== Boehm GC ====
 
The Boehm GC is a garbage collector for C/C++ programs that can be used to debug leaks in a program that does not require a garbage collector. It can be used with leaksoup or leak-soup.pl to find the roots of graphs of leaked objects. The leak detection tools find leaks using garbage collection while the app is running (which tends to hide many of the largest leaked object graphs, which can usually be reached through global variables). Some #defines can be flipped in xpcom/base/nsLeakDetector.cpp to cause it to print out all objects leaked at shutdown (into a different file). These shutdown leaks will include both one-time shutdown leaks (which aren't really a serious problem, but make it harder to find real leaks) and real leaks that had previously been rooted (as far as the GC was concerned) through a global variable. Some of the trace-malloc tools imitate its output format and therefore some of the processing tools are compatible. It works on Mac (Classic) and Linux (although slightly better on Mac). On Linux it can be built with --enable-boehm. On Mac, (?).
 
'''Update (2006-06-13): The code integrating the Boehm GC in Mozilla has not been tested for a number of years and is unlikely to work anymore.'''
 
==== Purify ====
 
Purify is a commercial tool that detects a number of types of problems, including leaks. I think it works internally rather like the Boehm GC. That is, it reports a subset of real leaks during the running of the app (while hiding many real leaks because they are rooted through globals), and then reports all leaks at shutdown. Things it reports as potential leaks (PLKs) should be ignored -- they just mean that an object is being held only by an pointer to an offset within it (which happens when an XPCOM object implements multiple interfaces (that don't inherit from each other) and is held only by a pointer to an interface other than the first). It is available on Windows and Solaris.
 
==== Valgrind ====
 
[http://valgrind.org/ Valgrind] is a free tool for Linux that (like Purify) detects a number of program errors at runtime. Like Purify, its leak reporting is only useful for the simplest types of leaks (those not involving reference counting or large object graphs).
 
==== Apple tools ====
 
Apple provides [http://developer.apple.com/documentation/Performance/Conceptual/ManagingMemory/Articles/FindingLeaks.html some tools] for Mac OS X that probably report similar problems to those reported by Purify and Valgrind.
 
=== Leak tools for debugging memory growth that is cleaned up on shutdown ===
 
== Debugging leaks using the refcount balancer ==
 
'''Strategy for finding leaks'''. Because (1) large graphs of leaked objects tend to include some objects pointed to by global variables that confuse GC-based leak detectors, which can make leaks look smaller (as in [https://bugzilla.mozilla.org/show_bug.cgi?id=99180 bug 99180]) or hide them completely and (2) large graphs of leaked objects tend to hide smaller ones, it's much better to go after the large graphs of leaks first. A good general pattern for finding and fixing leaks is to start with a task that you want not to leak (for example, reading email). Start finding and fixing leaks by running part of the task under nsTraceRefcnt logging, gradually building up from as little as possible to the complete task, and fixing most of the leaks in the first steps before adding additional steps. (By most of the leaks, I mean the leaks of large numbers of different types of objects or leaks of objects that are known to entrain many non-logged objects such as JS objects. Seeing a leaked GlobalWindowImpl, nsXULPDGlobalObject, nsXBLDocGlobalObject, or nsXPCWrappedJS is a sign that there could be significant numbers of JS objects leaked.) For example, start with bringing up the mail window and closing the window without doing anything. Then go on to selecting a folder, then selecting a message, and then other activities one does while reading mail (deleting messages, etc.). Once you've done this, and it doesn't leak much, then try the action under Purify or the Boehm GC to find the leaks of smaller graphs of objects. (When I refer to the size of a graph of objects, I'm referring to the number of objects, not the size in bytes. Leaking many copies of a string could be a very large leak, but the object graphs are small and easy to identify using GC-based leak detection.)
 
It is also possible to have a leak that is visible to the user (but not to many of our leak detection tools) by holding objects longer than one should. (For example, one could store an owning reference to every document ever loaded in an nsISupportsArray owned by a service that is destroyed at shutdown, causing every document to stay around until shutdown.) It is sometimes worth testing for this type of leak, especially if there are known leak problems that are visible to the user. The best way I know to do this is with [http://www.mozilla.org/performance/leak-brownbag.html#enabling-trace-malloc trace-malloc]. In a Linux build with trace-malloc enabled you can dump the existing allocations to a file by calling the function TraceMallocDumpAllocations from JavaScript (which is equivalent to calling NS_TraceMallocDumpAllocations from C. The following web page will allow dumping of allocations:
 
<script type="text/javascript">
    var filename = window.prompt("Filename to log: ");
    if (filename)
      TraceMallocDumpAllocations(filename);
</script>
 
One can then use the script mozilla/tools/trace-malloc/diffbloatdump.pl to compare trace-malloc dumps before and after doing an action that might leak. If there are significant differences, it might be worth examining the call stacks for the destructors of the objects in question to see what is extending their lifetime. (This can be done by examining the unprocessed output of an XPCOM_MEM_REFCNT_LOG, one of the nsTraceRefcnt logs.)
 
'''Using the refcount balancer'''. See the [http://www.mozilla.org/performance/leak-tutorial.html tutorial on finding leaks of XPCOM objects] for information finding leaks using the refcount balancer. (Be aware that the later sections on leak graphs that include JS objects have been out of date since the XPCDOM landing.)
 
'''Common leak patterns'''. When trying to find a leak of reference counted objects, there are a number of patterns that could cause the leak:
 
# Ownership cycles. The most common source of hard-to-fix leaks is ownership cycles. If you can avoid creating cycles in the first place, please do, since it's often hard to be sure to break the cycle in every last case. Sometimes these cycles extend through JS objects (discussed further below), and since JS is garbage-collected, every pointer acts like an owning pointer and the potential for fan-out is larger. See [https://bugzilla.mozilla.org/show_bug.cgi?id=106860 bug 106860] and [https://bugzilla.mozilla.org/show_bug.cgi?id=84136 bug 84136] for examples.
# Dropping a reference on the floor by:
## Forgetting to release (because you weren't using nsCOMPtr when you should have been): See [https://bugzilla.mozilla.org/show_bug.cgi?id=99180 bug 99180] or [https://bugzilla.mozilla.org/show_bug.cgi?id=93087 bug 93087] for an example or [https://bugzilla.mozilla.org/show_bug.cgi?id=28555 bug 28555] for a slightly more interesting one. This is also a frequent problem around early returns when not using nsCOMPtr.
## Double-AddRef: This happens most often when assigning the result of a function that returns an AddRefed pointer (bad!) into an nsCOMPtr without using dont_AddRef(). See [https://bugzilla.mozilla.org/show_bug.cgi?id=76091 bug 76091] or [https://bugzilla.mozilla.org/show_bug.cgi?id=49648 bug 49648] for an example.
## [Obscure] Double-assignment into the same variable: If you release a member variable and then assign into it by calling another function that does the same thing, you can leak the object assigned into the variable by the inner function. (This can happen equally with or without nsCOMPtr.) See [https://bugzilla.mozilla.org/show_bug.cgi?id=38586 bug 38586] and [https://bugzilla.mozilla.org/show_bug.cgi?id=287847 bug 287847] for examples.
# Dropping a non-refcounted object on the floor (especially one that owns references to reference counted objects). See [https://bugzilla.mozilla.org/show_bug.cgi?id=109671 bug 109671] for an example.
# Destructors that should have been virtual: If you expect to override an object's destructor (which includes giving a derived class of it an nsCOMPtr member variable) and delete that object through a pointer to the base class using delete, its destructor better be virtual. (But we have many virtual destructors in the codebase that don't need to be�don't do that.)
 
=== Debugging leaks that go through XPConnect ===
 
Many large object graphs that leak go through [http://www.mozilla.org/scriptable/ XPConnect]. This can mean there will be XPConnect wrapper objects showing up as owning the leaked objects, but it doesn't mean it's XPConnect's fault (although that [https://bugzilla.mozilla.org/show_bug.cgi?id=76102 has been known to happen], it's rare). Debugging leaks that go through XPConnect requires a basic understanding of what XPConnect does. XPConnect allows an XPCOM object to be exposed to JavaScript, and it allows certain JavaScript objects to be exposed to C++ code as normal XPCOM objects.
 
When a C++ object is exposed to JavaScript (the more common of the two), an XPCWrappedNative object is created. This wrapper owns a reference to the native object until the corresponding JavaScript object is garbage-collected. This means that if there are leaked GC roots from which the wrapper is reachable, the wrapper will never release its reference on the native object. While this can be debugged in detail, the quickest way to solve these problems is often to simply debug the leaked JS roots. These roots are printed on shutdown in DEBUG builds, and the name of the root should give the type of object it is associated with.
 
One of the most common ways one could leak a JS root is by leaking an nsXPCWrappedJS object. This is the wrapper object in the reverse direction -- when a JS object is used to implement an XPCOM interface and be used transparently by native code. The nsXPCWrappedJS object creates a GC root that exists as long as the wrapper does. The wrapper itself is just a normal reference-counted object, so a leaked nsXPCWrappedJS can be debugged using the normal refcount-balancer tools.
 
If you really need to debug leaks that involve JS objects closely, you can get detailed printouts of the paths JS uses to mark objects when it is determining the set of live objects by using the functions added in {{bug|378261}} and {{bug|378255}}.  (More documentation of this replacement for GC_MARK_DEBUG, the old way of doing it, would be useful.  It may just involve setting the XPC_SHUTDOWN_HEAP_DUMP environment variable to a file name, but I haven't tested that.)
 
== Post-processing of stack traces ==
 
On Mac and Linux, the stack traces generated by our internal debugging tools don't have very good symbol information (since they just show the results of <code>dladdr</code>).  The stacks can be significantly improved (better symbols, and file name / line number information) by post-processing.  Stacks can be piped through the scripts <code>mozilla/tools/rb/fix-linux-stack.pl</code> or <code>mozilla/tools/rb/fix-macosx-stack.pl</code> to do this.  These scripts are designed to be run on balance trees in addition to raw stacks; since they are rather slow, it is often better to generate balance trees (e.g., using <code>make-tree.pl</code> for the refcount balancer or <code>diffbloatdump.pl --use-address</code> for trace-malloc) and ''then'' run the balance trees (which are much smaller) through the post-processing.
 
== Getting symbol information for system libraries ==
 
=== Windows ===
 
Setting the environment variable <code>_NT_SYMBOL_PATH</code> to something like <code>symsrv*symsrv.dll*f:\localsymbols*http://msdl.microsoft.com/download/symbols</code> as described in [http://support.microsoft.com/kb/311503 Microsoft's article].  This needs to be done when running, since we do the address to symbol mapping at runtime.
 
=== Linux ===
 
Many Linux distros provide packages containing external debugging symbols for system libraries.  fix-linux-stack.pl uses this debugging information (although it does not verify that they match the library versions on the system).
 
For example, on Fedora, these are in *-debuginfo RPMs (which are available in yum repositories that are disabled by default, but easily enabled by editing the system configuration).
 
== Leak statistics on tinderbox ==
 
=== Reading the old-style leak stats ===
 
[ This section was updated on 2006-06-13 to indicate how these leak tests are currently presented. ]
 
The old-style (nsTraceRefcnt-based) leak stats look like this:
 
<table style="margin: 1em auto" border="1" width="120"><tr><td style="background: #1d1; text-align: center; font: medium monospace; padding: 2em 1em"><span style="text-decoration:underline; color: #00e; cursor:pointer">L</span> <span style="text-decoration:underline; color: #00e; cursor:pointer">C</span><br>RLk:700B</td></tr></table>
 
These statistics are collected using nsTraceRefcnt, which as I said above is not very good for aggregate statistics. The action tested is loading of a browser window and a run through the bloat URLs (bloaturls.txt). The RLk (leak) number is the number of bytes of leaks of objects that are logged by nsTraceRefcnt. This is just a subset of objects -- it includes only those objects that use NS_IMPL_ISUPPORTSn and friends or MOZ_COUNT_CTOR and MOZ_COUNT_DTOR. Therefore it doesn't include many of the largest objects, such as string buffers, and it accounts for the size of some other objects incorrectly.
 
=== Running the old-style leak tests ===
 
The old-style tests can be run on any standard --enable-debug build, or on any --disable-debug build with --enable-logrefcnt.
 
==== Mozilla Suite ====
 
# set the environment variable XPCOM_MEM_LEAK_LOG to leak.log (or XPCOM_MEM_BLOAT_LOG to bloat.log)
# ./mozilla -f bloaturls.txt
# Look at the top line of bloat.log or leak.log for the aggregate statistics (under headers Bytes/Leaked (for leaks) and Objects/Total (for "bloat"), and look at the other lines for the summary of objects.
 
==== Firefox ====
 
# load resource:///res/bloatcycle.html and tell the popup blocker to allow popups from it
# Edit the preferences file or use about:config to set the pref "dom.allow_scripts_to_close_windows" to true
# set the environment variable XPCOM_MEM_LEAK_LOG to leak.log (or XPCOM_MEM_BLOAT_LOG to bloat.log)
# ./firefox -no-remote resource:///res/bloatcycle.html
# Look at the top line of bloat.log or leak.log for the aggregate statistics (under headers Bytes/Leaked (for leaks) and Objects/Total (for "bloat"), and look at the other lines for the summary of objects.
 
=== Reading the new-style leak stats ===
 
The new-style (nsTraceMalloc-based) leak stats look like this:
 
<table style="margin: 1em auto" border="1" width="120"><tr><td style="background: #1d1; text-align: center; font: medium monospace; padding: 2em 1em"><span style="text-decoration:underline; color: #00e; cursor:pointer">L</span> <span style="text-decoration:underline; color: #00e; cursor:pointer">C</span><br>Lk:382KB<br>MH:7.75MB<br>A:391K</td></tr></table>
 
These statistics are generated using trace-malloc. They therefore give accurate aggregate statistics for all heap allocations during the test. Like the old-style leak statistics, the action tested is loading of a browser window and a run through the bloat URLs (bloaturls.txt). The Lk (leak) number is the total number of bytes (not counting any overhead in the allocator) allocated on the heap and not freed over the entire run. This number (as does the number in the old-style leak statistics) includes shutdown leaks, leaks that happen only once for a run of the browser, but there are more here since most shutdown leaks are not of objects logged by nsTraceRefcnt. The MH (max heap, or a meaningful bloat number that I gave a name other than �bloat”) is the number of bytes allocated on the heap at the point during the run when the heap was at its maximum size (again, excluding overhead). The A (allocations) number is the total number of allocations over the run, and is an indicator of a subset of performance rather than an indicator of memory use, although high allocation churn could contribute to fragmentation.
 
=== Running the new-style leak tests ===
 
Currently the new-style leak stats can be run only on Linux. Hopefully they will work on other platforms soon. They require a build with trace-malloc enabled.
 
Build with trace-malloc enabled (--enable-trace-malloc in your mozconfig file). Then, once you have a build, use the same steps as the old-style leak tests, except don't set the environment variables, and instead add the command line options when invoking mozilla or firefox: "--trace-malloc=malloc.log" (or, if you want the shutdown leaks report, also add "--shutdown-leaks=sdleak.log").  Then, to process the log, run "./run-mozilla.sh ./leakstats malloc.log".  (Omit the "./run-mozilla.sh" on Windows.)
 
This will produce a report like the following:
 
Leaks: 382739 bytes, 3465 allocations
Maximum Heap Size: 7751799 bytes
62095212 bytes were allocated in 391091 allocations.
 
== Other References ==
* [[Performance:Tools|Performance tools]]
* [http://people.mozilla.com/~dbaron/leak-screencasts/ Leak Debugging Screencasts] (dbaron)
* [[LeakingPages]] - the least of pages known to leak

Latest revision as of 22:57, 28 October 2014

This documentation has moved to MDN.

Retrieved from "https://wiki.mozilla.org/index.php?title=Performance:Leak_Tools&oldid=1028960"