QA:Home Page:Firefox 3.0 TestPlan:Leaks:LeakTesting-How-To

Introduction

Mozilla QA uses Trace Refcnt Debug Builds to test Firefox Components and Extensions for leaks. One part of this testing is manual testing. You can find the QA Firefox 3 Leak Testing Testplan here.

Many thanks to David Baron, Jonas Sicking, Jesse Ruderman and Peter Van der Beken for the help on this Project.

Leak Gauge

For testing Firefox and Extensions for Memory we recommend "Leak Gauge" developed by David Baron. Leak Gauge is designed to help testers isolate and simplify testcases for many classes of leaks (those that involve large graphs of core data structures) in Mozilla-based browsers. It is designed to print information about what has leaked by processing a log taken while running the browser. Such a log can be taken over a long session of normal browsing and then the log can be processed to find sites that leak. Once a site is known to leak, the logging can then be repeated to figure out under what conditions the leak occurs.

Leak Gauge is easy to use and you don't need to download any additional Software or Tool (like needed for Debug Builds).

Leak Gauge will create a leak log file. You can upload this Log File here to get a analysis if you have run into a memory leak. There are plans to move this upload later to a better location like AMO.

Using Leak Gauge

As mentioned before, you don't need to install additional software to run Leak Gauge, however some simple steps are needed to run Leak Gauge and to get the Leak Log.

1. Setting up the Environment Variable:

The way to create the log is to set the environment variables:

NSPR_LOG_MODULES=DOMLeak:5,DocumentLeak:5,nsDocShellLeak:5,NodeInfoManagerLeak:5
NSPR_LOG_FILE=nspr.log     (or any other filename of your choice)

in your shell and then run the program.

• In a Windows command prompt, set environment variables with

set VAR=value

• In an sh-based shell such as bash, set environment variables with

export VAR=value

• In a csh-based shell such as tcsh, set environment variables with

 setenv VAR value

As example for Windows, this should look like:

• 1. Start the Windows Command Prompt (Start -> Run -> cmd.exe or Start -> Programs -> Accessories -> Command Prompt

• 2. Set the env Variable:

C:\>set NSPR_LOG_MODULES=DOMLeak:5,DocumentLeak:5,nsDocShellLeak:5,NodeInfoManagerLeak:5

and now you need to define the Location of the Leak Log File:

C:\>set NSPR_LOG_FILE=c:\leak1.log

(or any other location/file name of your choice)

• 3. Start Firefox from the same command line and your leak log will be created.

Note:

• When you test Firefox or your Extensions for leaks, you need to enter the command above every time you start the Console/Terminal

• You may add the environmental variables to the environmental variables in the Windows Control Panel to run Firefox from the Windows GUI instead of the command line.

• Once you have this log from a complete run of the browser, you have to exit; otherwise it will look like everything leaked

• Every time you start Firefox after you have defined the environment variables, your existing leak log will be overwritten. So if you want to reproduce a leak, or when you want to save a leak log, you need to change the file name (like from leak1.log to leak2.log) next time before you run Firefox.

See also the steps for testing for more Information.

The Leak Gauge Leak Log File

You have setup the environment variables and you got a leak.log, Fine ! Now you need to upload this leak log on this site.

After you have uploaded a leak log, the site will display the output of the results from the leak log, which will tell you which of certain core objects leaked and the URLs associated with those objects.

A normal output, without any leaking object / memory leak should look like:

Results of processing log leak.log :
Summary:
Leaked 0 out of 11 DOM Windows
Leaked 0 out of 44 documents
Leaked 0 out of 3 docshells
Leaked content nodes in 0 out of 0 documents

A example output with leaking objects / memory leaks :

Results of processing log leak2.log :
Leaked outer window 2c6e410 at address 2c6e410.
Leaked outer window 2c6ead0 at address 2c6ead0.
Leaked inner window 2c6ec80 (outer 2c6ead0) at address 2c6ec80.
Summary:
Leaked 13 out of 15 DOM Windows
Leaked 35 out of 46 documents
Leaked 4 out of 4 docshells
Leaked content nodes in 42 out of 53 documents

If you see such a results, please see the "steps for testing" and the "how to file a leak bug section".

Debug Builds

You can also use debug builds for leak testing. Building and Running a Debug Build is more complex then using Leak Gauge. If you interested in running Leak Test Debug Builds you can find here more information.

Other Leak Tools

Please see this page when you are interested in the other leak testing tools we provide.

Leak Testing - Testing steps for Firefox / Firefox Extension Testing

First of all, its important to have some steps to reproduce, so developers can reproduce this leak and also for verifying a fixed leak bug. So if you can reproduce a memory leak 2 or more times, it should be filed as bug, see the "How to file a Memory Leak Bug ?" section.

Some general Tips for testing Firefox Extensions for Memory Leaks:

• First create a new Firefox Profile to ensure nothing else from a used profile cause a leak. Start your Leak Debug as described.
• Install the Extension and Restart Firefox
• After the Restart - Quit Firefox to make sure the Extension does not leak after installation in general
• Even when no Memory Leak was detected on start, use the features of this the Extension (in example with the QA Extension, use the Chat now button, the Login into Litmus etc)
• Close Firefox and upload the Leak Log here

• When a memory leak was detected save the leak.log/set the Leak Log environment variable to a different file name and start Firefox again to verify your steps to reproduce.

Note:: If the browser crashes during the run, the log is pretty much useless, since many objects will have been allocated but not freed at the time the crash occurs.

How to file a Memory Leak Bug ?

• When you have found a Memory Leak Bug in Firefox and you have solid steps to reproduce or a testcase, please file a Bug in Bugzilla.

Note: If you are unsure if you have found a memory leak, or you are not familiar with Bugzilla, you can also ping Tomcat or send a mail to tomcat@mozilla.com and we can recheck this leak and file the leak bug

• When you are unsure if its your extension that leak or Firefox itself, the best component in Bugzilla is Core:General
• Please add mlk as keyword when you file the bug. Also please request blocking Blocking 1.9? for this bug and set this Bug Blocking Bug 422471. Bug 422471 is the Tracking Bug for Memory Leaks found by the AMO-Editor Team/Extension Developers.
• Please include the steps to reproduce/the testcase to the bug. This help developers to reproduce and fix the bug.
• Please add the leak log / output from the Leak Gauge Script to the Bug.
• It would great if you cc Tomcat to the Bug. You can do this via entering just :tomcat in the cc field in bugzilla or the full email address cbook@mozilla.com

Questions / Feedback

In Case you need help or when you have questions regarding the leak testing of your extension, feel me to ping me on irc.mozilla.org, my nickname is "Tomcat" or send me an e-mail.