MozillaQualityAssurance:Home Page:Firefox 3.0 TestPlan:Leaks:LeakTesting-How-To: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
Line 1: Line 1:
= Firefox 3 Leak Testing How-To  =
= Firefox 3 Leak Testing How-To  =
Note: this is a draft !


= Introduction=
= Introduction=
Line 5: Line 7:
Mozilla QA uses Trace Malloc Debug Build to test Firefox Components and Extensions for leaks. One part of this testing is manual testing. You can find the QA Firefox 3 Leak Testing Testplan [http://wiki.mozilla.org/MozillaQualityAssurance:Home_Page:Firefox_3.0_TestPlan:Leaks here].  
Mozilla QA uses Trace Malloc Debug Build to test Firefox Components and Extensions for leaks. One part of this testing is manual testing. You can find the QA Firefox 3 Leak Testing Testplan [http://wiki.mozilla.org/MozillaQualityAssurance:Home_Page:Firefox_3.0_TestPlan:Leaks here].  


== Builds ==
== Leak Gauge ==
 
For testing Firefox and Extensions for Memory we recommend "Leak Gauge" developed by [http://dbaron.org/ David Baron]. Leak Gauge is designed to help testers isolate and simplify testcases for many classes of leaks (those that involve large graphs of core data structures) in Mozilla-based browsers. It is designed to print information about what has leaked by processing a log taken while running the browser. Such a log can be taken over a long session of normal browsing and then the log can be processed to find sites that leak. Once a site is known to leak, the logging can then be repeated to figure out under what conditions the leak occurs.
 
Leak Gauge is easy to use and '''you don't need''' to download any additional Software or Tool (like needed for Debug Builds). 
 
Leak Gauge will create a leak log file. You can upload this Log File [http://mxr.mozilla.org/mozilla/source/tools/footprint/leak-gauge.html here] to get a analysis if you have run into a memory leak. There are plans to move this upload later to a better location like AMO.
 
=== Using Leak Gauge ===
 
As mentioned before, you don't need to install addional software to run Leak Gauge, however some simple tests are needed to run Leak Gauge and to get the Leak Log.
 
1. Setting up the Environment Variable:


The Debug Builds for Leak Testing need to be build individually.
The way to create the  log is to set the environment variables:


Debug Builds (for Leak Testing) are much bigger then normal optimized Firefox Builds. A Debug Leak Testing can be need up to 1,2 GB Disk Space.
NSPR_LOG_MODULES=DOMLeak:5,DocumentLeak:5,nsDocShellLeak:5,NodeInfoManagerLeak:5
NSPR_LOG_FILE=nspr.log    (or any other filename of your choice)


There is right now '''no''' provided Leak Testing Build on the Mozilla FTP Server.  
in your shell and then run the program.


For Building and Running Debug Builds please see the [http://developer.mozilla.org/en/docs/Build_Documentation Build Documentation] provided on [http://developer.mozilla.org developer.mozilla.org]. There are Instruction available for [http://developer.mozilla.org/en/docs/Windows_Build_Prerequisites Windows], [http://developer.mozilla.org/en/docs/Linux_Build_Prerequisites Linux] and [http://developer.mozilla.org/en/docs/Mac_OS_X_Build_Prerequisites Mac OSX].
* In a Windows command prompt, set environment variables with
set VAR=value


If you build a Debug Build, you need to have trace-malloc activated in this builds. The .mozconfig file we use can be found for [http://people.mozilla.com/~cbook/memleak/mozconfigs/mozconfig-windows Windows XP], [http://people.mozilla.com/~cbook/memleak/mozconfigs/mozconfig-firefox-mac-i386-debug Mac OS 10.4] and [http://people.mozilla.com/~cbook/memleak/mozconfigs/mozconfig-firefox-linux-i686-debug Linux] here.
* In an sh-based shell such as bash, set environment variables with
export VAR=value


= Running a Leak Testing Build =
* In a csh-based shell such as tcsh, set environment variables with
  setenv VAR value


Open a Terminal window (or xterm or Gnome Terminal) and enter:
As example for Windows, this should look like:
* 1. Start the Windows Command Prompt (Start -> Run -> cmd.exe or Start -> Programs -> Accessories -> Command Prompt


export XPCOM_MEM_LEAK_LOG=~/tmp/leak.log
* 2. Set the env Variable:


'''Note that the directory you drop your leak.log into should already  
  C:\>set NSPR_LOG_MODULES=DOMLeak:5,DocumentLeak:5,nsDocShellLeak:5,NodeInfoManagerLeak:5
exist, so if it's not there, please create it first.'''


Run the debug build of firefox creating a new profile. For mac, this 
and now you need to define the Location of the Leak Log File:
would be:


  cd /location of your debug build/firefox-debug/dist/
  C:\>set NSPR_LOG_FILE=c:\leak1.log
FirefoxDebug.app/Contents/MacOS
(or any other location/file name of your choice)


./firefox-bin -P
* 3. Start Firefox and your leak log will be created.


linux (and when you use Mozilla Build on Windows) is:
Note:  


cd /location of your debug build/firefox-debug/dist/bin
* When you test Firefox or your Extensions for leaks, you need to type the command above every time you start the Console/Terminal


./firefox -P
* Once you have this log from a complete run of the browser, you have
to exit; otherwise it will look like everything leaked


* Every time you start Firefox after you have defined the environment variables, your existing leak log will be overwritten. So if you want to reproduce a leak, or when you want to save a leak log, you need to change the filename (like from leak1.log to leak2.log) next time before you run Firefox.


See also the steps for testing for more Information.


At the end of the run, the contents of ~/tmp/leak.log should have 
=== The Leak Gauge Leak Log File===
something in it.


= How do i know when i have found a leak ? =
You have setup the environment variables and you got a leak.log, Fine ! Now you need to upload this leak log on this [http://mxr.mozilla.org/mozilla/source/tools/footprint/leak-gauge.html site].


When you run Firefox like described above and quit Firefox later you get a Statistic in the Console where you have started Firefox.
After you have uploaded a leak log, the site will display the output of the results from the leak log, which will tell you which of certain core objects leaked and the URLs associated with those objects.


If no leak was found the output looks like the following:
A normal output, without any leaking object / memory leak should look like:


  nsStringStats
  Results of processing log leak.log :
=> mAllocCount:         79737
  Summary:
  => mReallocCount:         4990
  Leaked 0 out of 11 DOM Windows
  => mFreeCount:          79737
  Leaked 0 out of 44 documents
  => mShareCount:          58374
  Leaked 0 out of 3 docshells
  => mAdoptCount:          4593
  Leaked content nodes in 0 out of 0 documents
  => mAdoptFreeCount:      4593


and the leak.log stays empty.


When you run into a Memory leak the Stats look as example like this:


nsStringStats
== Debug Builds ==
=> mAllocCount:          32807
=> mReallocCount:        3994
=> mFreeCount:          32805  --  LEAKED 2 !!!
=> mShareCount:          30349
=> mAdoptCount:          3524
=> mAdoptFreeCount:      3524


When you run into a leak you see also a lot of information in the leaks.log. This log is needed for debugging and fixing the leak.
You can also use debug builds for leak testing. Building and Running a Debug Build is more complex then using Leak Gauge. If you interested in running Leak Test Debug Builds you can find [http://wiki.mozilla.org/MozillaQualityAssurance:Home_Page:Firefox_3.0_TestPlan:Leaks:LeakTesting-How-To:Debug_Builds here] more information.


= Leak Testing  - Example of testing steps for Extension Testing =
= Leak Testing  - Example of testing steps for Extension Testing =

Revision as of 21:46, 18 February 2008

Firefox 3 Leak Testing How-To

Note: this is a draft !

Introduction

Mozilla QA uses Trace Malloc Debug Build to test Firefox Components and Extensions for leaks. One part of this testing is manual testing. You can find the QA Firefox 3 Leak Testing Testplan here.

Leak Gauge

For testing Firefox and Extensions for Memory we recommend "Leak Gauge" developed by David Baron. Leak Gauge is designed to help testers isolate and simplify testcases for many classes of leaks (those that involve large graphs of core data structures) in Mozilla-based browsers. It is designed to print information about what has leaked by processing a log taken while running the browser. Such a log can be taken over a long session of normal browsing and then the log can be processed to find sites that leak. Once a site is known to leak, the logging can then be repeated to figure out under what conditions the leak occurs.

Leak Gauge is easy to use and you don't need to download any additional Software or Tool (like needed for Debug Builds).

Leak Gauge will create a leak log file. You can upload this Log File here to get a analysis if you have run into a memory leak. There are plans to move this upload later to a better location like AMO.

Using Leak Gauge

As mentioned before, you don't need to install addional software to run Leak Gauge, however some simple tests are needed to run Leak Gauge and to get the Leak Log.

1. Setting up the Environment Variable:

The way to create the log is to set the environment variables: 

NSPR_LOG_MODULES=DOMLeak:5,DocumentLeak:5,nsDocShellLeak:5,NodeInfoManagerLeak:5
NSPR_LOG_FILE=nspr.log     (or any other filename of your choice)

in your shell and then run the program.

  • In a Windows command prompt, set environment variables with
set VAR=value
  • In an sh-based shell such as bash, set environment variables with
export VAR=value
  • In a csh-based shell such as tcsh, set environment variables with
 setenv VAR value

As example for Windows, this should look like:

  • 1. Start the Windows Command Prompt (Start -> Run -> cmd.exe or Start -> Programs -> Accessories -> Command Prompt
  • 2. Set the env Variable:
C:\>set NSPR_LOG_MODULES=DOMLeak:5,DocumentLeak:5,nsDocShellLeak:5,NodeInfoManagerLeak:5

and now you need to define the Location of the Leak Log File: 

C:\>set NSPR_LOG_FILE=c:\leak1.log

(or any other location/file name of your choice)

  • 3. Start Firefox and your leak log will be created.

Note:

  • When you test Firefox or your Extensions for leaks, you need to type the command above every time you start the Console/Terminal
  • Once you have this log from a complete run of the browser, you have

to exit; otherwise it will look like everything leaked

  • Every time you start Firefox after you have defined the environment variables, your existing leak log will be overwritten. So if you want to reproduce a leak, or when you want to save a leak log, you need to change the filename (like from leak1.log to leak2.log) next time before you run Firefox.

See also the steps for testing for more Information.

The Leak Gauge Leak Log File

You have setup the environment variables and you got a leak.log, Fine ! Now you need to upload this leak log on this site.

After you have uploaded a leak log, the site will display the output of the results from the leak log, which will tell you which of certain core objects leaked and the URLs associated with those objects.

A normal output, without any leaking object / memory leak should look like: 

Results of processing log leak.log :
Summary:
Leaked 0 out of 11 DOM Windows
Leaked 0 out of 44 documents
Leaked 0 out of 3 docshells
Leaked content nodes in 0 out of 0 documents


Debug Builds

You can also use debug builds for leak testing. Building and Running a Debug Build is more complex then using Leak Gauge. If you interested in running Leak Test Debug Builds you can find here more information.

Leak Testing - Example of testing steps for Extension Testing

When you found a leak, its important to have some steps to reproduce, so developers can reproduce this leak and also for verifying a fixed leak bug.

As example here are the steps to reproduce from Bug 411576.

  • First create a new Firefox Profile to ensure nothing else from a used profile cause a leak. Start your Leak Debug as described.
  • Install a Extension and Restart Firefox
  • After the Restart - Quit Firefox to make sure the Extension does not leak after installation
  • When no memory leak was detected play a little with the extension to make sure the extension does not leak on specific steps to reproduce.

Some examples for in-detail testing of extensions:

  • Try out features of the Extension (like Buttons, Options, etc). After you have done this, quit firefox to check the leak.log
  • Go into the Extension Manager and check if the Firefox Extension Manager leaks when the Extension is installed
  • Use the Extension Options Button in the Extension Manager (see Bug 412136.
  • When a memory leak was detected save the leak.log and start Firefox again to check if the extension leaks only the first time after installation, or if the extension leak in general when installed
Retrieved from "https://wiki.mozilla.org/index.php?title=MozillaQualityAssurance:Home_Page:Firefox_3.0_TestPlan:Leaks:LeakTesting-How-To&oldid=83921"