MozillaQualityAssurance:Home Page:Firefox 3.0 TestPlan:Leaks:LeakTesting-How-To: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(6 intermediate revisions by 3 users not shown)
Line 1: Line 1:
{{draft}}
= Firefox 3 Leak Testing How-To  =
= Firefox 3 Leak Testing How-To  =
Note: this is a draft !


= Introduction=
= Introduction=
Line 5: Line 8:
Mozilla QA uses Trace Malloc Debug Build to test Firefox Components and Extensions for leaks. One part of this testing is manual testing. You can find the QA Firefox 3 Leak Testing Testplan [http://wiki.mozilla.org/MozillaQualityAssurance:Home_Page:Firefox_3.0_TestPlan:Leaks here].  
Mozilla QA uses Trace Malloc Debug Build to test Firefox Components and Extensions for leaks. One part of this testing is manual testing. You can find the QA Firefox 3 Leak Testing Testplan [http://wiki.mozilla.org/MozillaQualityAssurance:Home_Page:Firefox_3.0_TestPlan:Leaks here].  


== Builds ==
Many thanks to David Baron, Jonas Sicking, Jesse Ruderman and Peter Van der Beken for the help on this Project.
 
== Leak Gauge ==
 
For testing Firefox and Extensions for Memory we recommend "Leak Gauge" developed by [http://dbaron.org/ David Baron]. Leak Gauge is designed to help testers isolate and simplify testcases for many classes of leaks (those that involve large graphs of core data structures) in Mozilla-based browsers. It is designed to print information about what has leaked by processing a log taken while running the browser. Such a log can be taken over a long session of normal browsing and then the log can be processed to find sites that leak. Once a site is known to leak, the logging can then be repeated to figure out under what conditions the leak occurs.


The Debug Builds for Leak Testing need to be build individually.  
Leak Gauge is easy to use and '''you don't need''' to download any additional Software or Tool (like needed for Debug Builds).


Debug Builds (for Leak Testing) are much bigger then normal optimized Firefox Builds. A Debug Leak Testing can be need up to 1,2 GB Disk Space.  
Leak Gauge will create a leak log file. You can upload this Log File [http://mxr.mozilla.org/mozilla/source/tools/footprint/leak-gauge.html here] to get a analysis if you have run into a memory leak. There are plans to move this upload later to a better location like AMO.


There is right now '''no''' provided Leak Testing Build on the Mozilla FTP Server.
=== Using Leak Gauge ===


For Building and Running Debug Builds please see the [http://developer.mozilla.org/en/docs/Build_Documentation Build Documentation] provided on [http://developer.mozilla.org developer.mozilla.org]. There are Instruction available for [http://developer.mozilla.org/en/docs/Windows_Build_Prerequisites Windows], [http://developer.mozilla.org/en/docs/Linux_Build_Prerequisites Linux] and [http://developer.mozilla.org/en/docs/Mac_OS_X_Build_Prerequisites Mac OSX].
As mentioned before, you don't need to install additional software to run Leak Gauge, however some simple steps are needed to run Leak Gauge and to get the Leak Log.


If you build a Debug Build, you need to have trace-malloc activated in this builds. The .mozconfig file we use can be found for [http://people.mozilla.com/~cbook/memleak/mozconfigs/mozconfig-windows Windows XP], [http://people.mozilla.com/~cbook/memleak/mozconfigs/mozconfig-firefox-mac-i386-debug Mac OS 10.4] and [http://people.mozilla.com/~cbook/memleak/mozconfigs/mozconfig-firefox-linux-i686-debug Linux] here.
1. Setting up the Environment Variable:


= Running a Leak Testing Build =
The way to create the  log is to set the environment variables:


Open a Terminal window (or xterm or Gnome Terminal) and enter:
NSPR_LOG_MODULES=DOMLeak:5,DocumentLeak:5,nsDocShellLeak:5,NodeInfoManagerLeak:5
NSPR_LOG_FILE=nspr.log    (or any other filename of your choice)


export XPCOM_MEM_LEAK_LOG=~/tmp/leak.log
in your shell and then run the program.


'''Note that the directory you drop your leak.log into should already  
* In a Windows command prompt, set environment variables with
exist, so if it's not there, please create it first.'''
  set VAR=value


Run the debug build of firefox creating a new profile. For mac, this  
* In an sh-based shell such as bash, set environment variables with
would be:
  export VAR=value


cd /location of your debug build/firefox-debug/dist/
* In a csh-based shell such as tcsh, set environment variables with
FirefoxDebug.app/Contents/MacOS
  setenv VAR value


./firefox-bin -P
As example for Windows, this should look like:
* 1. Start the Windows Command Prompt (Start -> Run -> cmd.exe or Start -> Programs -> Accessories -> Command Prompt


linux (and when you use Mozilla Build on Windows) is:
* 2. Set the env Variable:  


  cd /location of your debug build/firefox-debug/dist/bin
  C:\>set NSPR_LOG_MODULES=DOMLeak:5,DocumentLeak:5,nsDocShellLeak:5,NodeInfoManagerLeak:5


./firefox -P
and now you need to define the Location of the Leak Log File:


C:\>set NSPR_LOG_FILE=c:\leak1.log
(or any other location/file name of your choice)


* 3. Start Firefox and your leak log will be created.


At the end of the run, the contents of ~/tmp/leak.log should have 
Note:
something in it.


= How do i know when i have found a leak ? =
* When you test Firefox or your Extensions for leaks, you need to type the command above every time you start the Console/Terminal


When you run Firefox like described above and quit Firefox later you get a Statistic in the Console where you have started Firefox.
* Once you have this log from a complete run of the browser, you have to exit; otherwise it will look like everything leaked


If no leak was found the output looks like the following:
* Every time you start Firefox after you have defined the environment variables, your existing leak log will be overwritten. So if you want to reproduce a leak, or when you want to save a leak log, you need to change the file name (like from leak1.log to leak2.log) next time before you run Firefox.


nsStringStats
See also the steps for testing for more Information.
=> mAllocCount:          79737
=> mReallocCount:        4990
=> mFreeCount:          79737
=> mShareCount:          58374
=> mAdoptCount:          4593
=> mAdoptFreeCount:      4593


and the leak.log stays empty.
=== The Leak Gauge Leak Log File===


When you run into a Memory leak the Stats look as example like this:
You have setup the environment variables and you got a leak.log, Fine ! Now you need to upload this leak log on this [http://mxr.mozilla.org/mozilla/source/tools/footprint/leak-gauge.html site].


nsStringStats
After you have uploaded a leak log, the site will display the output of the results from the leak log, which will tell you which of certain core objects leaked and the URLs associated with those objects.
=> mAllocCount:          32807
=> mReallocCount:        3994
=> mFreeCount:          32805  --  LEAKED 2 !!!
=> mShareCount:          30349
=> mAdoptCount:          3524
=> mAdoptFreeCount:      3524


When you run into a leak you see also a lot of information in the leaks.log. This log is needed for debugging and fixing the leak.
A normal output, without any leaking object / memory leak should look like:


= Leak Testing - Example of testing steps for Extension Testing =
  Results of processing log leak.log :
Summary:
Leaked 0 out of 11 DOM Windows
Leaked 0 out of 44 documents
Leaked 0 out of 3 docshells
Leaked content nodes in 0 out of 0 documents


When you found a leak, its important to have some steps to reproduce, so developers can reproduce this leak and also for verifying a fixed leak bug.
A example output with leaking objects / memory leaks :


As example here are the steps to reproduce from [https://bugzilla.mozilla.org/show_bug.cgi?id=411576 Bug 411576].  
Results of processing log leak2.log :
Leaked outer window 2c6e410 at address 2c6e410.
Leaked outer window 2c6ead0 at address 2c6ead0.
Leaked inner window 2c6ec80 (outer 2c6ead0) at address 2c6ec80.
Summary:
Leaked 13 out of 15 DOM Windows
Leaked 35 out of 46 documents
Leaked 4 out of 4 docshells
Leaked content nodes in 42 out of 53 documents
 
If you see such a results, please see the "steps for testing" and the "how to file a leak bug section".
 
== Debug Builds ==
 
You can also use debug builds for leak testing. Building and Running a Debug Build is more complex then using Leak Gauge. If you interested in running Leak Test Debug Builds you can find [http://wiki.mozilla.org/MozillaQualityAssurance:Home_Page:Firefox_3.0_TestPlan:Leaks:LeakTesting-How-To:Debug_Builds here] more information.
 
= Leak Testing  - Testing steps for Firefox / Firefox Extension Testing =
 
First of all, its important to have some steps to reproduce, so developers can reproduce this leak and also for verifying a fixed leak bug. So if you can reproduce a memory leak 2 or more times, it should be filed as bug, see ...
 
Some general Tips for testing Firefox Extensions for Memory Leaks:


* First create a new Firefox Profile to ensure nothing else from a used profile cause a leak. Start your Leak Debug as described.
* First create a new Firefox Profile to ensure nothing else from a used profile cause a leak. Start your Leak Debug as described.
* Install a Extension and Restart Firefox
* Install the Extension and Restart Firefox
* After the Restart - Quit Firefox to make sure the Extension does not leak after installation
* After the Restart - Quit Firefox to make sure the Extension does not leak after installation in general
* Even when no Memory Leak was detected on start, use the features of this the Extension (in example with the QA Extension, use the Chat now button, the Login into Litmus etc)
* Close Firefox and upload the Leak Log [http://mxr.mozilla.org/mozilla/source/tools/footprint/leak-gauge.html here]
 
* When a memory leak was detected save the leak.log/set the Leak Log environment variable to a different file name and start Firefox again to verify your steps to reproduce.
 
'''Note:''': '''If the browser crashes during the run, the log is pretty much useless, since many objects will have been allocated but not freed at the time the crash occurs.'''
 
== How to file a Memory Leak Bug ? ==
 
* When you have found a Memory Leak Bug in Firefox and you have solid steps to reproduce or a testcase, please file a Bug in Bugzilla.


** When no memory leak was detected play a little with the extension to make sure the extension does not leak on specific steps to reproduce.
''Note: If you are unsure if you have found a memory leak, or you are not familiar with Bugzilla, you can also ping Tomcat or send a mail to [mailto:tomcat@mozilla.com tomcat@mozilla.com] and we can recheck this leak and file the leak bug''


Some examples for in-detail testing of extensions:
* When you are unsure if its your extension that leak or Firefox itself, the best component in Bugzilla is Core:General
* Please add '''mlk''' as keyword when you file the bug.
* Try out features of the Extension (like Buttons, Options, etc)
* Please include the steps to reproduce/the testcase to the bug. This help developers to reproduce and fix the bug.
  After you have done this, quit firefox to check the leak.log
* Please add the leak log / output from the Leak Gauge Script to the Bug.
* It would great if you cc Tomcat to the Bug. You can do this via entering just ''':tomcat''' in the cc field in bugzilla or the full email address '''cbook@mozilla.com'''


* Go into the Extension Manager and check if the Firefox Extension Manager 
== Questions / Feedback ==
leaks when the Extension is installed


* Use the Extension Options Button in the Extension Manager (see 
In Case you need help or when you have questions regarding the leak testing of your extension, feel me to ping me on irc.mozilla.org, my nickname is "Tomcat" or send me an [mailto:tomcat@mozilla.com e-mail].
Bug [https://bugzilla.mozilla.org/show_bug.cgi?id=412136 412136].


** When a memory leak was detected save the leak.log and start Firefox again to check if the extension leaks only the first time after installation, or if the extension leak in general when installed
{{draft}}

Latest revision as of 20:39, 29 January 2009

Draft-template-image.png THIS PAGE IS A WORKING DRAFT Pencil-emoji U270F-gray.png
The page may be difficult to navigate, and some information on its subject might be incomplete and/or evolving rapidly.
If you have any questions or ideas, please add them as a new topic on the discussion page.

Firefox 3 Leak Testing How-To

Note: this is a draft !

Introduction

Mozilla QA uses Trace Malloc Debug Build to test Firefox Components and Extensions for leaks. One part of this testing is manual testing. You can find the QA Firefox 3 Leak Testing Testplan here.

Many thanks to David Baron, Jonas Sicking, Jesse Ruderman and Peter Van der Beken for the help on this Project.

Leak Gauge

For testing Firefox and Extensions for Memory we recommend "Leak Gauge" developed by David Baron. Leak Gauge is designed to help testers isolate and simplify testcases for many classes of leaks (those that involve large graphs of core data structures) in Mozilla-based browsers. It is designed to print information about what has leaked by processing a log taken while running the browser. Such a log can be taken over a long session of normal browsing and then the log can be processed to find sites that leak. Once a site is known to leak, the logging can then be repeated to figure out under what conditions the leak occurs.

Leak Gauge is easy to use and you don't need to download any additional Software or Tool (like needed for Debug Builds).

Leak Gauge will create a leak log file. You can upload this Log File here to get a analysis if you have run into a memory leak. There are plans to move this upload later to a better location like AMO.

Using Leak Gauge

As mentioned before, you don't need to install additional software to run Leak Gauge, however some simple steps are needed to run Leak Gauge and to get the Leak Log.

1. Setting up the Environment Variable:

The way to create the log is to set the environment variables: 

NSPR_LOG_MODULES=DOMLeak:5,DocumentLeak:5,nsDocShellLeak:5,NodeInfoManagerLeak:5
NSPR_LOG_FILE=nspr.log     (or any other filename of your choice)

in your shell and then run the program.

  • In a Windows command prompt, set environment variables with
set VAR=value
  • In an sh-based shell such as bash, set environment variables with
export VAR=value
  • In a csh-based shell such as tcsh, set environment variables with
 setenv VAR value

As example for Windows, this should look like:

  • 1. Start the Windows Command Prompt (Start -> Run -> cmd.exe or Start -> Programs -> Accessories -> Command Prompt
  • 2. Set the env Variable:
C:\>set NSPR_LOG_MODULES=DOMLeak:5,DocumentLeak:5,nsDocShellLeak:5,NodeInfoManagerLeak:5

and now you need to define the Location of the Leak Log File: 

C:\>set NSPR_LOG_FILE=c:\leak1.log

(or any other location/file name of your choice)

  • 3. Start Firefox and your leak log will be created.

Note:

  • When you test Firefox or your Extensions for leaks, you need to type the command above every time you start the Console/Terminal
  • Once you have this log from a complete run of the browser, you have to exit; otherwise it will look like everything leaked
  • Every time you start Firefox after you have defined the environment variables, your existing leak log will be overwritten. So if you want to reproduce a leak, or when you want to save a leak log, you need to change the file name (like from leak1.log to leak2.log) next time before you run Firefox.

See also the steps for testing for more Information.

The Leak Gauge Leak Log File

You have setup the environment variables and you got a leak.log, Fine ! Now you need to upload this leak log on this site.

After you have uploaded a leak log, the site will display the output of the results from the leak log, which will tell you which of certain core objects leaked and the URLs associated with those objects.

A normal output, without any leaking object / memory leak should look like: 

Results of processing log leak.log :
Summary:
Leaked 0 out of 11 DOM Windows
Leaked 0 out of 44 documents
Leaked 0 out of 3 docshells
Leaked content nodes in 0 out of 0 documents

A example output with leaking objects / memory leaks : 

Results of processing log leak2.log :
Leaked outer window 2c6e410 at address 2c6e410.
Leaked outer window 2c6ead0 at address 2c6ead0.
Leaked inner window 2c6ec80 (outer 2c6ead0) at address 2c6ec80.
Summary:
Leaked 13 out of 15 DOM Windows
Leaked 35 out of 46 documents
Leaked 4 out of 4 docshells
Leaked content nodes in 42 out of 53 documents

If you see such a results, please see the "steps for testing" and the "how to file a leak bug section".

Debug Builds

You can also use debug builds for leak testing. Building and Running a Debug Build is more complex then using Leak Gauge. If you interested in running Leak Test Debug Builds you can find here more information.

Leak Testing - Testing steps for Firefox / Firefox Extension Testing

First of all, its important to have some steps to reproduce, so developers can reproduce this leak and also for verifying a fixed leak bug. So if you can reproduce a memory leak 2 or more times, it should be filed as bug, see ...

Some general Tips for testing Firefox Extensions for Memory Leaks:

  • First create a new Firefox Profile to ensure nothing else from a used profile cause a leak. Start your Leak Debug as described.
  • Install the Extension and Restart Firefox
  • After the Restart - Quit Firefox to make sure the Extension does not leak after installation in general
  • Even when no Memory Leak was detected on start, use the features of this the Extension (in example with the QA Extension, use the Chat now button, the Login into Litmus etc)
  • Close Firefox and upload the Leak Log here
  • When a memory leak was detected save the leak.log/set the Leak Log environment variable to a different file name and start Firefox again to verify your steps to reproduce.

Note:: If the browser crashes during the run, the log is pretty much useless, since many objects will have been allocated but not freed at the time the crash occurs.

How to file a Memory Leak Bug ?

  • When you have found a Memory Leak Bug in Firefox and you have solid steps to reproduce or a testcase, please file a Bug in Bugzilla.

Note: If you are unsure if you have found a memory leak, or you are not familiar with Bugzilla, you can also ping Tomcat or send a mail to tomcat@mozilla.com and we can recheck this leak and file the leak bug

  • When you are unsure if its your extension that leak or Firefox itself, the best component in Bugzilla is Core:General
  • Please add mlk as keyword when you file the bug.
  • Please include the steps to reproduce/the testcase to the bug. This help developers to reproduce and fix the bug.
  • Please add the leak log / output from the Leak Gauge Script to the Bug.
  • It would great if you cc Tomcat to the Bug. You can do this via entering just :tomcat in the cc field in bugzilla or the full email address cbook@mozilla.com

Questions / Feedback

In Case you need help or when you have questions regarding the leak testing of your extension, feel me to ping me on irc.mozilla.org, my nickname is "Tomcat" or send me an e-mail.

Draft-template-image.png THIS PAGE IS A WORKING DRAFT Pencil-emoji U270F-gray.png
The page may be difficult to navigate, and some information on its subject might be incomplete and/or evolving rapidly.
If you have any questions or ideas, please add them as a new topic on the discussion page.
Retrieved from "https://wiki.mozilla.org/index.php?title=MozillaQualityAssurance:Home_Page:Firefox_3.0_TestPlan:Leaks:LeakTesting-How-To&oldid=126126"