MozillaQualityAssurance:Home Page:Firefox 3.0 TestPlan:Leaks:LeakTesting-How-To: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(5 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{draft}}
= Firefox 3 Leak Testing How-To  =
= Firefox 3 Leak Testing How-To  =
Note: this is a draft !


= Introduction=
= Introduction=
Line 5: Line 8:
Mozilla QA uses Trace Malloc Debug Build to test Firefox Components and Extensions for leaks. One part of this testing is manual testing. You can find the QA Firefox 3 Leak Testing Testplan [http://wiki.mozilla.org/MozillaQualityAssurance:Home_Page:Firefox_3.0_TestPlan:Leaks here].  
Mozilla QA uses Trace Malloc Debug Build to test Firefox Components and Extensions for leaks. One part of this testing is manual testing. You can find the QA Firefox 3 Leak Testing Testplan [http://wiki.mozilla.org/MozillaQualityAssurance:Home_Page:Firefox_3.0_TestPlan:Leaks here].  


== Builds ==
Many thanks to David Baron, Jonas Sicking, Jesse Ruderman and Peter Van der Beken for the help on this Project.
 
== Leak Gauge ==
 
For testing Firefox and Extensions for Memory we recommend "Leak Gauge" developed by [http://dbaron.org/ David Baron]. Leak Gauge is designed to help testers isolate and simplify testcases for many classes of leaks (those that involve large graphs of core data structures) in Mozilla-based browsers. It is designed to print information about what has leaked by processing a log taken while running the browser. Such a log can be taken over a long session of normal browsing and then the log can be processed to find sites that leak. Once a site is known to leak, the logging can then be repeated to figure out under what conditions the leak occurs.
 
Leak Gauge is easy to use and '''you don't need''' to download any additional Software or Tool (like needed for Debug Builds). 
 
Leak Gauge will create a leak log file. You can upload this Log File [http://mxr.mozilla.org/mozilla/source/tools/footprint/leak-gauge.html here] to get a analysis if you have run into a memory leak. There are plans to move this upload later to a better location like AMO.
 
=== Using Leak Gauge ===
 
As mentioned before, you don't need to install additional software to run Leak Gauge, however some simple steps are needed to run Leak Gauge and to get the Leak Log.
 
1. Setting up the Environment Variable:
 
The way to create the  log is to set the environment variables:


The Debug Builds for Leak Testing need to be build individually.  
NSPR_LOG_MODULES=DOMLeak:5,DocumentLeak:5,nsDocShellLeak:5,NodeInfoManagerLeak:5
NSPR_LOG_FILE=nspr.log    (or any other filename of your choice)


Debug Builds (for Leak Testing) are much bigger then normal optimized Firefox Builds. A Debug Leak Testing can be need up to 1,2 GB Disk Space.  
in your shell and then run the program.


There is right now '''no''' provided Leak Testing Build on the Mozilla FTP Server.
* In a Windows command prompt, set environment variables with
set VAR=value


For Building and Running Debug Builds please see the [http://developer.mozilla.org/en/docs/Build_Documentation Build Documentation] provided on [http://developer.mozilla.org developer.mozilla.org]. There are Instruction available for [http://developer.mozilla.org/en/docs/Windows_Build_Prerequisites Windows], [http://developer.mozilla.org/en/docs/Linux_Build_Prerequisites Linux] and [http://developer.mozilla.org/en/docs/Mac_OS_X_Build_Prerequisites Mac OSX].
* In an sh-based shell such as bash, set environment variables with
export VAR=value


If you build a Debug Build, you need to have trace-malloc activated in this builds. The .mozconfig file we use can be found for [http://people.mozilla.com/~cbook/memleak/mozconfigs/mozconfig-windows Windows XP], [http://people.mozilla.com/~cbook/memleak/mozconfigs/mozconfig-firefox-mac-i386-debug Mac OS 10.4] and [http://people.mozilla.com/~cbook/memleak/mozconfigs/mozconfig-firefox-linux-i686-debug Linux] here.
* In a csh-based shell such as tcsh, set environment variables with
  setenv VAR value


= Running a Leak Testing Build =
As example for Windows, this should look like:
* 1. Start the Windows Command Prompt (Start -> Run -> cmd.exe or Start -> Programs -> Accessories -> Command Prompt


Open a Terminal window (or xterm or Gnome Terminal) and enter:
* 2. Set the env Variable:  


  export XPCOM_MEM_LEAK_LOG=~/tmp/leak.log
  C:\>set NSPR_LOG_MODULES=DOMLeak:5,DocumentLeak:5,nsDocShellLeak:5,NodeInfoManagerLeak:5


'''Note that the directory you drop your leak.log into should already 
and now you need to define the Location of the Leak Log File:
exist, so if it's not there, please create it first.'''


Run the debug build of firefox creating a new profile. For mac, this 
C:\>set NSPR_LOG_FILE=c:\leak1.log
would be:
(or any other location/file name of your choice)


cd /location of your debug build/firefox-debug/dist/
* 3. Start Firefox and your leak log will be created.
FirefoxDebug.app/Contents/MacOS


./firefox-bin -P
Note:


linux (and when you use Mozilla Build on Windows) is:
* When you test Firefox or your Extensions for leaks, you need to type the command above every time you start the Console/Terminal


cd /location of your debug build/firefox-debug/dist/bin
* Once you have this log from a complete run of the browser, you have to exit; otherwise it will look like everything leaked


./firefox -P
* Every time you start Firefox after you have defined the environment variables, your existing leak log will be overwritten. So if you want to reproduce a leak, or when you want to save a leak log, you need to change the file name (like from leak1.log to leak2.log) next time before you run Firefox.


See also the steps for testing for more Information.


=== The Leak Gauge Leak Log File===


At the end of the run, the contents of ~/tmp/leak.log should have 
You have setup the environment variables and you got a leak.log, Fine ! Now you need to upload this leak log on this [http://mxr.mozilla.org/mozilla/source/tools/footprint/leak-gauge.html site].  
something in it.


= How do i know when i have found a leak ? =
After you have uploaded a leak log, the site will display the output of the results from the leak log, which will tell you which of certain core objects leaked and the URLs associated with those objects.


When you run Firefox like described above and quit Firefox later you get a Statistic in the Console where you have started Firefox.
A normal output, without any leaking object / memory leak should look like:


If no leak was found the output looks like the following:
Results of processing log leak.log :
Summary:
Leaked 0 out of 11 DOM Windows
Leaked 0 out of 44 documents
Leaked 0 out of 3 docshells
Leaked content nodes in 0 out of 0 documents


nsStringStats
A example output with leaking objects / memory leaks :
=> mAllocCount:         79737
=> mReallocCount:        4990
=> mFreeCount:          79737
=> mShareCount:          58374
=> mAdoptCount:          4593
=> mAdoptFreeCount:      4593


and the leak.log stays empty.
Results of processing log leak2.log :
Leaked outer window 2c6e410 at address 2c6e410.
Leaked outer window 2c6ead0 at address 2c6ead0.
Leaked inner window 2c6ec80 (outer 2c6ead0) at address 2c6ec80.
Summary:
Leaked 13 out of 15 DOM Windows
Leaked 35 out of 46 documents
Leaked 4 out of 4 docshells
Leaked content nodes in 42 out of 53 documents


When you run into a Memory leak the Stats look as example like this:
If you see such a results, please see the "steps for testing" and the "how to file a leak bug section".


nsStringStats
== Debug Builds ==
=> mAllocCount:          32807
=> mReallocCount:        3994
=> mFreeCount:          32805  --  LEAKED 2 !!!
=> mShareCount:          30349
=> mAdoptCount:          3524
=> mAdoptFreeCount:      3524


When you run into a leak you see also a lot of information in the leaks.log. This log is needed for debugging and fixing the leak.
You can also use debug builds for leak testing. Building and Running a Debug Build is more complex then using Leak Gauge. If you interested in running Leak Test Debug Builds you can find [http://wiki.mozilla.org/MozillaQualityAssurance:Home_Page:Firefox_3.0_TestPlan:Leaks:LeakTesting-How-To:Debug_Builds here] more information.


= Leak Testing  - Example of testing steps for Extension Testing =
= Leak Testing  - Testing steps for Firefox / Firefox Extension Testing =


When you found a leak, its important to have some steps to reproduce, so developers can reproduce this leak and also for verifying a fixed leak bug.
First of all, its important to have some steps to reproduce, so developers can reproduce this leak and also for verifying a fixed leak bug. So if you can reproduce a memory leak 2 or more times, it should be filed as bug, see ...


As example here are the steps to reproduce from [https://bugzilla.mozilla.org/show_bug.cgi?id=411576 Bug 411576].
Some general Tips for testing Firefox Extensions for Memory Leaks:


* First create a new Firefox Profile to ensure nothing else from a used profile cause a leak. Start your Leak Debug as described.
* First create a new Firefox Profile to ensure nothing else from a used profile cause a leak. Start your Leak Debug as described.
* Install a Extension and Restart Firefox
* Install the Extension and Restart Firefox
* After the Restart - Quit Firefox to make sure the Extension does not leak after installation
* After the Restart - Quit Firefox to make sure the Extension does not leak after installation in general
* When no memory leak was detected play a little with the extension to make sure the extension does not leak on specific steps to reproduce.  
* Even when no Memory Leak was detected on start, use the features of this the Extension (in example with the QA Extension, use the Chat now button, the Login into Litmus etc)
* Close Firefox and upload the Leak Log [http://mxr.mozilla.org/mozilla/source/tools/footprint/leak-gauge.html here]
 
* When a memory leak was detected save the leak.log/set the Leak Log environment variable to a different file name and start Firefox again to verify your steps to reproduce.
 
'''Note:''': '''If the browser crashes during the run, the log is pretty much useless, since many objects will have been allocated but not freed at the time the crash occurs.'''
 
== How to file a Memory Leak Bug ? ==
 
* When you have found a Memory Leak Bug in Firefox and you have solid steps to reproduce or a testcase, please file a Bug in Bugzilla.
 
''Note: If you are unsure if you have found a memory leak, or you are not familiar with Bugzilla, you can also ping Tomcat or send a mail to [mailto:tomcat@mozilla.com tomcat@mozilla.com] and we can recheck this leak and file the leak bug''


Some examples for in-detail testing of extensions:
* When you are unsure if its your extension that leak or Firefox itself, the best component in Bugzilla is Core:General
* Please add '''mlk''' as keyword when you file the bug.
* Try out features of the Extension (like Buttons, Options, etc). After you have done this, quit firefox to check the leak.log
* Please include the steps to reproduce/the testcase to the bug. This help developers to reproduce and fix the bug.
* Please add the leak log / output from the Leak Gauge Script to the Bug.
* It would great if you cc Tomcat to the Bug. You can do this via entering just ''':tomcat''' in the cc field in bugzilla or the full email address '''cbook@mozilla.com'''


* Go into the Extension Manager and check if the Firefox Extension Manager leaks when the Extension is installed
== Questions / Feedback ==


* Use the Extension Options Button in the Extension Manager (see Bug [https://bugzilla.mozilla.org/show_bug.cgi?id=412136 412136].
In Case you need help or when you have questions regarding the leak testing of your extension, feel me to ping me on irc.mozilla.org, my nickname is "Tomcat" or send me an [mailto:tomcat@mozilla.com e-mail].


* When a memory leak was detected save the leak.log and start Firefox again to check if the extension leaks only the first time after installation, or if the extension leak in general when installed
{{draft}}

Latest revision as of 20:39, 29 January 2009

Draft-template-image.png THIS PAGE IS A WORKING DRAFT Pencil-emoji U270F-gray.png
The page may be difficult to navigate, and some information on its subject might be incomplete and/or evolving rapidly.
If you have any questions or ideas, please add them as a new topic on the discussion page.

Firefox 3 Leak Testing How-To

Note: this is a draft !

Introduction

Mozilla QA uses Trace Malloc Debug Build to test Firefox Components and Extensions for leaks. One part of this testing is manual testing. You can find the QA Firefox 3 Leak Testing Testplan here.

Many thanks to David Baron, Jonas Sicking, Jesse Ruderman and Peter Van der Beken for the help on this Project.

Leak Gauge

For testing Firefox and Extensions for Memory we recommend "Leak Gauge" developed by David Baron. Leak Gauge is designed to help testers isolate and simplify testcases for many classes of leaks (those that involve large graphs of core data structures) in Mozilla-based browsers. It is designed to print information about what has leaked by processing a log taken while running the browser. Such a log can be taken over a long session of normal browsing and then the log can be processed to find sites that leak. Once a site is known to leak, the logging can then be repeated to figure out under what conditions the leak occurs.

Leak Gauge is easy to use and you don't need to download any additional Software or Tool (like needed for Debug Builds).

Leak Gauge will create a leak log file. You can upload this Log File here to get a analysis if you have run into a memory leak. There are plans to move this upload later to a better location like AMO.

Using Leak Gauge

As mentioned before, you don't need to install additional software to run Leak Gauge, however some simple steps are needed to run Leak Gauge and to get the Leak Log.

1. Setting up the Environment Variable:

The way to create the log is to set the environment variables: 

NSPR_LOG_MODULES=DOMLeak:5,DocumentLeak:5,nsDocShellLeak:5,NodeInfoManagerLeak:5
NSPR_LOG_FILE=nspr.log     (or any other filename of your choice)

in your shell and then run the program.

  • In a Windows command prompt, set environment variables with
set VAR=value
  • In an sh-based shell such as bash, set environment variables with
export VAR=value
  • In a csh-based shell such as tcsh, set environment variables with
 setenv VAR value

As example for Windows, this should look like:

  • 1. Start the Windows Command Prompt (Start -> Run -> cmd.exe or Start -> Programs -> Accessories -> Command Prompt
  • 2. Set the env Variable:
C:\>set NSPR_LOG_MODULES=DOMLeak:5,DocumentLeak:5,nsDocShellLeak:5,NodeInfoManagerLeak:5

and now you need to define the Location of the Leak Log File: 

C:\>set NSPR_LOG_FILE=c:\leak1.log

(or any other location/file name of your choice)

  • 3. Start Firefox and your leak log will be created.

Note:

  • When you test Firefox or your Extensions for leaks, you need to type the command above every time you start the Console/Terminal
  • Once you have this log from a complete run of the browser, you have to exit; otherwise it will look like everything leaked
  • Every time you start Firefox after you have defined the environment variables, your existing leak log will be overwritten. So if you want to reproduce a leak, or when you want to save a leak log, you need to change the file name (like from leak1.log to leak2.log) next time before you run Firefox.

See also the steps for testing for more Information.

The Leak Gauge Leak Log File

You have setup the environment variables and you got a leak.log, Fine ! Now you need to upload this leak log on this site.

After you have uploaded a leak log, the site will display the output of the results from the leak log, which will tell you which of certain core objects leaked and the URLs associated with those objects.

A normal output, without any leaking object / memory leak should look like: 

Results of processing log leak.log :
Summary:
Leaked 0 out of 11 DOM Windows
Leaked 0 out of 44 documents
Leaked 0 out of 3 docshells
Leaked content nodes in 0 out of 0 documents

A example output with leaking objects / memory leaks : 

Results of processing log leak2.log :
Leaked outer window 2c6e410 at address 2c6e410.
Leaked outer window 2c6ead0 at address 2c6ead0.
Leaked inner window 2c6ec80 (outer 2c6ead0) at address 2c6ec80.
Summary:
Leaked 13 out of 15 DOM Windows
Leaked 35 out of 46 documents
Leaked 4 out of 4 docshells
Leaked content nodes in 42 out of 53 documents

If you see such a results, please see the "steps for testing" and the "how to file a leak bug section".

Debug Builds

You can also use debug builds for leak testing. Building and Running a Debug Build is more complex then using Leak Gauge. If you interested in running Leak Test Debug Builds you can find here more information.

Leak Testing - Testing steps for Firefox / Firefox Extension Testing

First of all, its important to have some steps to reproduce, so developers can reproduce this leak and also for verifying a fixed leak bug. So if you can reproduce a memory leak 2 or more times, it should be filed as bug, see ...

Some general Tips for testing Firefox Extensions for Memory Leaks:

  • First create a new Firefox Profile to ensure nothing else from a used profile cause a leak. Start your Leak Debug as described.
  • Install the Extension and Restart Firefox
  • After the Restart - Quit Firefox to make sure the Extension does not leak after installation in general
  • Even when no Memory Leak was detected on start, use the features of this the Extension (in example with the QA Extension, use the Chat now button, the Login into Litmus etc)
  • Close Firefox and upload the Leak Log here
  • When a memory leak was detected save the leak.log/set the Leak Log environment variable to a different file name and start Firefox again to verify your steps to reproduce.

Note:: If the browser crashes during the run, the log is pretty much useless, since many objects will have been allocated but not freed at the time the crash occurs.

How to file a Memory Leak Bug ?

  • When you have found a Memory Leak Bug in Firefox and you have solid steps to reproduce or a testcase, please file a Bug in Bugzilla.

Note: If you are unsure if you have found a memory leak, or you are not familiar with Bugzilla, you can also ping Tomcat or send a mail to tomcat@mozilla.com and we can recheck this leak and file the leak bug

  • When you are unsure if its your extension that leak or Firefox itself, the best component in Bugzilla is Core:General
  • Please add mlk as keyword when you file the bug.
  • Please include the steps to reproduce/the testcase to the bug. This help developers to reproduce and fix the bug.
  • Please add the leak log / output from the Leak Gauge Script to the Bug.
  • It would great if you cc Tomcat to the Bug. You can do this via entering just :tomcat in the cc field in bugzilla or the full email address cbook@mozilla.com

Questions / Feedback

In Case you need help or when you have questions regarding the leak testing of your extension, feel me to ping me on irc.mozilla.org, my nickname is "Tomcat" or send me an e-mail.

Draft-template-image.png THIS PAGE IS A WORKING DRAFT Pencil-emoji U270F-gray.png
The page may be difficult to navigate, and some information on its subject might be incomplete and/or evolving rapidly.
If you have any questions or ideas, please add them as a new topic on the discussion page.
Retrieved from "https://wiki.mozilla.org/index.php?title=MozillaQualityAssurance:Home_Page:Firefox_3.0_TestPlan:Leaks:LeakTesting-How-To&oldid=126126"