MailNews:Memory Bloat and Leak Tests

<< Back to MailNews:Performance Testing

Note: MailNews here refers to either Thunderbird, or the Mail/Newsgroups/Address Book part of SeaMonkey

Aim

• To provide a continuous check within Mailnews and its sub-components for the following items:
  1. Total memory leaks.
  2. Total memory usage (aka bloat).
• Provide a consistent number from build to build where no source code has changed.
• Provide reasonable coverage for the normal activities that a user performs.

Method

Current Method:

• Measure Leaks and Bloats, in a similar way to Firefox (using XPCOM_MEM_LEAK_LOG and --trace-malloc).
• The method of testing memory bloat and leaks will be the same sequence of events, using the same set of test data.
  1. Startup Main Mail Window
  2. Open Address Book and Message Composition Windows
  3. Close Address Book and Message Composition Windows
  4. Quit the application

Future improvements will be discussed on the discussion page.

Results

Printed via standard tinderbox methods, these will consist of:

• Mail RLk
  • Reference Count Leaks
• Mail Lk
  • Total Bytes malloc'ed and not free'd
• Mail MH
  • Maximum Heap size
• Mail A
  • Allocations - number of calls to malloc and friends.

There are currently no graphs for these results.

Manually Running Tests

Setting up

Build Set up

1. Build Thunderbird or SeaMonkey trunk with your standard mozconfig file, but with the following options set:
   • ac_add_options --enable-debug
   • ac_add_options --enable-trace-malloc

Profile Set Up

1. Enter the executable directory. Use the command:
   • <executable_name> -createProfile <profilename>
2. Now go into the /mailnews/test/performance/bloat directory and run the command:
   • python setUpBloatTest.py --binary-dir=<directory where executable is> --profile-dir=<directory for profile created in step 1>

These two commands will set up the profile the same as on the tinderbox.

Reference Count Leak Test

1. set XPCOM_MEM_BLOAT_LOG to 1 in your environment (see also Debugging Memory Leaks)
2. Start the appropriate executable (using -P <profilename> if required)
3. Once the test is completed (application automatically shuts down), you'll get a dump of the objects allocated and which ones are leaking.

Leak, Max Heap and Allocations Test

1. Start the appropriate executable with the following arguments:
   • --trace-malloc malloc.log
   • --shutdown-leaks sdleak.log
   • -P <profilename> (if required)
2. Wait for the test to complete and the application to shut down.
3. The malloc.log file can then be run through leakstats in the <objdir>/tools/trace-malloc/ directory to obtain the statistics:
   • leakstats malloc.log

See Debugging Memory Leaks for more information on what to do with this data.

Code Locations

The code used to drive this will be in Buildbot.

The files specific to mailnews testing will be stored in:

/mailnews/test/performance/

These files consist of:

• Overlays (used to provide the hooks for the javascript):
  • bloat/bloatAddrOverlay.xul
  • bloat/bloatComposeOverlay.xul
  • bloat/bloatMainOverlay.xul
• Javascript files (used to drive the tests):
  • bloat/bloatAddrOverlay.js
  • bloat/bloatComposeOverlay.js
  • bloat/bloatMainOverlay.js
• Preference Settings (used to provide a defined profile, see below):
  • common/mailnewsTestPrefs.js
• Python scripts (used to set up the profile):
  • bloat/setUpBloatTest.py
  • common/setUpCommonMailNews.py
• Perl script used to run the actual tinderbox test:
  • bloat/runtest.pl

Pre-defined profile

Initial Setup:

• One POP3 account (Mails TBD)
• One Identity
• One SMTP server defined (not used)

Future requirements/possibilities:

• One Address Book where PAB has ~1000 entries
• Large Message folders
• IMAP
• NNTP Server and subscribed newsgroup.

08/04/2008: prefs.js created via TB with the above settings, the first section is for preferences included in tinderbox, the second section is ones which don't currently get set.

user_pref("mail.account.account1.server", "server1");
user_pref("mail.account.account2.identities", "id1");
user_pref("mail.account.account2.server", "server2");
user_pref("mail.accountmanager.accounts", "account1,account2");
user_pref("mail.accountmanager.defaultaccount", "account2");
user_pref("mail.accountmanager.localfoldersserver", "server1");
user_pref("mail.identity.id1.fullName", "Tinderbox");
user_pref("mail.identity.id1.smtpServer", "smtp1");
user_pref("mail.identity.id1.useremail", "tinderbox@invalid.com");
user_pref("mail.identity.id1.valid", true);
user_pref("mail.root.none-rel", "[ProfD]Mail");
user_pref("mail.root.pop3-rel", "[ProfD]Mail");
user_pref("mail.server.server1.directory-rel", "[ProfD]Mail/Local Folders");
user_pref("mail.server.server1.hostname", "Local Folders");
user_pref("mail.server.server1.name", "Local Folders");
user_pref("mail.server.server1.type", "none");
user_pref("mail.server.server1.userName", "nobody");
user_pref("mail.server.server2.check_new_mail", false);
user_pref("mail.server.server2.directory-rel", "[ProfD]Mail/tinderbox");
user_pref("mail.server.server2.download_on_biff", true);
user_pref("mail.server.server2.hostname", "tinderbox");
user_pref("mail.server.server2.login_at_startup", false);
user_pref("mail.server.server2.name", "tinderbox@invalid.com");
user_pref("mail.server.server2.type", "pop3");
user_pref("mail.server.server2.userName", "tinderbox");
user_pref("mail.smtp.defaultserver", "smtp1");
user_pref("mail.smtpserver.smtp1.hostname", "tinderbox");
user_pref("mail.smtpserver.smtp1.username", "tinderbox");
user_pref("mail.smtpservers", "smtp1");
user_pref("mail.startup.enabledMailCheckOnce", true);
user_pref("mailnews.start_page_override.mstone", "1.9pre");

Preferences in generated profile, but not set:

user_pref("mail.root.none", "/home/moztest/.thunderbird/t7i1txfw.minimum/Mail");
user_pref("mail.root.pop3", "/home/moztest/.thunderbird/t7i1txfw.minimum/Mail");
user_pref("mail.server.server1.directory", "/home/moztest/.thunderbird/t7i1txfw.minimum/Mail/Local Folders");
user_pref("mail.server.server2.directory", "/home/moztest/.thunderbird/t7i1txfw.minimum/Mail/tinderbox");
user_pref("mail.attachment.store.version", 1);
user_pref("mail.folder.views.version", 1);
user_pref("mail.spam.version", 1);
user_pref("mailnews.quotingPrefs.version", 1);
user_pref("mailnews.ui.threadpane.version", 6);