Thunderbird:Testing:Memory Usage Problems

From MozillaWiki
Jump to: navigation, search

Slowness, for any reason, can be frustrating.

To fix slowness, you must do some investigation. And more than one cause is possible, so please use the full list below. If you are unable to find a cause on your own, please provide the detailed results of your investigation with this list in a support forum or bug report.

This document may seem technical, but it's not complicated taken a step at a time. All types of performance issues covered, memory, IO, and CPU, because often users see multiple symptoms.

Diagnosis Steps

Before you begin diagnosing:

  • Some steps are *tests* which should be considered temporary, not a suggestion to forever disable a feature you need. So please do not to skip steps like disabling of antivirus - all steps are essential to a full diagnosis.
  • Do not delete files - they can be useful for future diagnosis. Rename or move files instead of deleting.

Describe your problem clearly but as briefly as possible in a support forum, newsgroup, or existing or new bug report:

  • use numbers - for example cpu percentage, amount of memory, number of folders and messages, etc.
  • precisely state when and where you see slowness or increased memory (startup, shutdown, or "normal operation"), and sequence of steps to reproduce your problem
  • cite: step numbers below which help or fail bug numbers, support topic URL(s)
  • list account types (imap, pop, rss, etc) obtainable from Help | Troubleshooting
  • give Thunderbird version number from Help | About, and last version where you did NOT have the problem.

Steps

  1. Are you using the newest update of Thunderbird? (do Help | About | Check for Updates, or visit [1])
  2. Start Thunderbird in Safe Mode (instructions). If Thunderbird safe mode helps then visit Tools | Add-ons | Extensions and determine which add-on might be causing the problem by disabling/enabling individual add-ons and...
  3. Is Thunderbird better with OS started in safe mode? Instructions: win10, win8, win7, winXP, MacOS. If performance is improved, then suspect software such as antivirus or firewall not working properly with Thunderbird. Test with firewall or antivirus software disabled or removed. Ad-Aware example If performance is better, find which step helps :
    • run run only ONE package, and no more, of: antivirus, firewall, malware
    • check that you have the newest version of AV/firewall software
    • with AV/firewall software turned on, uninstall Thunderbird, DELETE the Thunderbird PROGRAM directory (not data directory), download, reinstall, and retest Thunderbird. If using beta or Earlybird get it from channels page.
    • temporarily disable antivirus software (including on MacOS), and if problem is gone see AV performance issues for settings and information which may help.
    • check with vendor's software support
  4. If you have gmail accounts, is problem seen after unsubscribing the "All Mail" folder?
  5. If you have newsgroup accounts, then in Account Settings|Server Settings disable "Check for new messages at startup". If problem is gone you may be seeing many newsgroups with many unread messages, bug 185634, bug 541001, or [2].
  6. Are any of these indexing methods enabled:
  7. On Windows, did you move the Thunderbird profile or any Thunderbird account from the default location "%APPDATA%" to a different Windows directory? If yes, then MS-Windows Search will attempt to index all Thunderbird disk activity. Disable Windows indexing for your Thunderbird data.
  8. If Thunderbird memory usage is more than expected by hundreds of megabytes (MB), then perhaps one or more folders (internally a database) are not being closed, or some other bug. Please :
    • Get a diagnostic log of Thunderbird activity with MSGDB:5,timestamp using logging instructions.
    • File a bug, then edit the new bug and attach the zipped log file using "Add an attachment".
  9. Does problem resolve by disabling global indexing? Tools|Options|Advanced|General|Enable Global Search and Indexer (linux: Edit|Preferences|Advanced|... Mac: cmd , advanced tab) and restart Thunderbird. If the problem goes away:
    • Does Thunderbird have extremely high IO, seem to be stuck on indexing per Tools|Activity Manager, or is global-messages-db.sqlite (file used for global search) in your profile folder too big? A rule of thumb is it should not be more than 10-20% of the combined the size of all messages (in directories Mail and ImapMail).
      • Rename global-messages-db.sqlite and restart Thunderbird. global-messages-db.sqlite will rebuild but your system load will be high until all your messages are reindexed.
      • Please file a bug about your problem.
  10. Is there an indexing or Search Messages (Ctrl+K) problem, or is global-messages-db.sqlite corrupt? Shut down Thunderbird, in your profile keep the sqlite file but rename it for later diagnosis, and restart Thunderbird. Note, rebuilding the global index may take a long time and use CPU resources, but it should not redownload your mail
  11. Are any folders 2-4GB in size? Large folders take lots of memory and time to load. Check using foldersize add-on. Also Help | Troubleshooting lists large folders.
  12. Does disabling interval checks for new mail help? (Account Settings | Server Settings | uncheck "Check for new messages every N minutes"). If it does help, then ...
    • Is a low value (1-3 minutes) specified for checking for new mail or draft autosave? does changing it to 4-10 minutes help - go to, respectively, Account Settings | Server Settings | Check for new messages every N minutes ... and Options | Composition | General | Auto Save every N minutes.
    • What type of mail account is this (imap or pop), and how many other accounts are the same type?
    • Are you absolutely certain that antivirus software is not to blame?
  13. Are any non-Inbox IMAP folders configured to check for new messages (via "always check this folder" in properties)? Or have hidden preference mail.check_all_imap_folders_for_new set to true? These increase load on both Thunderbird and the mail server. bug 729504
  14. Do you have thousands of contacts in your address book? (possible related issues: google contacts add-on], [12]
  15. Is panacea.dat cache file in your profile too big? (>10MB for most people is probably too big. Rule of thumb for panacea size with thunderbird stopped: #folders*0.3k. See bug 65086, bug 1084276)
  16. If error message indicates a script problem (ex. "Unresponsive Script"), please set javascript.options.mem.log to true and provide a couple log entries of the form CC(T+1186.6) collected: 131 (131 waiting for GC), suspected: 98, duration: 274 ms.
    • Has a proxy ever been defined, and is it working correctly or was the proxy removed? (A proxy can cause "Unresponsive script")
  17. If using high CPU, check to see if mozilla updater.exe is running in taskmgr [13]
  18. Is your Thunderbird profile or any part of Thunderbird on a non-local hard drive, i.e. a "special" drive or mount point? If yes, does it preform better when it is moved to the local disk?
  19. If you suspect slow or high disk activity, on a Windows sytem make sure you have at least 20% free disk space and then do defrag.
  20. If you have RSS news reader account, in Account Settings|Server Settings disable "Check for new messages at startup". If problem is gone you may be seeing something like bug 551144 (fixed in version 13).
  21. Is the account Outlook or Hotmail? bug 1355885
  22. Check if filter logging is enabled at Tools > Filters
  23. Common resource requirements/maintenance steps:
    • Do you have the minimum hardware requirements, and enough memory? Note, even having the minimum or recommended sized system may not be sufficient for your workload.
    • Consider the possibility of hardware problems [14], [15], [16]
    • Are your folders well maintained, "too large", or perhaps broken:
      • Do you see high memory or long delay when opening a specific folder? Or when moving message to a specific folder?
      • Do any folders lose their sort order or column settings?
      • Do you empty trash and junk on a regular basis? You should not be using them for long term storage. Does performance improve if you empty them? (see File menu)
      • Do any Inbox or filtered folders have several thousand messages? If so, especially for Inbox, consider moving some messages to another folder. Perhaps use Archive.
      • If you are using a version prior to 5, do you regularly (daily or weekly) File > Compact Folders?
  24. If you are running an old version of linux, consider updating. hangs on Ubuntu 10.04, 14.04 works
  25. If a bug report in References below matches your problem, please add yourself to the bug report and add details about your situation.
  26. If you do not find an existing bug report and still see a problem ...

Please give details about your results of doing the list above in a support forum or bug report.

References

(Not Responding) examples (and cause, if known) - (Plusnet Protect/McAfee), McAfee, WSS, Windows Search Indexing and [17], McAfee, McAfee, McAfee (2013), Windows 8, use XP3 compatibility mode (Endicott), use XP3 compatibility mode (missychrissy), use XP3 compatibility mode (WaddyT/2013),Windows 8, use XP3 compatibility mode (kgcode)

Unresponsive Script examples (and cause, if known) -

Unresolved bugs and references:

  • McAfee antivirus is causing major performance and problems with TB24 (and was with TB17 and TB16). Symptom: slow delete and read, "Unresponsive Script", "(Not Responding)". Solution
  • bug 929281 - Long startup and "Unresponsive Script" with many pop accounts with version 24 (regression in TB18)
  • bug 1116196 - Every startup, Warning: Unresponsive Script after approx 15 seconds, with system integration prompt (regression in TB18)
  • bug 764306 - When trying to scroll large folders or switch some folders the application beachballs with high CPU (xref: bug 872869)
  • bug 558528 Larger buffer size in file I/O by Mail&News to improve performance on slower drives (eg network drives/shares).
  • High mem usage, increasing with every delete. No bug report. Solved by right+click, compact.
  • bug 632791 Gloda (Global Search) indexes one message per minute with high CPU. Excessively sized global-messages-db.sqlite caused by buggered, excessively sized offline store. [because offline folder storage has many duplicate messages. but it should not]
  • bug 778907 Highlighting/selecting 18000+ of emails inside a folder keeps giving the javascript is unresponsive script ("bulkoperations", i.e. not a general performance issue)
  • bug 563677 HTML Mail View slow (45++ sec) for pop and local folders - speed up FolderUriForPath (from bug 545126)
  • bug 487832 - Thunderbird startup performance issues [meta]
  • ref: Thunderbird Performance knowledge base article
  • ref: Standard diagnostic for Thunderbird knowledge base article
  • ref: bug 480841 Implement memory reporter for Gloda
  • ref: bug 480843 Implement memory reporter for mailnews
  • Lightning related:
    • bug 441710 - Thunderbird freezes/slow for 3-60 seconds caused by calendar/lightning [meta]
    • bug 753343 - Soft-hang and massive memory leaks with Lightning and ics enabled
    • bug 501689 - Investigate making storage asynchronous // bug 462277 - (calcache) [meta][GSoC 2014] Turn on offline cache by default for new calendars
    • bug 498968 - Create a new, faster storage "storage2" provider
  • less selective list of bugs inclusive of those cited above

Bugs fixed but not released. Typically fixed in unreleased versions (eg. trunk regressions):

  • none

Fixed bugs/issues:

Performance bugs fixed in 38.0:

  • bug 872869 nsMsgDatabase::ListAllKeys() very slow with high CPU downloading messages into large folder (xref: bug 764306) (regression in TB16)
  • bug 1135310 - closing idle folder databases potentially broken

Performance bugs fixed in 37.0:

  • bug 846123 - Thunderbird 100% CPU for minutes when copying a large number of messages (IMAP Online Copy, Copy between Offline-Use=Off folders) (landed in TB37)

Performance bugs fixed in 31.0:

  • bug 870556 - O(n^2) performance freezes UI for several minutes fetching new mail from IMAP server for very large folder
  • bug 975795 - extremely slow for large selections, folderDisplay.canArchiveSelectedMessages is too slow (TB30)
  • bug 1019969 - (resolved WFM, cause and fix unknown) Thunderbird 18-30b1 gets "Unresponsive script" 45 seconds after launch/startup. OK in 31.0
  • bug 926181 - Thunderbird 24 OSX freezes on startup with profile on network drive (AFP) - fixed via bug 918612 - Mac: Firefox 24 doesn't start, or very slow startup

Performance bugs fixed in 24.0:

  • bug 812923 - Dragging a huge number of messages onto a folder freezes TB even BEFORE dropping (TB24)
  • bug 750781 - Thunderbird 12 veryyyyy slow when opening Message. Sometimes shows "Not Responding" (TB21)
  • bug 810637 - HUGE memory consumption when downloading headers for large IMAP folder with mailnews.customDBHeaders (TB19)

Performance bugs fixed in 17.0:

  • bug 806760 - TB16 Redownloads messages / Constantly bringing folders "up to date" / horrible IMAP performance (TB16 regression)
  • bug 769346 - Poor performance of Move, Copy and Delete to Local Folders due to "one write request per a line of message data"

Performance bugs fixed in 16.0.2:

  • bug 803843 - IMAP mailfiles keep growing to gigabyte size

Performance bugs fixed in 16:

  • bug 793455 - DB cache is closing databases too frequently (regression in 15.0)
  • bug 770416 - (insignificant?) memory leak on some IMAP operations

Performance bugs fixed in 15.0.1:

  • bug 782899 - Daily hangs and leaks memory like crazy when trying to delete messages in the results of a filter search (regression in 15.0)

Performance bugs fixed in 15.0:

  • bug 784286 - Abnormal memory requirements viewing plain-text mail leading to crash
  • bug 723248 - add support for closing inactive databases (folders)
  • bug 762169 - Always asking to compact folders starting version 12

Performance bugs fixed in 14.0:

  • bug 538378 - Mass delete of messages resulted in massive memory usage // bug 296453 - Racing CPU, slow performance moving/deleting/dragging large number of messages

Performance bugs fixed in 13.0:

  • bug 551144 - checking rss feeds has problem freezing Thunderbird UI and high hard drive usage (not a v3 or v3.1 regression)

Performance bugs fixed in 10.0:

Performance bugs fixed in 9.0:

  • bug 689742 - Disable hardware acceleration (direct2d) in Thunderbird

Performance bugs fixed in 5.0

  • bug 574952 - Thunderbird 3.1 account list / folder pane slow to scroll on Mac (Mac-only)
  • bug 617839 - Switching to threads with a large number of emails is extremely slow

Performance bugs fixed in 3.1.5:

  • bug 542234 memory increases about 12MB on each check for new mail for pop accounts, 3.0 & 3.1
  • bug 599119 slow message display/slow navigation - plugin directory scans cause excessive overhead

Performance bugs fixed in 3.0.9:

  • bug 542234 memory increases about 12MB on each check for new mail for pop accounts, 3.0 & 3.1

Performance bugs fixed in 3.0.5:

  • bug 530098 gloda message deletion processing is expensive

Performance bugs fixed in 3.1:

  • bug 533776 gloda indexes while offline, if we startup without a network connection
  • bug 539389 Very slow file manipulation deleting or moving messages (profile stored on synchronized network folder). Tb3 requests write for each line of mail data. Should use buffering.
  • bug 545126HTML Mail View very slow (45++ sec) for pop and local folders (*not* fixed in v3.0.x) / bug 536873 slow load HTML messages with View Message Body as Original HTML - sits at "Loading Message ..." or presents blank screen
  • bug 530098 gloda message deletion processing is expensive (*not* fixed in v3.0.x)
  • bug 493848 gloda activity manager should not create events for event-driven indexing, never display in status bar (*not* fixed in v3.0.x)
  • bug 543737 gloda needs to avoid indexing ridiculously large message bodies (especially from offline store corruption) (indexing stops at 20k per message part)(*not* fixed in v3.0.x)
  • bug 560695 clicking (more) widget in headers with lots of recipients too slow (*not* fixed in v3.0.x)

Performance bugs fixed in 3.0.4:

  • bug 533776 gloda indexes while offline, if we startup without a network connection
  • bug 539389 Very slow file manipulation deleting or moving messages (profile stored on synchronized network folder). Tb3 requests write for each line of mail data. Should use buffering.

Memory related bugs fixed in 3.0.2/3.0.3:

  • bug 540214 imap STATUS not closing db - mail.check_all_imap_folders_for_new and "check this folder for new messages" causes high memory and error message Unable to open the summary file for xxxxx. Perhaps there was an error on disk, or the full path is too long.

Performance related bugs fixed in 3.0.1:

  • bug 534835 offline folder size constantly increases using Exchange IMAP (Exchange returns wrong rfc822 size) - messages redownload
  • bug 537820 quick search in smart folders/saved searches should prefer offline when it has a choice
  • bug 533865 Doing a quick search in a cross-folder saved search clears cached results

Memory related bugs fixed in 3.0

  • bug 525646 Mass delete of messages resulted in massive memory usage (not sustained usage, and was v3-only behavior I think, i.e. not from v2)

Fixed in version 2 and / or prior to releasing version 3.0:

  • RSS
  • google desktop search
  • bug 465353 general sluggish, lack of responsiveness during indexing of gloda (indexing speed kills CPU and performance)
  • bug 518918 Thunderbird 3pre high memory and consuming 100% CPU making mail nearly unusable, need to handle UID's > 0x7FFFFFFF (fixed in 3.0.0)
  • bug 465122 Gloda: Does not close msf files on initial indexing
  • bug 387403 Thunderbird eat memory until crash when opening e-mail with broken vcard (fixed in version 2.0.0.13)
  • bug 216535 large mailboxes seem to cause pathological memory usage, was duped to ...
  • bug 266679 Thunderbird consumes excessive file-handles and memory (fixed 1.5.<something>, and before release of 2.0)

Memory Overview

Related Articles

If you don't need background information about memory, please skip to diagnosis steps.

"How much memory is needed run Thunderbird?" is a frequently asked question. System Requirements defines minimum Thunderbird machine requirements for a "mythical" user. But, these are just minimums, and do not describe how much memory you might need in your "real" environment. A better answer of what is enough memory might be ...:

a) It depends on your Thunderbird configuration (number of large folders, number of accounts, etc)
b) It depends on what you are measuring or doing in Thunderbird at a specific time, because some operations require more memory than others. Like when you are doing mundane things like reading new messages versus when you are searching for messages.
c) Current versions require roughly at least 80 MB to 100 MB more memory than version 2. So (for example) if your version 2 used 50MB, then v3 might use 130 MB to 150 MB.

Which memory to measure ...

  • MS Windows - Working Set/Mem Usage of thunderbird.exe (start taskmgr from ctrl+alt+delete, click Processes tab), and both Total and Available physical memory from the Performance tab.
  • Mac - Real Private Memory of thunderbird (process manager, right click on column bar, select Real)
  • linux - RSS/RES (Reserved Segment Size) for thunderbird

Rules of thumb (these are not "laws") for an average user with NO addons ... after a day of using Thunderbird, if memory usage is :

  • less than 250MB, then you are OK.
  • not much above your version 2 memory usage plus 100MB, then you are probably OK.
  • is over 300MB, then you may have a problem that needs analysis.
  • increasing significantly over a period of day or more, and never drops back below 300MB, i.e. it increases cumulatively over time then you may have a problem. For example, if you click on a folder and memory increases by 10MB and stays at that level or higher forever, and next time you click the same folder and memory increases another 10MB and stays at that level or higher forever ..., then you have a problem.

Note: memory usage is affected by the number and types of extensions (add-ons) installed, number of folders and accounts, number of messages in folders, and bugs.

Other Software

Add-ons / Extensions reported to have caused high memory, performance, or other problems:

Programs (firewall, antivirus, etc) reported to have caused high memory, performance or other problems:

  • ogplanet Game Launcher/npOGPPlugin.dll bug 543082 slow opening an email on windows
  • Secunia PSI
  • THE SHIELD bug 542156
  • Eset NOD32 bug 541197 Repeated attempts to create huge temporary file during message synchronization due to Eset NOD32 causes slow gloda indexing
  • F-Prot Antivirus may cause very slow access to mailbox files. See: Antivirus Related Performance Issues

Miscellaneous

Memory Leaks

Most people have heard the term 'Memory leaks'. The classic memory leak is a program not releasing some temporarily used memory. If you are interested in helping address memory leaks ...

  • Anyone can help find js leaks with Thunderbird started in safe mode. Install the Leak Monitor detector add-on, perform a specific function with the leak detector enabled, and then file a bug with information about what function you performed, and the results of the detector.
  • The technically inclined are encouraged to reference Performance:Leak Tools. Detailed leak analysis using the tools and methods described there will help improve Thunderbird, and your help in filing bugs the results of your findings will be appreciated.

Terminology

  • memory usage - varies per OS: in Windows XP taskmgr the column is 'Mem Usage', in Vista it is 'Working Set'
  • Glossary

References