MailNews:Minimizing Bandwidth Usage

Make Thunderbird more friendly for unreliable and low-bandwidth conditions.

Thunderbird currently has a number of features that could help in these conditions, such as 'offline folders', 'MIME parts on-demand' and caching. But these features do not always work well in both unreliable and low-bandwidth conditions or when combined together.

This wiki pulls together inter-related issues in one place for discussion and to propose a solution.

User Experience (UX)

The desired user experience in unreliable/low-bandwidth conditions could be described as:

Only download what I ask for and only download it once.

There are a number of bugs/features at the moment that are stopping this from happening, namely bug 405437, bug 345832 and bug 439731. More on these later.

Desired Behaviour

1) Only download messages that I ask for

2) Once a message or part of a message (MIME part) has been downloaded it should always be stored locally:

• in offline storage if folder is __offline__ and storage rules allow (e.g. diskspace) OR
• in cache

3) A message should always be retrieved from local storage if possible.

Current Behaviour

NOTE: This refers to TB3, however currently the only significant difference is the new auto-sync feature in TB3.

1) Auto-sync (on by default) will download all messages in an __offline__ folder. We may not want this in all cases i.e. for large messages.

2) Auto-sync meets the desired behaviour in most cases (see here for more information). So lets just look at scenarios in which auto-sync is not applicable i.e. non __offline__ folders and messages that we do not want to be auto-synced.

a) If all non-inline MIME parts < ~30KB:

• message is downloaded and stored when viewed

b) If one or more non-inline MIME parts > ~ 30KB:

• MIME parts are downloaded on-demand and cached sometimes (there are constraints such as cache size)

3) Messages in offline store are retrieved from local storage. Cached MIME parts are retrieved from cache, however large MIME parts (i.e. attachments are rarely cached)

NOTE: Effectively this means that messages with large MIME parts (i.e. attachments) are not cached and not available offline. They have to be downloaded each time they are viewed.

Implementation

What Needs To Be Done

Although poor handling of MIME parts is a major obstacle to minimising bandwidth usage in TB it is also the most difficult to address. To get immediate results a low-hanging fruit approach is proposed, in rough order of priority:

1) Add on/off/on-demand features to auto-sync giving user full control of when sync takes place

2) User configurable auto-syncs strategies to prioritise and include/exclude messages downloaded by auto-sync

3) Better handling of MIME parts so they are only downloaded once

a) Caching of large MIME parts e.g. attachments

b) Support for offline storage of MIME parts

Related Bugs

Implementing these features, particularly 3a) and 3b), will have the side benefit of fixing (or partly fixing) a number of existing bugs related to messages or message parts being repeatedly downloaded:

• bug 345832 - IMAP attachments preloading (not on demand)
• bug 439731 - Reloading of read messages when using IMAP account (enable disk cache)
• bug 405437 - IMAP mail are not cached for offline use if message size is bigger than mail.imap.mime_parts_on_demand_threshold

How To Do It

With the exception of on-demand sync 1) and 2) can be implemented using strategies. These features and more will be supported in an extension, 'Incommunicado' (see below).

On-demand auto-sync (an oxymoron I know) however requires some development. Emre has proposed the following implementation:

• Add a new operation mode (lets say on-demand) to nsAutoSyncManager.
• Add new method to nsAutoSyncManager to explicitly tell to re-prioritize the queue.
• When running in on-demand mode, it should ignore idle events.
• When running in this mode, nsAutoSyncManager should keep downloading all pending messages until either it is all done, or the user explicitly says STOP.

Addressing 3) is non-trivial and will be tackled at a later stage, in the meantime it may be possible to avoid current problems by increasing the cache size and configuring it to save to disk rather than memory (bug 439731 #12).

Milestones

Thunderbird 3.0b2:

• on-demand auto-sync
• auto-sync strategy to exclude messages greater than MAX_SIZE (Incommunicado)
• on/off/on-demand auto-sync UI (Incommunicado)

Thunderbird 3.0pre1:

• save cache to disk and increase size bug 439731 #12. (Temporary fix)
• priority auto-sync folders (Incommunicado)
• advanced on-demand auto-sync runtime dialog (Incommunicado)
  • priority folders only
  • select alternate strategy (e.g. sync all messages)

February/March 2009:

• support MIME parts in offline store
• always cache messages or message parts not saved in offline store
• better coordination between cache and offline store (e.g. no duplication)

Incommunicado Extension

Target Features

Target features for first release of the Incommunicado extension in rough priority order

• Simple auto-sync strategy (see nsIAutoSyncManager)
  • only download messages greater than MAX_SIZE on user request
• User interface
  • add column showing state of message, i.e. o - header only, x - downloaded
  • icon to toggle offline/online plus support for additional states (below)
• Priority folders
  • user can select which folders are highest sync priority
• Incommunicado state
  • user can select 'incommunicado' in addition to online/offline
  • custom auto-sync strategy
    • only download messages greater than MAX_SIZE on user request
    • only priority folders are downloaded

Other Concerns

Some other features/behaviour of TB that may need to be investigated to have complete control over bandwidth usage:

• Auto-update
• Junk mail