canmove, Confirmed users
2,237
edits
MailNews:Minimizing Bandwidth Usage: Difference between revisions
MailNews:Minimizing Bandwidth Usage (view source)
Revision as of 23:18, 23 August 2015
, 23 August 2015→Related Bugs: update bug status
(Short introduction) |
m (→Related Bugs: update bug status) |
||
| (17 intermediate revisions by one other user not shown) | |||
| Line 1: | Line 1: | ||
Make Thunderbird more friendly for unreliable and low-bandwidth conditions. | Make Thunderbird more friendly for unreliable and low-bandwidth conditions. | ||
Thunderbird currently has a number of features that | Thunderbird currently has a number of features that ''could'' help in these conditions, such as 'offline folders', 'MIME parts on-demand' and caching. But these features do not always work well in both unreliable '''and''' low-bandwidth conditions '''or''' when combined together. | ||
This wiki pulls together inter-related issues in one place for discussion and to propose a solution. | This wiki pulls together inter-related issues in one place for discussion and to propose a solution. | ||
== User Experience (UX) == | |||
The desired user experience in unreliable/low-bandwidth conditions could be described as: | |||
'''''Only download what I ask for and only download it once.''''' | |||
There are a number of bugs/features at the moment that are stopping this from happening, namely {{bug|405437}}, {{bug|345832}}, {{bug|439731}} and {{bug|470624}} (see [[#Related Bugs |Related Bugs]]). | |||
=== Desired Behaviour === | |||
1) Only download messages that I ask for | |||
2) Once a message or part of a message (MIME part) has been downloaded it should '''always''' be stored locally: | |||
*in offline storage if folder is __offline__ '''and''' storage rules allow (e.g. diskspace) ''OR'' | |||
*in cache | |||
3) A message should '''always''' be retrieved from local storage if possible. | |||
=== Current Behaviour === | |||
'''''NOTE:''''' ''This refers to TB3, however currently the only significant difference is the new auto-sync feature in TB3.'' | |||
1) Auto-sync (on by default) will download '''all''' messages in an __offline__ folder. We may not want this in all cases i.e. for large messages. | |||
2) Auto-sync meets the desired behaviour in most cases ([[MailNews:Better Faster IMAP Plan|see here for more information]]). So lets just look at scenarios in which auto-sync is not applicable i.e. non __offline__ folders and messages that we do not want to be auto-synced. | |||
: a) If all non-inline MIME parts < ~30KB: | |||
* message is downloaded and stored when viewed | |||
: b) If one or more non-inline MIME parts > ~ 30KB: | |||
* MIME parts are downloaded on-demand and cached ''sometimes'' (there are constraints such as cache size) | |||
3) Messages in offline store are retrieved from local storage. Cached MIME parts are retrieved from cache, however large MIME parts (i.e. attachments are rarely cached) | |||
'''''NOTE:''''' ''Effectively this means that messages with large MIME parts (i.e. attachments) are not cached and not available offline. They have to be downloaded each time they are viewed.'' | |||
== Implementation == | |||
=== What Needs To Be Done === | |||
Although poor handling of MIME parts is a major obstacle to minimising bandwidth usage in TB it is also the most difficult to address. To get immediate results a low-hanging fruit approach is proposed, in rough order of priority: | |||
1) Add on/off/on-demand features to auto-sync giving user full control of when sync takes place | |||
2) User configurable auto-syncs strategies to prioritise and include/exclude messages downloaded by auto-sync | |||
3) Better handling of MIME parts so they are only downloaded once | |||
: a) Caching of large MIME parts e.g. attachments | |||
: b) Support for offline storage of MIME parts | |||
=== Related Bugs === | |||
Implementing these features, particularly 3a) and 3b), will have the side benefit of fixing (or partly fixing) a number of existing bugs related to messages or message parts being repeatedly downloaded: | |||
* <strike>{{bug|345832}}</strike> - INVALID - IMAP attachments preloading (not on demand) | |||
* <strike>{{bug|439731}}</strike> - FIXED - Reloading of read messages when using IMAP account (enable disk cache) | |||
* <strike>{{bug|405437}}</strike> - WFM IMAP mail are not cached for offline use if message size is bigger than mail.imap.mime_parts_on_demand_threshold | |||
* {{bug|470624}} - described in this wiki - RFE: On-demand Auto-Sync | |||
=== How To Do It === | |||
With the exception of on-demand sync 1) and 2) can be implemented using strategies. These features and more will be supported in an extension, [[#Incommunicado Extension | Incommunicado]]. | |||
On-demand auto-sync ({{bug|470624}}) (an oxymoron I know) however requires some development. [[User:Emre|Emre]] has proposed the following implementation: | |||
* Add a new operation mode (lets say on-demand) to nsAutoSyncManager. | |||
* Add new method to nsAutoSyncManager to explicitly tell to re-prioritize the queue. | |||
* When running in on-demand mode, it should ignore idle events. | |||
* When running in this mode, nsAutoSyncManager should keep downloading all pending messages until either it is all done, or the user explicitly says STOP. | |||
Addressing 3) is non-trivial and will be tackled at a later stage, in the meantime it may be possible to avoid current problems by increasing the cache size and configuring it to save to disk rather than memory ([https://bugzilla.mozilla.org/show_bug.cgi?id=439731#c12 bug 439731 #12]). | |||
==== On-Demand Sync ==== | |||
===== State Diagram ===== | |||
The finite state diagram below describes the sync states of a folder during on-demand sync. This is based on how auto-sync currently works, the notable differences are: | |||
# The SyncCompleted and SyncFailed '''end''' states. In auto-sync a folder is simply removed from the queue either because all of its messages have been downloaded or the number of retries was exceeded. | |||
# Substates are clearly defined. In auto-sync the substates ReadyToDiscover and ReadyToUpdate are inferred by a folder being in the Discover or Update queue respectively. In on-demand sync a folder's state will need to be explicitly defined either by a substate property or by a function that checks other attributes such as the state of internal queues. | |||
# Folders can only be in one state at a time. This is implied by the previous point. In auto-sync a folder can be in more than one queue at a time (I think), which as an implicit state breaks the FSM model. | |||
<small> | |||
<small> | |||
<pre> | |||
.---------------. | |||
. --------------- . | |||
|| SyncCompleted || | |||
' --------------- ' | |||
'---------------' | |||
^ | |||
| | |||
/ | |||
. discoverMessages() | |||
| .-------. | |||
syncCompleted() / \ | |||
. ' . | |||
\ pendingHeaders() > 0 | | pendingMessages() > 0 downloadMessages() | |||
------. \ .----------------. \ v .----------------. .----------------. | |||
/ ------------- / `-> --------------- / `-> --------------- / `-> ------------------ | |||
------>|IdleCompleted| |ReadyToDiscover| |ReadyToDownload| |DownloadInProgress| | |||
------------- <- / --------------- <- / --------------- <- / ------------------ | |||
^ \ `-----------------' `-----------------' `-----------------' ^ \ | |||
| \ pendingHeaders() = 0 pendingMessages() = 0 downloadOK() | \ | |||
/ \ / \ | |||
/ . . . | |||
/ | | downloadFail() | |||
/ headersOutOfDate() | | | |||
/ | | | | |||
. | retriesRemaing() > 0 | | |||
| . . . | |||
updateOK() / \ / | |||
| | \ | | |||
| v \ v | |||
| ------------- ------------ | |||
| |ReadyToUpdate| |DownloadFail| | |||
| ------------- ------------ | |||
| \ \ | |||
| \ \ | |||
| . . | |||
. | retriesRemaing() = 0 | |||
\ updateHeaders() | | |||
\ | . | |||
\ . / | |||
\ / | | |||
\ | updateFail() retriesRemaining() = 0 v | |||
\ v .-------------------------------------. .----------------------------------------. .------------. | |||
---------------- / `-> ---------- / `->. ------------ . | |||
|UpdateInProgress| |UpdateFail| || SyncFailed || | |||
---------------- <- / ---------- ' ------------ ' | |||
`--------------------------------------' '------------' | |||
retriesRemaing() > 0 | |||
</pre> | |||
</small> | |||
</small> | |||
=== Milestones === | |||
'''Thunderbird 3.0b2:''' | |||
* on-demand auto-sync ({{bug|470624}}) | |||
* auto-sync strategy to exclude messages greater than MAX_SIZE (Incommunicado) | |||
* on/off/on-demand auto-sync UI (Incommunicado) | |||
'''Thunderbird 3.0pre1:''' | |||
* save cache to disk and increase size ([https://bugzilla.mozilla.org/show_bug.cgi?id=439731#c12 bug 439731 #12]) (Temporary fix) | |||
* priority auto-sync folders (Incommunicado) | |||
* advanced on-demand auto-sync runtime dialog (Incommunicado) | |||
** priority folders only | |||
** select alternate strategy (e.g. sync all messages) | |||
'''February/March 2009:''' | |||
* support MIME parts in offline store | |||
* always cache messages or message parts '''not''' saved in offline store | |||
* better coordination between cache and offline store (e.g. no duplication) | |||
== Incommunicado Extension == | |||
=== Target Features === | |||
Target features for first release of the Incommunicado extension in rough priority order | |||
* Simple auto-sync strategy (see nsIAutoSyncManager) | |||
** only download messages greater than MAX_SIZE on user request | |||
* User interface | |||
** add column showing state of message, i.e. o - header only, x - downloaded | |||
** icon to toggle offline/online plus support for additional states (below) | |||
* Priority folders | |||
** user can select which folders are highest sync priority | |||
* Incommunicado state | |||
** user can select 'incommunicado' in addition to online/offline | |||
** custom auto-sync strategy | |||
*** only download messages greater than MAX_SIZE on user request | |||
*** only priority folders are downloaded | |||
== Other Concerns == | |||
Some other features/behaviour of TB that may need to be investigated to have complete control over bandwidth usage: | |||
* Auto-update | |||
* Junk mail | |||