Thunderbird:Thunderbird Post 3.1 BackendPlans

1. Pluggable Mail Store

See bug 402392. There are two main goals here - replacing our Berkeley Mailbox store with a MailDir implementation by default, to avoid all the issues that arise from having all messages in a folder in a single file (compaction, mailbox contention, virus checker deleting file, inability to do incremental backup, etc), and allowing other stores to be implemented (e.g., a sqlite store).

This entails making the backend code use the pluggable store interface instead of assuming berkeley mailbox, implementing a berkeley mailbox store plugin, implementing a maildir store plugin, and writing some migration code to migrate from berkeley mailbox format to maildir.

My thinking is to have maildir be the default store going foward, in the release after Miramar. We will most likely migrate profiles by default, instead of just using MailDir for new accounts/profiles. We may have UI to allow the user to stick with Berkeley Mailbox format, if only to optionally maintain compatibility with previous versions of Thunderbird.

2. Make it easier to get new communication data into Thunderbird.

Currently, this is very hard to do, as jcranmer and rkent have discovered, trying to add web-forums and Exchange messages respectively, into Thunderbird. It entails adding a new nsIMsgIncomingServer object, new nsIMsgFolder object, new nsIMsgService service, etc. The first two contain dozens and dozens of attributes and methods, most of which are implemented by common base classes (e.g., nsMsgDBFolder, nsMsgIncomingServer). Doing a new data source in js requires implementing all these methods by hand. Doing it in C++ is only marginally easier, if you inherit from the common base classes, or delegate to them.

Current Approaches, as I understand them:

RSS approach - RSS inherits from nsLocalMailFolder, nsMailboxServer, and implements nsRssService. It stores messages in the local mail folder, and each message essentially contains a link to the rss message. For simple datasources, like Twitter, this would be a fairly simple approach.

RKent's Exchange approach - tweak the backend code to allow instantiation of the common base classes, and implement new objects that delegate most methods to the core classes, overriding only the ones that it needs to specialize.

Jcranmer's Web-forum approach - Allow js objects to extend the core c++ objects, and implement the web-forum specific things in the js objects.

Possible new approaches:

Implement a library that does most of the common implementation, with a pluggable mechanism for doing the data source-specific things (e.g., enumerating containers/folders, pulling in messages, etc).

Hook in at the folder pane level - extension would add a new ftvItem sub-class, and handle everything at the UI level. The select handler would load a new deck into what was the thread pane, and handle clicks on items loading something to a new deck in the message pane, etc. In other words, the extension would have to implement everything, including command handling.

Replacing Mork with Sqlite

Although we can now open Mork databases asynchronously, Mork's all in-memory design, among many other things, makes it desirable to switch to a more scalable database. Asuth has suggested using a very simple schema, where each row consists of a key and a json-like blob representation of a msg hdr.