User:Jminta/Steel

From MozillaWiki
Jump to: navigation, search

Below is a proposal for a FUEL-like set of interfaces to facilitate Thunderbird extension development. STEEL is the Scriptable Thunderbird Easy Extension Library.

Your comments are welcome on the discussion page or beneath the appropriate heading.

Motivations

In my view, the starting point for a FUEL-like library needs to be the intersections between significant numbers of existing Thunderbird extensions. These extensions are solving acknowledged problems for users, and where many of them follow similar patterns, we should streamline the process to encourage further innovation. Common extension patterns are

  • Sync - various extensions sync data within thunderbird to external sources (address book, preferences, calendar)
  • Message management - extensions allow new annotations or make it easier to add existing annotations to messages
  • Views on email - extensions to allow new views on existing mail
  • What else?

Under-exploited/common difficulty areas

  • Message body text alterations
  • Alternate mail-views, ie gmail's thread view, calendar based views, clouds

See also this rough bugzilla query.

Comments:

Implementation Plan

Because of the size of this project (and the mess of the mailnews interfaces) these interfaces should be implemented in stages. The following is a proposed implementation strategy.

STEEL 0.1 (basic mail operations)

all relevant FUEL interfaces

steelIAccount interface - basic implementation

  • Implement the all/has/get accessors
  • Full support for accessing folders,

steelIFolder interface - full implementation

steelIMessage interface - basic implementation

  • Limited support for tags
  • Limited support for altering message text

May just return email addresses instead of steelIPersons at this stage

This stage should include enough to allow for a non-RDF rewrite of the account pane, along with many other RDF-driven UI elements

STEEL 0.2 (more thunderbird features)

steelIAddressBook interface - full implementation

  • This likely depends on the switch to mozStorage

steelIPerson interface - full implemenation

  • see above

steelIAddressBooks interface - full implementation

steelIAccount interface

  • Full support for adding/modifying/deleting accounts

steelIMessage interface

  • Full support for tags

STEEL 0.3 (still more thunderbird functions)

steelISearchTerms interface - full implementation

implement the Application's search() function

steelIAccounts interface - full implementation

  • implement add() and delete() functions

Interfaces

steelIListener

/**
 * Many mail-news operations are asynchronous.  When accessing information that
 * requires an asynchronous result to be returned, you should pass that method a
 * steelIListener. (These methods will all return an id of to allow you to track
 * requests.)  Reports from the asynchronous methods will be returned through
 * this listener.
 */
[scriptable, uuid(edb2e41c-9ba7-489e-ba2a-1ff3312ca915)]
interface steelIListener : nsISupports {
  /**
   * Called when one or more results to your request are available.
   *
   * @param requestId  the id of the original request
   * @param result     an array of results of the appropriate type
   */
  void onGetResult(in AString requestId, in nsIVariant result);

  /**
   * Called when an operation has failed for some reason.
   *
   * @param requestId   the id of the original request
   * @param description a description of the error.
   *
   * @note An onError() call will be immediately followed by a call to
   * onOperationComplete.
   */
   void onError(in AString requestId, in AString description);

  /**
   * Called when all results to your request have been passed to onGetResult.
   *
   * @param requestId  the id of the original request
   * @param allResults an array of all results that were passed to onGetResult
   *
   * @note You can omit the onGetResult function if you simply wish to handle
   *       all results at once via this method.
   */
  void onOperationComplete(in AString requestId, in nsIVariant allResults);
};
Comments
  • srobertson: I've always found the listener style of async programing to be somewhat cumbersome. You end up creating all these little listener objects.

One idea I'll throw out here is to use use the "deferred object" approach popularized by Twisted and MochiKit. See http://www.mochikit.com/doc/html/MochiKit/Async.html

I think it's more of a natural fit for Javascript programers. They have the option to make their callback a function, or any other method of a Javascript object they choose.

Here's an example of what the code might look like for an extension doing a search.

Example 1: Using listeners. 

var MySearchListener = {  
  onGetResult : function (requestId, result){
    this.msgList.appendItem(result);
  },

  onOperationComplete(requestId, in nsIVariant allResults){
    // Operation is done, how do we know if it succeeded or failed? 
  }

} 

// Assume this method is an event handler for a button click

function onSearchClicked(){
  // Start a search that return all messages that contain 
  // thunderbird and steel
  MySearchListener.msgList = document.getElementByID('message-list');
  Application.search(['thunderbird', 'steel'], MySearchListener);
}

Example 2: Using deferred objects 

function onSearchClicked(){
  /* In this example, Application.search() returns a deferred object. 
     This is a promise that this operation will complete in the future.
  */ 

  var msgList = document.getElementByID('message-list');
  var deferred = Application.search('thunderbird', 'steel');
  // When a message is returned invoke the onNewMessage, 
  // notice that I can pass arbitrary values to my callback method. 
  // In this case I pass it the UI element that I'll update as new 
  // messages are found.

  deferred.addCallBack(onNewMessage, msgList);

  // And in case there are errors, call my onSearchError function.
  deferred.addErrBack(onSearchError, msgList); 
}

function onNewMessage(message, msgList){
  msgList.append(message);
}

function onSearchError(failure, msgList){
  // whoops we had an error, clear the msgList and display 
  // the problem to the user.
  msgList.clear();
  alert(failure);
}

jminta's reply: This is an interesting approach, and especially if this is the way other js frameworks are doing things, we should consider it. I should point out though that you can make your first approach look *almost* identical to the second, like so: 

function onSearchClicked(){
  var msgList = document.getElementByID('message-list');
  var deferred = {};
  Application.search('thunderbird', 'steel', deferred);

  deferred.onGetResult = onNewMessage;
  deferred.onError = onSearchError;
}
    • srobertson's retort: Yes you could write it like that. I like it, it's very much like XMLHTTPRequest which I think JS types would be familiar with. However I was glossing over the details of a deferred object which let you call addCallback multiple times to chain callbacks and errbacks together. An in depth guide to this technique can be found here http://twistedmatrix.com/projects/core/documentation/howto/defer.html To be honest 98% of the time I never need to chain callbacks, though it is neat when you get to use it.



A few other points:

  • Your point about a lack of status results/error handlers is crucial. I definitely think we should add an onError() method to the listener.
  • Your second approach allows arguments to be appended to listener calls. Unfortunately, the strict nature of IDL would make this difficult to implement, and we're stuck with those restrictions. (It could be done as an array of arguments, but that's getting weird.)
    • srobertson's reply You are correct, it is weird with the IDL. I'm not familiar with FUEL, I was assuming that FUEL was a thin layer of javascript on top of xpcom components. So for example the IDL could be done as an array of arguments and our thin layer of JS would hide the XPCOM weirdness from the average developer.
  • The notion of appending listener methods *after* a request has been dispatched strikes me as odd. I've written async methods that actually call their listeners *before* the dispatch call returns. Is this normal programming in other frameworks?
    • srobertson: It is indeed very odd the first time you encounter it. A deferred object keeps track of whether it has been fired or not, so if you call addCallback() on the deferred after it has been fired your call back is immediately executed. Generally since most of this is processed in a run loop of some sort, your code that is asking for a search to be preformed is actually scheduling the search to happen in the next pass of the run loop. However the deferred is returned in the current iteration of the run loop so your registering of callbacks always happens before the real search is prefromed. This approach is really common in async frameworks that don't use threads.


BenB: Personally, I like it best to be able to just pass a normal JS function as callback, and get the result of the function as parameter, and pass the error callback as other parameter. Then I can write this as inner function. I can even use the variables from the calling function. For example: 

var subject = myMsg.subject;
myMsg.getMessageContent(function(body)
{
  alert(subject + "\n" + body);
},
error);

This allows me to read heavy async code straight from top to bottom without jumping around from function to function, essentially *definining* one function per async function *call*.

error is defined somewhere in my code as 

function error(exception)
{
  ...
}

Mark Finkle told me you can accept JS functions as XPCOM params by defining an interface with [function, scriptable], see nsIDOMEvent.

If the results are trippling in, e.g. messages in a folder, you can use a different function which is called per result: 

var numberOfMails = 0;
myFolder.getMessages(function(msg) // once per message
{
  msg.getMessageContent(function(body)
  {
    dump(body + "\n");
    numberOfMails++;
  }, error);
},
function() // finished
{
  dump("OK, got all msgs in that folder, total " + numberOfMails + " mails\n");
},
error);

Of course you can convert the listener approach to that, but you have to create a helper object every time: 

var numberOfMails = 0;
myFolder.getMessages(new SteelListener(function(msg) // once per message
{
  msg.getMessageContent(new SteelListener(null, function(body)
  {
    dump(body + "\n");
    numberOfMails++;
  });
},
function() // finished
{
  dump("OK, got all msgs in that folder, total " + numberOfMails + " mails\n");
});


BenB: Why do you need the requestId? The caller can (and IMHO should) instantiate one object per function call, if he cares about where the results are coming from, and pass that instance whatever context it needs.

steelIApplication

/**
 * The core STEEL object, available in the global scope
 */
[scriptable, uuid(f265021a-7f1d-4b4b-bdc6-9aedca4d8f13)]
interface steelIApplication : nsISupports {
  /**
   * The steelIAccounts object
   */
  readonly attribute steelIAccounts accounts;
  /**
   * The steelIAddressBooks object
   */
  readonly attribute steelIAddressBooks addressBooks;

  /**
   * The steelIPreferences object
   */
  readonly attribute steelIPreferences preferences;

  /**
   * The steelIExtensions object
   */
  readonly attribute steelIExtensions extensions;

  /**
   * Will return all messages matching the search terms to the listener.
   *
   * @param terms  an array of the steelISearchTerms object for this search
   * @param listener  the listener to return results to
   *
   * @returns  an ID for this request
   *
   * @note multiple steelISearchTerms will be combined with an (inclusive) OR 
   * operator.  Call "add" multiple times on the same steelISearchTerms to use
   * the AND operator.
   */ 
  AString search([array] in steelISearchTerms terms, 
                 in steelIListener listener);

  /**
   * Returns a new steelISearchTerms object
   */
  steelISearchTerms getNewSearchTerms();

  /**
   * Adds an event listener to the appropriate event.  Note that we will fire a
   * lot of additional events here, including:
   *    onAccountCreated
   *    onAccountDeleted
   *    onAccountModified
   *    onNewMail
   *    onMessageShow
   *    xxx-what else?
   *
   * @param event    the name of the event to listen for
   * @param listener the steelIEventListener to call when the event happens
   */
   void  addListener(in AString event, in steelIEventListener listener);
};
Comments

steelIAccounts

[scriptable, uuid(db57aa8f-d9d9-47f8-8a4f-865d0a3661a8)]
interface steelIAccounts : nsISupports {
  /**
   * An array of all steelIAccounts for this Thunderbird profile
   */
  readonly attribute nsIVariant all;

  /**
   * Gets the specified steelIAccount
   *
   * XXX- what is the unique feature of account? We may just need to create our
   * own ids on the front-side here
   * 
   * @param id  the id for the address book
   */
  steelIAccount get(in AString id);

  /**
   * Returns true if the specified account exists
   *
   * @param id  the id for the account
   */
  boolean has(in AString id);

  /**
   * Adds this account to Thunderbird
   *
   * @param account  the steelIAccount for the new account to add
   */
   void add(in steelIAccount account);

  /**
   * Deletes this account from Thunderbird
   *
   * @param account  the steelIAccount to delete
   */
  void delete(in steelIAccount account);
};
Comments:

steelIAccount

/**
 * A simplified view of a mail account.
 */
[scriptable, uuid(42d33892-36e5-439b-8e57-7c473599b9a9)]
interface steelIAccount : nsISupports {
  /**
   * The (outgoing) address associated with this account
   */
  readonly attribute AString address;

  /**
   * The display name for this account
   */
  readonly attribute AString displayName;

  /**
   * An array of steelIFolder objects for the top-level folders of this account
   *
   * XXX- can we do this in a sync way?  May need a getFolders() method with a
   * listener
   */
  readonly attribute nsIVariant folders;

  /**
   * A constant corresponding to the type of the account
   */
  readonly attribute interger type;

  const unsigned long ACCOUNT_TYPE_IMAP = 1;
  const unsigned long ACCOUNT_TYPE_POP = 2;

  /**
   * Returns a blank message that can be sent from this account.
   */
  steelIMessage newMessage();
};
Comments:
  • srobertson: In response to your getFolders() comment. I'd encourage you to make the programming model as async as possible. If we avoid synchronous methods we can keep the UI responsive and not have to worry about whether this folder is local or remote on a mail server.
    • jminta's response: I agree that blocking the UI for a long time would be a bad idea, but at the same time, async requests are inherently more complex. Especially for get() methods, I think we win if we can abstract away the async part and expose a synchronous method to these consumers.
      • srobertson again: It's been my experience that you need to develop the knack for async. When you have toolkits that provide both async and sync everoyne will take the path of least resistane and go for the sync methods first because they are so easy to get your head around. Then they get disappointed because performance is so horrendous. I propose that you worry about async for version .1 and introduce sync methods in the future if there becomes a big demand for them. Just my two cents.
      • BenB agrees with srobertson

steelIFolder

/**
 * Interface representing a folder in a particular account
 */
[scriptable, uuid(65b542cf-d984-414c-955d-58367d834b2f)]
interface steelIFolder : nsISupports {
  /**
   * An array of steelIMessages for all of the unread messages in this folder
   */
  readonly attribute nsIVariant unreadMsgs;

  /**
   * An array of steelIMessages for *all* messages in this folder
   */
  readonly attribute nsIVariant allMsgs;

  /**
   * Determines if a message exists in this folder
   */
  boolean has(in AString msgID);

  /**
   * Returns the steelIMessage for the given ID (if it exists).
   */
  steelIMessage get(in AString msgID);

  /**
   * An array of steelIFolders for all subfolders of this folder.
   */
  readonly attribute nsIVariant subfolders;

  /**
   * Compacts this folder on the server.
   */
  void compact();

  /**
   * Marks all messages in the folder as read
   */
  void markAsRead();
};
Comments:
  • BenB: There should be a way to get the original nsI interface for this folder, same with Message and Account, because the STEEL API may not provide some more advanced attributes like quota or whatever, or the object may be required as parameter for some other function (which is not in the STEEL API).

steelIMessage

/**
 * Interface representing a folder in a particular account
 */
[scriptable, uuid(cad7b25c-f59f-409c-8b43-0972afc5e6ce)]
interface steelIMessage : nsISupports {
  /**
   * A unique id for this message
   */
   readonly attribute AString id;

  /**
   * Some messages cannot have their attributes edited.  For instance, if the
   * message has been received, trying to alter the from address would be 
   * non-sensical.  This attribute 
  readonly attribute boolean isMutable;

  /**
   * If you want to work from a new copy of this message, you can clone it. This
   * message will initially be mutable.
   */
  steelIMessage clone();

  /**
   * Get a new steelIMessage for replying to this message.  It will have the to,
   * from, and account fields already set.  The body attribute will be the
   * quoted text of this current message;
   *
   * @param replyToAll  If true, all recipients will be added to the "to" field
   *                    for the message.
   */
  steelIMessage reply(in boolean replyToAll);

  /**
   * Get a new steelIMessage for forwarding this message.  The "to" field will
   * be blank and will need to be set before you send the message.
   */
  steelIMessage forward();

  /**
   * Actually send this message
   */
  void send();

  /**
   * Deletes the message from the server
   */
   void delete();

  /**
   * A steelIPerson object corresponding to the sender of this message
   */
  attribute steelIPerson from;

  /**
   * An array of steelIPersons that the message is directed to
   */
  attribute nsIVariant to;

  /**
   * The subject of this message
   */
  attribute AString subject;

  /**
   * The text of this message.  Note that in the case this is an html message,
   * this will be the actual HTML source.
   */
  attribute AString body;

  /**
   * Whether or not the message has been marked read.
   */
  attribute boolean read;

  /**
   * An integer priority for this message
   * xxx-range?
   */
   attribute integer priority;

  /**
   * Whether or not the message has been flagged for follow-up
   */
   attribute boolean flagged;

  /**
   * Array of strings for the tags for this message
   */
   readonly attribute nsIVariant tags;

   /**
    * Adds an additional tag to this message
    */
   void addTag(in steelITag tag);

   /**
    * Removes a particular tag from this message
    */
   void removeTag(in steelITag tag);

  /**
   * Whether or not the message has been marked is junk.
   */
  attribute boolean junk;

  /**
   * A javascript date object for the date and time the message was transmitted.
   */
  readonly attribute nsIVariant date;

  /**
   * An array of nsIFile objects for the message's attachments
   */
  readonly attribute nsIVariant attachments;
};
Comments:

We might want to steal methods like 

  walk() Returns an  iterator for processing each  part of a multipart mime message
getPayLoad(part, decode) To access an individual part of a message and automatically decode it from mime to text.
attach(payload) adding additional parts to a message.
  • Eyalroz: I also think there's a need for a message to be manipulable as a hierarchy of MIME parts. However:
  1. Iterating is not enough. One needs to be able to add and remove parts in the tree.
  2. The backend has terribly poor support for this. Just look at the state of libmime. (Also have a look at my aborted attempt in bug 248846)
  • jcranmer: Rather long list; sorry if it seems like I'm being a tad specific here, but I'm thinking more from an implementor's point of view...
    • Perhaps have both a MIME view and an attachment view? Some users may prefer just getting a list of all attachments (needs to be somewhat more rigorously defined) while others may want precise MIME multipart access.
      • jminta reply: This is starting to feel a bit too technical for entry-level extension writers. If you can show me basic extension plans that involve mime-specific stuff, I could be persuaded though.
      • pbrunschwig: take e.g. Enigmail. It needs to access encapsulated PGP/MIME parts as created by Mailman or as specified in RFC 3156, section 6.1. And it doesn't care at all for any (file) attachments to the message. Today this is damn hard or almost impossible to achieve.
    • Generic header get/setters. sync may be problematic here: IMAP/NNTP will not have these headers available by default except under certain circumstances.
      • jminta reply: Yeah, I'm starting to come over to the side of basic header manipulation, even though I consciously left it off in the beginning.
    • The forward method may want to be somewhat more fine-tuned (i.e., potentially overriding forward preference settings).
      • jminta reply: I think this is ok because the forward() method simply returns a steelIMessage that you can modify before you call send(). And since we'll have easy pref getters, applying additional forwarding prefs is simple.
    • What is the definition of body? Suppose I have a multipart/alternative message--do we show only the first one? All of them? What about other multipart/s? What happens if I have a multipart/mixed with a Content-Disposition: inline? Need to define attachments a little more clearly here as well.
      • jminta reply: I'm inclined to just say it's the text representation of the message. For more intricate messages/display bits I think extension authors should use the regular interfaces. Again, if this turns out to be a common use case for extensions, we can expand it, but I haven't seen data to suggest it
    • an enumerator of headers as well as as getters and potentially setters would be a fine thing

steelIPerson

[scriptable, uuid(b1052725-4531-449c-becf-7b16b26b8f23)]
interface steelIPerson : nsISupports {
  /**
   * The default (primary) email address for this person.  You should get other
   * addresses via the get() method.
   */
  attribute AString defaultAddress;

  /**
   * The display name for this person.  You can get and set the first/last name
   * properties via the appropriate methods.
   */
  attribute AString displayName;

  /**
   * Gets a particular property of this person, e.g. phone, website, etc
   *
   * @param property  the name of the property to get
   */
  nsIVariant get(in AString property);

  /**
   * Returns true if there is some value set for this property, false otherwise
   *
   * @param property  the name of the property to check
   */
  boolean has(in AString property);

  /**
   * Sets a particular property for this person.
   *
   * @param property  the name of the property to set
   * @param value     the value of the property
   *
   * @note  If this person is not in an address book, setting a property will
   *        only alter the current session.  If the person is in a book, setting
   *        it will alter that person's entry in the book.
   */
  void set(in AString property, in nsIVariant value);

  /**
   * The address book this card is a part of.  Null if the person is not in any
   * book.
   */
  attribute steelIAddressBook addressbook;
};
Comments:

Its unclear to me how you're going to handle a) big updates, b) asynchronous connections.

So for instance, someone creates a new card and fills in every field, the extension would come along and call set(property, value) for each one. Will we be writing to the database for each one, or can we somehow batch the updates?

Admittedly the address book doesn't handle updating of cards on asynchronous connections (i.e. ldap) at the moment but that is planned. It would be nice to see something in here for how we should handle that. I'm not sure what form this would take at the moment.

--Standard8 11:36, 27 November 2007 (PST)

jminta reply: I'm not terribly concerned about the async case for the purposes of STEEL. To me, this feels like an advanced case that normal extension writers won't encounter. Those that want to really flex their LDAP muscles could still use the normal XPCOM interfaces. If we did want to implement this, we could have a statusListener attribute that takes a steelIListener that would be passed results when the async transactions complete.

Standard8 reply: If its possible, I'd rather we add the flexibility in there to being with, rather than the exception to the rule. The mailnews code has already suffered with some of the implementation being the local ab (mork) specific, and I think some extensions have as well. Now we have things like Mac OS X address book and the possibility of writeable ldap before 3.x (and maybe even an sql backend) we should be encouraging extensions to be generic across their handling of different address book types.

steelIAddressBooks

[scriptable, uuid(3c964f9d-d7e8-4967-bbdf-e481aaffcf6e)]
interface steelIAddressBooks : nsISupports {
  /**
   * An array of all steelIAcddressBooks for this Thunderbird profile
   */
  readonly attribute nsIVariant all;

  /**
   * Gets the specified steelIAddressBook
   *
   * XXX- what is the unique feature of a-book? We may just need to create our
   * own ids on the front-side here
   * 
   * @param id  the id for the address book
   */
  steelIAddressBook get(in AString id);

  /**
   * Returns true if the specified addressBook exists
   *
   * @param id  the id for the addressBook
   */
  boolean has(in AString id);

  /**
   * Adds this addressBook to Thunderbird
   *
   * @param account  the steelIAddressBook for the new account to add
   */
   void add(in steelIAddressBook addressBook);

  /**
   * Deletes this addressBook from Thunderbird
   *
   * @param addressBook  the steelIAddressBook to delete
   */
  void delete(in steelIAddressBook addressBook);
};
Comments:

Currently the unique feature of an address book is the RDF resource URI/pref id. I wouldn't want us to use the RDF resource URI at the moment because its really bad in the mork case (moz-abmdbdirectory://abook.mab), but likewise I'm not sure if I really want to expose the pref id. I'd like to change the RDF resource URI mork, so maybe that would be what to pick up, but possibly some further thought/discussion is required here. --Standard8 12:06, 27 November 2007 (PST)

steelIAddressBook

[scriptable, uuid(0f5f5f51-5f6d-4853-ba61-a2f734d78e18)]
interface steelIAddressBook : nsISupports {
  /**
   * An array of steelIPersons (and steelIAddressBooks) for all cards in this
   * address book.  steelIAddressBooks will be returned for mailing lists. You
   * should use the instanceof operator, or the QueryInterface method to check
   * that you have the people.
   */
  readonly attribute nsIVariant cards;

  /**
   * Adds a person (or mailing list) to this address book.
   *
   * @param card  the steelIPerson or steelIAddressBook to add
   */
  void add(in nsIVariant card);

  /**
   * Deletes a person (or mailing list) to this address book.
   *
   * @param card  the steelIPerson or steelIAddressBook to delete
   */  
  void delete(in nsIVariant card);

  /**
   * Returns true if this person (or mailing list) is in this address book
   *
   * @param card  the steelIPerson or steelIAddressBook to check
   */ 
  boolean has(in steelIPerson);

  /**
   * Returns a steelIPerson object for the given email address
   *
   * @param email  the email address of the person to get.
  steelIPerson get(in AString email);
};
Comments:

steelISearchTerms

[scriptable, uuid(2c2f46f7-c22e-4406-aba4-6f995f23c317)]
interface steelISearchTerms : nsISupports {
  /**
   * Adds an additional query term to the search.
   *
   * @param field    the field of messages to search
   * @param value    the value to locate in the field
   *
   * @note Multiple calls to "add" will be combined with an AND operator
   */
  void add(in AString field, in AString value);

  /**
   * Removes all current query terms from the object
   */
  void clear();

  /**
   * If true, the search will exclude all items matching this query, rather than
   * including them.  False by default.
   */
  attribute boolean isNegative;
};
Comments:

FUEL Interfaces

The following interfaces can be copied straight over from FUEL:

Retrieved from "https://wiki.mozilla.org/index.php?title=User:Jminta/Steel&oldid=109373"