Calendar:Performance and Speed

There's been some talk recently about the speed of the calendar backend. The large motivation for the back-end rewrite was to try and improve performance with large calendar files. However, recent reports that current performance wins, although signficant, are not substantial enough for some use-cases.

The Problem

I spent considerable time examining the code flow and profiling various aspects of getItems in calStorageCalendar.js. The following data was obtained for a calendar of approximately 1200 items, showing the month-view for February 2006. Therefore, getItems was called with the following arguments

 this.mCalendar.getItems(this.mCalendar.ITEM_FILTER_COMPLETED_ALL |
                      this.mCalendar.ITEM_FILTER_CLASS_OCCURRENCES |
                      this.mCalendar.ITEM_FILTER_TYPE_ALL,
                              0,
                              29 Jan 2006
                              4 March 2006,
                              this.mOperationListener);

The average time for this call to getItems to complete was approximately 3500ms. Of this, about 750ms was spent within the actual views creating and laying out the boxes. The problem is therefore that it takes 2750ms to return the appropriate items.

During this time about 400ms was spent in getEventFromRow, and crucially, about 2000ms was spent in getAdditionalDataForItem. Preliminary tests indicate that the largest percentage of this time (about 30%) was spent in setting additional properties on the item in question (see #1270).

The important fact to note is that in most cases, the vast majority of information that was added to the item object is unused.

The proposal

This is only a straw-man sketch. I plan to flesh this out further in the near future.

• getItems needs to accept an additional parameter that specifies the list of properties that are needed to be added to the item. No other items will be added to the item before it is returned.
• This means that the item will no longer be suitable for direct editing. Instead, we'll need to rely much more heavily on getItem calls to actually obtain a complete object for editing. This, however, requires that the full expense of building an item from a row is only expended when absolutely necessary.

Alternative Solution

A different approach to solve the problem is to have a special calIItemBase implementation. This implementation will fetch the data from the sqlite database on request. This means that it doesn't need the expensive construction.

The advantage is that no interfaces are changed, so the callers can stay unchanged. This makes this easier to implement. When implementing, it should be tried to share code with the current impl, where possible.

• jminta: I'd argue that the callers would change slightly, because it ought to be able to specify whether it wants to use this special calIItemBase implementation. We would add another flag to pass into get items. In addition to things like ITEM_FILTER_CLASS_OCCURRENCES, we would have USE_DYNAMIC_ITEM. There are some callers that probably want a complete item (like the unifinder with its many columns), but others, like the views, do want this dynamic item. Both dynamic and non-dynamic items would implement the same interface, but the underlying structure would be different. Also note that this would make it much more difficult to implement the changes in Calendar:Architecture because this *requires* the ability to have multiple copies of the same object, some dynamic and some not.

• mvl I don't see why callers would want the non-dynamic version. When they need certain information, they will get it. Wether that information is already in memory or still only in sqlite doesn't matter. Just as long as there is a right amount of caching, so that you don't have to get everything again when scrolling the unifinder, for example.