Lightning Offline Sync
This page is for tracking the Lightning Offline Sync Feature (bug 380060) to be completed as part of the Google Summer of Code 2011 programme.
Contents
The Idea
Essentially the idea is to have the cached calendar function as a full-blown offline counterpart as well without having the need to implement any additional providers or managers. The information about offline user actions is retained in the cache and replayed when going online.
The state transition table below gives the states which the offline item will transition to so as to preserve minimum number of calls to the network providers.
Testing
Testing needs to be done for events, todos, recurring events and todos and the 3 modes, startup, online, offline.
Scenario |
Events |
TODOs |
Recurring Events and TODOs |
Startup to online |
Changes should be played back. In case of conflicts (server copy changed) User is prompted For Full sync provider, Need to resolve the deletion of cache Update offline flag |
Changes should be played back.
In case of conflicts (server copy changed) User is prompted |
Same |
Offline to Online |
Changes played back |
|
|
Offline to offline |
Changes remain in the cache, offline flag stays |
Changes remain in the cache and viewed visually |
|
Online to online |
Playback the changes, update cache |
|
|
Online to Offline |
No refresh. |
|
|
TODOs
Item Name |
Priority |
Owner |
Deadline |
Progress |
Comments |
---|---|---|---|---|---|
Adapt schema initialization and versioning code (calCalendarManager) |
|
mohit |
|
done |
Added versioning code in calStorageUpgradeHelper.jsm file. cal_events and cal_todos have an extra "offline_journal" column. and DB SCHEMA Version is 20. |
Add search flags "ITEM_FILTER_OFFLINE_DELETED", "ITEM_FILTER_OFFLINE_CREATED" and "ITEM_FILTER_OFFLINE_UPDATED" |
|
wolfgang |
|
done | |
|
|
wolfgang |
|
cancelled | Not needed since ALL_ITEMS = 0xffff and our flags are = [29..31] |
Adapt getItems in calStorageCalendar with the above |
|
mohit |
|
done | |
Adapt mSelectNonRecurringEventsByRange and this.mSelectNonRecurringTodosByRange queries |
|
mohit |
|
done | |
Write offline addItem method (calStorageCalendar::addOfflineItem) |
|
mohit | |
done |
To be discussed |
Write offline delete method (calStorageCalendar::deleteOfflineItem) |
|
mohit | |
done | |
Write offline modify method (calStorageCalendar::modifyOfflineItem) |
|
mohit |
|
done | *Note* cachedCalendar::modifyItem deletes the old item before putting in the new item with the changes hence the offline_flag is lost. Solution: make offline_journal flag part of the interface.. |
Test the above |
|
wolfgang/mohit | |
done | Testing steps should be discussed |
Write replay method |
|
mohit |
|
done | |
Write "switch code" in calCachedCalendar |
|
wolfgang | |
done |
switch code = offline to online mode and back |
test the above |
|
wolfgang/mohit |
|
done |
Testing steps shud be discussed |
Design a workflow for handling errors pertaining to unavailability of server + replaying the changes when back online | |
mohit | |
done | Needs discussion with Wolfgang afterwards |
Implement the above workflow after discussion | |
mohit | |
done | |
Adapt code from Lightning 1.0b2 to HEAD |
|
mohit |
|
done | Should be done at the very end of the above |
Workflows
Offline operations
- addItem:
- (entry) calCachedCalendar::addItem
- calStorageCalendar::addOfflineItem
- setup calOperationListener
- execute calStorageCalendar::addItem with above listener
- listener: update flag with 'c' value (see table below)
- listener: invoke original listener with result code
- modifyItem
- (entry) calCachedCalendar::modifyItem
- calStorageCalendar::modifyOfflineItem
- setup calOperationListener
- execute calStorageCalendar::modifyItem with above listener
- listener: update flag with 'm' value (see table below)
- listener: invoke original listener with result code
- deleteItem
- (entry) calCachedCalendar::deleteItem
- calStorageCalendar::deleteOfflineItem
- update flag with 'd' value (see table below)
- invoke original listener with result code
Note: we cannot delete the record yet as we will need to pass the complete calItem back to the online/uncached calendar during reconcilliation
- getItem/getItems
- code forwarded immediately to the calStorageCalendar
Note: getItems excludes results with offline column set to 'd', unless the ITEM_FILTER_OFFLINE_DELETED flag is passed via the item filter
Transition between online and offline modes
Nothing seems to be required at this stage.
Transition between offline and online modes
New code must be added to calCachedCalendar::onOfflineStatusChanged:
- manage deletions
- setup calOperationListener "listener1"
- invoke calStorageCalendar::getItems with ITEM_FILTER_OFFLINE_DELETE in item_filter and the above listener
- listener1 execution:
- onGetResult:
- setup empty calOperationListener "listener2"
- invoke calCachedCalendar::deleteItem (this) with each resulting events and listener2
- onOperationComplete:
- perform next step ("manage additions")
- onGetResult:
- manage additions
- setup calOperationListener "listener1"
- invoke calStorageCalendar::getItems with ITEM_FILTER_OFFLINE_CREATED in item_filter and the above listener
- listener1 execution:
- onGetResult:
- setup empty calOperationListener "listener2"
- invoke calCachedCalendar::addItem (this) with each resulting events and listener2
- onOperationComplete:
- perform next step ("manage modifications")
- onGetResult:
- manage additions
- setup calOperationListener "listener1"
- invoke calStorageCalendar::getItems with ITEM_FILTER_OFFLINE_MODIFIED in item_filter and the above listener
- listener1 execution:
- onGetResult:
- setup empty calOperationListener "listener2"
- invoke calCachedCalendar::modifyItem (this) with each resulting events and listener2
- onOperationComplete:
- perform calCachedCalendar::refresh (this)
- onGetResult:
Note: reconciliation must occur before calIChangeLog method because "uncached calendars" (at least calDavCalendar) contain code that manages conflicts during data upload. Otherwise, the local changes would immediately be overwritten by server-side changes.
Efficient model for state updates during offline operations
From/To | Created | Modified | Deleted |
---|---|---|---|
(without state) | Mark as "created" | Mark as "modified" | Mark as "deleted" |
Created | (none) | Keep as "created" | Delete record |
Modified | (none) | Keep as "modified" | Mark as deleted |
Deleted | (none) | (none) | (none) |
Additional issues
- CalDAV provider does not handle offline mode at all due to the fact that it performs the same DAV checks every time it's instantiated
- (done) Calendar properties (organizerCN, organizerCN, imip.identity + CalDAV capabilities + items metadata) should be stored in the cache for offline use + most of those attributes should never be queried anymore
- (postponed) certain checks should be only performed in a separate method that should be executed when the calendar is created from the calendar creation dialog
- (done) the calDavCalendar does not know its storage counterpart when calICalendar operations are first invoked, this should preferably be done right after instantiation, especially since "cache mode" will become the sole available mode, sooner or later
- (cancelled: since those interfaces are not used by the same components, it makes sense to keep them distinct)
we should probably merge calIOfflineStorage with calISyncWriteCalendar into a single interface as those are actually part of the same functionality (and choose the former name since it's more significant) - any calendars that are not available due to network errors should automatically be put in offline mode and enable further operations
- more or less off-topic: the caldav conflict dialog does not show the differences between events
ICS Provider Issues
- ICS Provider knows nothing about the cache, hence it needs to be modified properly to allow it to make use of cache for playing back the entries to the server.
- Memory calendar assumes that every new item that is added needs to have no ID (it generates one for the item) This is not true for cached entries, they already have an entry.
- Possible loss of offline cached entries during a full sync.