Lightning Offline Sync

From MozillaWiki
Jump to: navigation, search

This page is for tracking the Lightning Offline Sync Feature (bug 380060) to be completed as part of the Google Summer of Code 2011 programme.

The Idea

Essentially the idea is to have the cached calendar function as a full-blown offline counterpart as well without having the need to implement any additional providers or managers. The information about offline user actions is retained in the cache and replayed when going online.

The state transition table below gives the states which the offline item will transition to so as to preserve minimum number of calls to the network providers.


Testing

Testing needs to be done for events, todos, recurring events and todos and the 3 modes, startup, online, offline.

Testing Scenarios
Scenario
Events
TODOs
Recurring Events and TODOs
Startup to online

Changes should be played back.

In case of conflicts (server copy changed) User is prompted For Full sync provider, Need to resolve the deletion of cache

Update offline flag

Changes should be played back.

In case of conflicts (server copy changed) User is prompted

Same
Offline to Online
Changes played back


Offline to offline
Changes remain in the cache, offline flag stays
Changes remain in the cache and viewed visually

Online to online
Playback the changes, update cache


Online to Offline
No refresh.



TODOs

Item Name

   Priority

Owner

Deadline

Progress

Comments

Adapt schema initialization and versioning code (calCalendarManager)


mohit


done

Added versioning code in calStorageUpgradeHelper.jsm file. cal_events and cal_todos have an extra "offline_journal" column. and DB SCHEMA Version is 20.

Add search flags "ITEM_FILTER_OFFLINE_DELETED", "ITEM_FILTER_OFFLINE_CREATED" and "ITEM_FILTER_OFFLINE_UPDATED"


wolfgang


done

Check whether "ITEM_FILTER_ALL_ITEMS" is used in the views, if so, we need to create "ITEM_FILTER_ALL_ONLINE_ITEMS" ("ITEM_FILTER_ALL_ITEMS" & ~ITEM_FILTER_OFFLINE_DELETED)


wolfgang


cancelled Not needed since ALL_ITEMS = 0xffff and our flags are = [29..31]

Adapt getItems in calStorageCalendar with the above and the GUI code if required


mohit


done

Adapt mSelectNonRecurringEventsByRange and this.mSelectNonRecurringTodosByRange queries


mohit


done

Write offline addItem method (calStorageCalendar::addOfflineItem)


mohit
done

To be discussed

Write offline delete method (calStorageCalendar::deleteOfflineItem)


mohit
done

Write offline modify method (calStorageCalendar::modifyOfflineItem)


mohit


done *Note* cachedCalendar::modifyItem deletes the old item before putting in the new item with the changes hence the offline_flag is lost. Solution: make offline_journal flag part of the interface.. 

Test the above


wolfgang/mohit
done Testing steps should be discussed

Write replay method


mohit


done

Write "switch code" in calCachedCalendar


wolfgang
done

switch code = offline to online mode and back

test the above


wolfgang/mohit


done

Testing steps shud be discussed

Design a workflow for handling errors pertaining to unavailability of server + replaying the changes when back online
mohit
done Needs discussion with Wolfgang afterwards
Implement the above workflow after discussion
mohit
done

Adapt code from Lightning 1.0b2 to HEAD


mohit


done Should be done at the very end of the above


Workflows

Offline operations

  • addItem:
  1. (entry) calCachedCalendar::addItem
  2. calStorageCalendar::addOfflineItem
    1. setup calOperationListener
    2. execute calStorageCalendar::addItem with above listener
    3. listener: update flag with 'c' value (see table below)
    4. listener: invoke original listener with result code
  • modifyItem
  1. (entry) calCachedCalendar::modifyItem
  2. calStorageCalendar::modifyOfflineItem
    1. setup calOperationListener
    2. execute calStorageCalendar::modifyItem with above listener
    3. listener: update flag with 'm' value (see table below)
    4. listener: invoke original listener with result code
  • deleteItem
  1. (entry) calCachedCalendar::deleteItem
  2. calStorageCalendar::deleteOfflineItem
    1. update flag with 'd' value (see table below)
    2. invoke original listener with result code

Note: we cannot delete the record yet as we will need to pass the complete calItem back to the online/uncached calendar during reconcilliation

  • getItem/getItems
  1. code forwarded immediately to the calStorageCalendar

Note: getItems excludes results with offline column set to 'd', unless the ITEM_FILTER_OFFLINE_DELETED flag is passed via the item filter

Transition between online and offline modes

Nothing seems to be required at this stage.

Transition between offline and online modes

New code must be added to calCachedCalendar::onOfflineStatusChanged:

  • manage deletions
  1. setup calOperationListener "listener1"
  2. invoke calStorageCalendar::getItems with ITEM_FILTER_OFFLINE_DELETE in item_filter and the above listener
  3. listener1 execution:
    1. onGetResult:
      1. setup empty calOperationListener "listener2"
      2. invoke calCachedCalendar::deleteItem (this) with each resulting events and listener2
    2. onOperationComplete:
      1. perform next step ("manage additions")
  • manage additions
  1. setup calOperationListener "listener1"
  2. invoke calStorageCalendar::getItems with ITEM_FILTER_OFFLINE_CREATED in item_filter and the above listener
  3. listener1 execution:
    1. onGetResult:
      1. setup empty calOperationListener "listener2"
      2. invoke calCachedCalendar::addItem (this) with each resulting events and listener2
    2. onOperationComplete:
      1. perform next step ("manage modifications")
  • manage additions
  1. setup calOperationListener "listener1"
  2. invoke calStorageCalendar::getItems with ITEM_FILTER_OFFLINE_MODIFIED in item_filter and the above listener
  3. listener1 execution:
    1. onGetResult:
      1. setup empty calOperationListener "listener2"
      2. invoke calCachedCalendar::modifyItem (this) with each resulting events and listener2
    2. onOperationComplete:
      1. perform calCachedCalendar::refresh (this)

Note: reconciliation must occur before calIChangeLog method because "uncached calendars" (at least calDavCalendar) contain code that manages conflicts during data upload. Otherwise, the local changes would immediately be overwritten by server-side changes.

Efficient model for state updates during offline operations

From/To Created Modified Deleted
(without state) Mark as "created" Mark as "modified" Mark as "deleted"
Created (none) Keep as "created" Delete record
Modified (none) Keep as "modified" Mark as deleted
Deleted (none) (none) (none)

Additional issues

  1. CalDAV provider does not handle offline mode at all due to the fact that it performs the same DAV checks every time it's instantiated
    1. (done) Calendar properties (organizerCN, organizerCN, imip.identity + CalDAV capabilities + items metadata) should be stored in the cache for offline use + most of those attributes should never be queried anymore
    2. (postponed) certain checks should be only performed in a separate method that should be executed when the calendar is created from the calendar creation dialog
    3. (done) the calDavCalendar does not know its storage counterpart when calICalendar operations are first invoked, this should preferably be done right after instantiation, especially since "cache mode" will become the sole available mode, sooner or later
  2. (cancelled: since those interfaces are not used by the same components, it makes sense to keep them distinct) we should probably merge calIOfflineStorage with calISyncWriteCalendar into a single interface as those are actually part of the same functionality (and choose the former name since it's more significant)
  3. any calendars that are not available due to network errors should automatically be put in offline mode and enable further operations
  4. more or less off-topic: the caldav conflict dialog does not show the differences between events

ICS Provider Issues

  1. ICS Provider knows nothing about the cache, hence it needs to be modified properly to allow it to make use of cache for playing back the entries to the server.
  2. Memory calendar assumes that every new item that is added needs to have no ID (it generates one for the item) This is not true for cached entries, they already have an entry.
  3. Possible loss of offline cached entries during a full sync.