Apps/Sync/Spec2

This document serves as an entirely speculative revision to the Apps/Sync/Spec specification, with an attempt to make it a bit cleaner and more general.

Expectations

The model of sync is a stream of updates. All clients both put their local updates into this stream, and read the collective stream. Everything has to be represented as a concrete item in the stream, meaning that delete actions are also present in the stream.

There is no conflict resolution, so clients must make sure they do not overwrite each other's updates. If a conflict cannot be resolved without interaction (e.g., simple overwrite is not considered acceptable, and automatic merging is not possible) then it must be possible to represent the conflicted state directly, and at some point some client can resolve the conflict (possibly with user interaction) and put the unconflicted object into the stream.

The stream is ordered, along a single timeline. The timeline markers should not be seen as based on any time or clock, as this leads to confusion and it's not clear whose "now" we are talking about. Instead the server has a counter, and all clients work from that counter. (The counter need not be an uninterrupted stream of integers, just increasing.)

All interaction between client and server should happen without user intervention. Everything is expected to be highly asynchronous, and the server may reject requests or be unavailable for short periods of time, and this should not affect user experience.

We expect for a new client to be able to create a good-enough duplicate of the data in other clients. "Good-enough" because some data might be kept by clients but expired by the server because it was marked as not being permanently interesting.

For "known" datatypes the sync server ensures the integrity of data, according to the most up-to-date notion of correctness for the data type. As such the sync server must be updated frequently, but clients will be protected from some other rogue clients. (Note: not sure if this is a practical expectation?)

Protocol

We'll go out-of-order time-wise, and forget about authentication for now.

Everything happens at a single URL endpoint, we'll call it /user

Each object looks like this:

   {type: "type_name",
    id: "unique identifier among objects of this type",
    expires: timestamp,
    data: {the thing itself}
   }

Note the data can be any JSONable object, including a string.

The expires key is entirely optional, and allows the server to delete the item (if it has not otherwise been updated).

The id key must be unique for the type (submitting another key by the same id means that you are overwriting that object).