This is a working proposal for the backend storage architecture of PiCL server. It's based on a massively-sharded and cross-DC-replicated MySQL installation, and is far from final. All feedback welcome!
The goals areGoals:
* Scale to billions of users. Quickly. Easily.
The basic principlesBoundary Conditions:
* Each user's data is completely independent, there's no need for queries that cross multiple user accounts.
** This means that our data storage problem is ''embarrassingly shardable''. Good times!
* The client-facing API is strongly consistent, and exposes an atomic check-and-set operation.
** This rules out a raft of eventually-consistent datastores that might otherwise work very nicely.
Basic Principles:
* Each user account is assigned to a particular '''shard''', identified by an integer.