Labs/Weave/ServerDataCharacteristics

special characteristics of our data:

• Collections_with_timestamps is 21% of our queries, and expected to rise
  • It can be memcached away
• If collections_with_timestamps is (mostly) optimized away, we have an approximate read-write ratio of 5-2
  • Almost all read queries are range-based, and, usually lower-bound only (modified since)
  • Diversity of served data makes caching almost impossible
• No query in the db is executed outside of a user's scope
  • Only collections_with_timestamps and collections_with_quotas are executed outside of user+collection scope
  • Users are very unlikely to have contention with themselves
  • No joins anywhere in the system.
• A lengthy primary key is unavoidable (due to the places id)
• Most data is entered as an upsert
  • Backend scripts purge old data in certain collections; External deletes seem to be of entire collections
• Officially, we only support a restricted list of collections; unofficially, there's support for arbitrary collection creation