Identity/AttachedServices/DeploymentPlanning/TrafficModel

Overview

Herein we will construct a high-level aggregate model of how users will interact with the PICL backend services. It's an attempt to inform capacity and loadtest planning based on some real data from existing services, some educated guesswork, and some hand-waving.

Assumptions:

• ten times as many users on the new system as there are on current sync.
• roughly similar per-user behaviour as current sync (e.g. data volume, change rates).
• roughly similar per-user keysigning/authentication overheads as current persona.
• err on the side of caution; overestimate in the face of ambiguity.

Raw data and some preliminary analysis can be found in https://id.etherpad.mozilla.org/picl-user-model

Users and Devices

Due to the enormously more effective Setup/Signin UI, we should be ready to support the following very soon after launch time:

• 20 million total user accounts
• 15 million active users hitting the service in a 24-hour period

This is only a small fraction of the firefox userbase, but it's a lot more than current sync. TODO: Not Ambitious Enough?

The number of devices per user is a power-law-distribution in current sync, and we anticipate the same for PICL. Erring on the side of caution:

• 90% of users have 1 device
• 8% of users have 2 devices
• 2% of users have 3 devices
• negligible number of users with > 4 devices
  • 1.12 devices-per-user

The split between desktop and mobile device will mirror that seen in current sync:

• 85% of devices are desktop
• 15% of devices are mobile

Account Operations

These are the major operations that can be performed on the Firefox Accounts server. TODO: use data from persona.org to inform these numbers

Create Account

During the changeover, we expect an influx of account setup activity as users migrate over. Let's say two weeks to get to 20 million user accounts:

• 17 create-account operations per second

This should drop somewhat after initial release, so let's plan on that as peak capacity for now.

Register Device

There are 1.12 devices-per-user, which will all eventually need to be registered after the users account has been created. To allow for resets, upgrades, etc, that's also posit that each device needs to re-register itself once every two weeks:

• 19 register-device operations per second due to initial signup
• (20mil * 1.12) register-device operations every two weeks due to re-registration
  • 37 register-device operations per second

Sign Certificate

Certificates last for 1 hour. Lets say each device is online and syncing for 12 hours each day, making 12 sign-certificate requests per device per day.

• (15mil * 1.12 * 12) sign-certificate requests per day
  • 2333 sign-certificate operations per second

Change Password

Rare. Do we need to factor this in at this stage?

Detach Device

Rare. Do we need to factor this in at this stage?

Delete Account

Rare. Do we need to factor this in at this stage?

Scrypt Helper

The scrypt helper is used by mobile devices during authentication aka "device registration". At 37 register-device operations per second and 15% mobile devices, we have:

• 5.5 scrypt-helper requests per second

Sync Storage Operations

Here we're basically taking the current traffic to sync and multiplying it by 10.

?XXX TODO? item size breakdown for each datatype; we have this data from sync, just need to run the numbers...

History Data

Based on FHR data, the distribution of history item counts is roughly (after lots of rounding-up):

• 25% have less than 200 history entries
• 50% have less than 1,500 history entries
• 75% have less than 6,000 history entries
• 95% have less than 30,000 history entries
• absolute maximum is around 100,000 history entries

Extrapolating up from sync read/write rates, we expect:

• 175 history-data reads per second
• 1200 history-data writes per second

Bookmarks Data

Based on FHR data, the distribution of bookmark item counts is roughly (after lots of rounding-up):

• 25% have less than 30 bookmarks
• 50% have less than 40 bookmarks
• 75% have less than 70 bookmarks
• 95% have less than 400 bookmarks
• absolute maximum is around 10,000 bookmarks

The data from server-side sync databases skews higher, with more users in the 100-1000 bookmark bracket. We'll assume FHR is more representative of the general firefox population.

Extrapolating up from sync read/write rates, we expect:

• 130 bookmark-data reads per second
• 250 bookmark-data writes per second

Passwords Data

Based on server-side sync data:

• 75% have less than 100 passwords
• almost everyone has less than 1000 passwords

Extrapolating up from sync read/write rates, we expect:

• 100 password-data reads per second
• 240 password-data writes per second

Tabs Data

?XXX TODO?: get item-count data for this; it's very expensive to get out of sync because tabs are stored as opaque blobs in memcached.

Extrapolating up from sync read/write rates, we expect:

• 420 tabs-data reads per second
• 1010 tabs-data writes per second

Aggregate

Based on current sync traffic, we would aim for the following:

• 3500 reads per second
• 3900 writes per second

However, the reads-per-second figure here is skewed by some very read-heavy collections that will not appear in PICL. Instead, let's just add up the figures for individual data-types above and then round it up a little:

• 900 reads per second
• 3000 writes per second