Identity/AttachedServices/DeploymentPlanning/TrafficModel - Revision history

Rfkelly: /* Firefox Account Operations */

2013-08-15T22:54:02Z

Firefox Account Operations

← Older revision		Revision as of 22:54, 15 August 2013
Line 41:		Line 41:
	These are the major operations that can be performed on the Firefox Accounts server.		These are the major operations that can be performed on the Firefox Accounts server.

	'''TODO: any other potentially-troublesome operations that we should scope out here?~~'''~~		'''TODO:''' estimated data sizes, which translates into data volume rate in/out
			'''TODO:''' any other potentially-troublesome operations that we should scope out here?


Line 74:		Line 75:

	'''TODO:''' That's a lot. Adjust certificate duration to bring this number down?		'''TODO:''' That's a lot. Adjust certificate duration to bring this number down?


	== Scrypt Helper ==		== Scrypt Helper ==

Rfkelly: /* Users and Devices */

2013-08-15T05:18:28Z

Users and Devices

← Older revision		Revision as of 05:18, 15 August 2013
Line 35:		Line 35:
	* '''15% of devices are mobile'''		* '''15% of devices are mobile'''

	~~'''TODO''' compare~~ this ~~with~~ the overall ~~breakdown of the~~ firefox userbase~~, adjust as appropriate~~.		Note that this is a much higher percentage of mobile devices than seen in the overall firefox userbase. It seems safe to skew towards more mobile devices; it will affect the requirements for the Scrypt Helper but otherwise does not change the numbers.


	== Firefox Account Operations ==		== Firefox Account Operations ==

Rfkelly at 05:02, 15 August 2013

2013-08-15T05:02:52Z

← Older revision		Revision as of 05:02, 15 August 2013
Line 41:		Line 41:

	These are the major operations that can be performed on the Firefox Accounts server.		These are the major operations that can be performed on the Firefox Accounts server.

	~~'''TODO: use data from persona.org to inform these numbers as well'''~~

	'''TODO: any other potentially-troublesome operations that we should scope out here?'''		'''TODO: any other potentially-troublesome operations that we should scope out here?'''
Line 74:		Line 72:
	** '''2500 sign-certificate operations per second'''		** '''2500 sign-certificate operations per second'''

	Sanity-check: according to KPI data, this is higher than the rate of ~~login attempts~~ on persona.org.		Sanity-check: according to KPI data, this is higher than the rate of signins on persona.org.

	'''TODO:''' That's a lot. Adjust certificate duration to bring this number down?		'''TODO:''' That's a lot. Adjust certificate duration to bring this number down?

Rfkelly at 04:59, 15 August 2013

2013-08-15T04:59:21Z

← Older revision		Revision as of 04:59, 15 August 2013
Line 55:		Line 55:
	Note that each create-account operation on the Firefox Accounts server involves sending a confirmation email.		Note that each create-account operation on the Firefox Accounts server involves sending a confirmation email.

	~~As another point of reference~~, this is higher than the rate of account creation on persona.org.		Sanity-check: according to KPI data, this is higher than the rate of account creation on persona.org.


	=== Establish Session ===		=== Establish Session ===
Line 72:		Line 73:
	* (15mil * 1.2 * 12) sign-certificate requests per day		* (15mil * 1.2 * 12) sign-certificate requests per day
	** '''2500 sign-certificate operations per second'''		** '''2500 sign-certificate operations per second'''

			Sanity-check: according to KPI data, this is higher than the rate of login attempts on persona.org.

	'''TODO:''' That's a lot. Adjust certificate duration to bring this number down?		'''TODO:''' That's a lot. Adjust certificate duration to bring this number down?

Rfkelly: /* Create Account */

2013-08-15T04:55:34Z

Create Account

← Older revision		Revision as of 04:55, 15 August 2013
Line 55:		Line 55:
	Note that each create-account operation on the Firefox Accounts server involves sending a confirmation email.		Note that each create-account operation on the Firefox Accounts server involves sending a confirmation email.

			As another point of reference, this is higher than the rate of account creation on persona.org.

	=== Establish Session ===		=== Establish Session ===

Rfkelly at 04:43, 15 August 2013

2013-08-15T04:43:04Z

← Older revision		Revision as of 04:43, 15 August 2013
Line 30:		Line 30:
	** Let's round it up to: '''1.2 devices-per-user'''		** Let's round it up to: '''1.2 devices-per-user'''

	The split between desktop and mobile ~~device~~ will mirror that seen in current sync:		The split between desktop and mobile devices will mirror that seen in current sync:

	* '''85% of devices are desktop'''		* '''85% of devices are desktop'''

Rfkelly at 04:39, 15 August 2013

2013-08-15T04:39:34Z

← Older revision		Revision as of 04:39, 15 August 2013
Line 104:		Line 104:
	* 75% have less than 6,000 history entries		* 75% have less than 6,000 history entries
	* 95% have less than 30,000 history entries		* 95% have less than 30,000 history entries
	* ~~absolute maximum~~ is around 100,000 history entries		* expected peak is around 100,000 history entries

	Extrapolating up from sync read/write rates, we expect:		Extrapolating up from sync read/write rates, we expect:
Line 120:		Line 120:
	* 75% have less than 70 bookmarks		* 75% have less than 70 bookmarks
	* 95% have less than 400 bookmarks		* 95% have less than 400 bookmarks
	* ~~absolute maximum~~ is around 10,000 bookmarks		* expected peak is around 10,000 bookmarks

	The data from server-side sync databases skews higher, with more users in the 100-1000 bookmark bracket. We'll assume FHR is more representative of the general firefox population.		The data from server-side sync databases skews higher, with more users in the 100-1000 bookmark bracket. We'll assume FHR is more representative of the general firefox population.
Line 135:		Line 135:

	* 75% have less than 100 passwords		* 75% have less than 100 passwords
	* ~~almost everyone has less than 1000~~ passwords		* expected peak is around 1,000 passwords

	Extrapolating up from sync read/write rates, we expect:		Extrapolating up from sync read/write rates, we expect:
Line 144:		Line 144:

	=== Tabs Data ===		=== Tabs Data ===

	~~'''?XXX TODO?:''' get item-count data for this; it's very expensive to get out of sync because tabs are stored as opaque blobs in memcached.~~

	Extrapolating up from sync read/write rates, we expect:		Extrapolating up from sync read/write rates, we expect:

Rfkelly at 04:36, 15 August 2013

2013-08-15T04:36:12Z

← Older revision		Revision as of 04:36, 15 August 2013
Line 15:		Line 15:
	== Users and Devices ==		== Users and Devices ==

	Due to the enormously more effective Setup/Signin UI, we ~~should~~ be ~~ready to support~~ the following ~~very soon after launch time~~:		Due to the enormously more effective Setup/Signin UI, we expect to be supporting the following userbase:

	* '''20 million total user accounts'''		* '''20 million total user accounts'''
	* '''15 million active users hitting the service in a 24-hour period'''		* '''15 million active users hitting the service in a 24-hour period'''

	This is only a small fraction of the firefox userbase, but it's a lot more than current sync. ~~'''TODO: Not Ambitious Enough?'''~~		This is only a small fraction of the firefox userbase, but it's a lot more than current sync.

	The number of devices per user is a power-law-distribution in current sync, and we anticipate the same for PICL. Erring on the side of caution:		The number of devices per user is a power-law-distribution in current sync, and we anticipate the same for PICL. Erring on the side of caution:
Line 28:		Line 28:
	* 2% of users have 3 devices		* 2% of users have 3 devices
	* negligible number of users with > 4 devices		* negligible number of users with > 4 devices
	** '''1.12 devices-per-user'''		** Let's round it up to: '''1.2 devices-per-user'''

	The split between desktop and mobile device will mirror that seen in current sync:		The split between desktop and mobile device will mirror that seen in current sync:
Line 35:		Line 35:
	* '''15% of devices are mobile'''		* '''15% of devices are mobile'''

			'''TODO''' compare this with the overall breakdown of the firefox userbase, adjust as appropriate.

	~~== Account Operations ==~~

	~~These are the major operations that can be performed on the~~ Firefox ~~Accounts server. '''TODO: use data from persona.org to inform these numbers'''~~		== Firefox Account Operations ==

			These are the major operations that can be performed on the Firefox Accounts server.

	~~=== Create Account ===~~		'''TODO: use data from persona.org to inform these numbers as well'''

	~~During the changeover,~~ we ~~expect an influx of account setup activity as users migrate over. Let~~'~~s say two weeks to get to 20 million user accounts:~~		'''TODO: any other potentially-troublesome operations that we should scope out here?'''

	* '''17 create-account operations per second'''

	~~This should drop somewhat after initial release, so let's plan on that as peak capacity for now.~~		=== Create Account ===

			During the changeover, we expect an influx of account setup activity as users migrate over. Let's say two weeks for the existing sync userbase to migrate across, with that representing the peak rate of account creation:

	~~=== Register Device ===~~		* Rounds up to: '''2 create-account operations per second'''

	~~There are 1.12 devices~~-~~per-user, which will all eventually need to be registered after~~ the ~~users account has been created~~. ~~To allow for resets, upgrades, etc, that's also posit that each device needs to re-register itself once every two weeks:~~		Note that each create-account operation on the Firefox Accounts server involves sending a confirmation email.

	* 19 register-device operations per second due to initial signup
	* (20mil * 1.12) register-device operations every two weeks due to re-registration
	** '''37 register-device operations per second'''

	=== ~~Sign Certificate~~ ===		=== Establish Session ===

	~~Certificates last for~~ 1 ~~hour~~. ~~Lets say each~~ device is ~~online~~ and ~~syncing for 12 hours~~ each ~~day, making 12 sign-certificate requests per~~ device ~~per day.~~		Each of the 1.2 devices-per-user needs to establish a session token with the Firefox Accounts server. They must do this at initial device setup, whenever the user changes their password, and whenever the device is updated. Let's posit that password changes are negligible and so each device establishes a fresh session once every six weeks:

	* (~~15mil~~ * 1.12 * 12) ~~sign~~-~~certificate requests per day~~		* (2 * 1.2) establish-session operations per second due to initial setup
	** '''~~2333 sign~~-~~certificate~~ operations per second'''		* (20mil * 1.2) establish-session operations every six weeks due to device update etc
			** Rounds up to: '''10 register-device operations per second'''


	=== ~~Change Password~~ ===		=== Sign Certificate ===

	~~Rare~~. ~~Do we need to factor this in at this stage?~~		Each device needs a valid BrowserID identity certificate in order to talk to the storage servers. Certificates last for 1 hour. Lets say each device is online and syncing for 12 hours each day, making 12 sign-certificate requests per device per day.

			* (15mil * 1.2 * 12) sign-certificate requests per day
			** '''2500 sign-certificate operations per second'''

	~~=== Detach Device ===~~		'''TODO:''' That's a lot. Adjust certificate duration to bring this number down?

	~~Rare. Do we need to factor this in at this stage?~~

			== Scrypt Helper ==

	~~=== Delete Account ===~~		The scrypt helper is used by mobile devices during authentication, required as part of the establish-session operation. At 10 establish-session operations per second and 15% mobile devices, we have:

	~~Rare~~. ~~Do we need to factor this in at this stage?~~		* '''1.5 scrypt-helper requests per second'''


	== ~~Scrypt Helper~~ ==		== Sync Storage Operations ==

			Here we're basically taking the current traffic to sync and multiplying it by 10.

	~~The scrypt helper is used by mobile devices during authentication aka "device registration". At 37 register-device operations per second and 15% mobile devices,~~ we have:		'''?XXX TODO?''' item size breakdown for each datatype; we have this data from sync, just need to run the numbers...

	* '''5.5 scrypt-helper requests per second'''

			=== Create Account ===

	~~== Sync Storage Operations ==~~		Since the storage servers are a separate system, they will also require some initialization for each new user.

	~~Here we~~'~~re basically taking the current traffic to sync and multiplying it by 10.~~		* '''2 create-account operations per second'''

	~~'''?XXX TODO?''' item size breakdown for each datatype; we have this data from sync, just need to run the numbers...~~

	=== History Data ===		=== History Data ===

Rfkelly: /* Register Device */

2013-08-13T22:47:36Z

← Older revision		Revision as of 22:47, 13 August 2013
Line 52:		Line 52:
	=== Register Device ===		=== Register Device ===

	There are 1.12 devices-per-user, which will all ~~have~~ to be registered ~~when~~ the users account ~~is registered~~. To allow for resets, upgrades, etc, that's also posit that each device needs to re-register itself once every two weeks:		There are 1.12 devices-per-user, which will all eventually need to be registered after the users account has been created. To allow for resets, upgrades, etc, that's also posit that each device needs to re-register itself once every two weeks:

	* 19 register-device operations per second due to initial signup		* 19 register-device operations per second due to initial signup
	* (20mil * 1.12) register-device operations every two weeks due to re-registration		* (20mil * 1.12) register-device operations every two weeks due to re-registration
	** '''37 register-device operations per second'''		** '''37 register-device operations per second'''


	=== Sign Certificate ===		=== Sign Certificate ===

Rfkelly at 05:20, 13 August 2013

2013-08-13T05:20:32Z

← Older revision		Revision as of 05:20, 13 August 2013
Line 80:		Line 80:

	Rare. Do we need to factor this in at this stage?		Rare. Do we need to factor this in at this stage?


	== Scrypt Helper ==		== Scrypt Helper ==
Line 92:		Line 93:
	Here we're basically taking the current traffic to sync and multiplying it by 10.		Here we're basically taking the current traffic to sync and multiplying it by 10.

			'''?XXX TODO?''' item size breakdown for each datatype; we have this data from sync, just need to run the numbers...

	=== History Data ===		=== History Data ===
Line 142:		Line 144:
	=== Tabs Data ===		=== Tabs Data ===

	?XXX TODO? get item-count data for this; it's very expensive to get out of sync because tabs are stored as opaque blobs in memcached.		'''?XXX TODO?:''' get item-count data for this; it's very expensive to get out of sync because tabs are stored as opaque blobs in memcached.

	Extrapolating up from sync read/write rates, we expect:		Extrapolating up from sync read/write rates, we expect: