CloudServices/Loop/MySQL - Revision history

Ametaireau: *, not -

2014-10-24T12:54:29Z

*, not -

← Older revision		Revision as of 12:54, 24 October 2014
Line 23:		Line 23:
	The plan is to migrate automatically the data using the following process:		The plan is to migrate automatically the data using the following process:

	- Read from MySQL, if the keys aren't present, read them from redis and insert them into MySQL;		* Read from MySQL, if the keys aren't present, read them from redis and insert them into MySQL;
	- After some time, just drop the data in Redis because we don't need it anymore.		* After some time, just drop the data in Redis because we don't need it anymore.

Ametaireau: Update the overall information since we now need to migrate the data.

2014-10-24T12:53:51Z

Update the overall information since we now need to migrate the data.

← Older revision		Revision as of 12:53, 24 October 2014
Line 7:		Line 7:
	=== Tentative schedule ===		=== Tentative schedule ===

	We ~~need~~ to have this landed on our prod stack '''before''' we hit firefox 35 release if possible.		We would like to have this landed on our prod stack '''before''' we hit firefox 35 release if possible.
	~~Depending if we can drop the data or not, the schedule would change~~.

	~~In case~~ the data ~~can be dropped but not after a certain point in time~~, we ~~need~~ to ~~define what this~~ time is and ~~it would mean~~ that we ~~deploy something early on:~~		* '''30 of october''' (next week): Finish the coding;
			* '''31st of october''': Deploy it to the staging server and see if we're able to migrate data correctly (QA!);
			* Once this is done, we plan to have some feature-freeze time when we have time to investigate how MySQL is behaving and to any bugfix that's needed.

			=== Data migration ===

			After a quick discussion with hello product folks, we need to actually migrate the redis data to the MySQL database (from Redis).

	* '''28 of october''': Code ready, we ~~are ready for deployment;~~		Good news is that it means we have more time than we expected to actually do the migration, the only limiting factor we have is the size of the redis storage (currently set to 4GB).
	* '''29 of october''': MySQL deployed on the staging stack, QA can verify;
	* Once it's deployed to staging and verified, we ~~can deploy~~ to ~~prod.~~
	* Once this is done, we ~~plan to~~ have ~~some feature-freeze time when we have time to investigate how MySQL~~ is ~~behaving and~~ to ~~any bugfix that's needed~~.

	~~In case we need data migration, it means that we need~~ to ~~write~~ the ~~scripts~~ to ~~do this data migration~~, ~~decide on how to deal~~ with ~~that~~ and ~~make it happen before~~ the ~~firefox 35 release~~.		I don't think 4GB will be enough to handle the Firefox release user base, but I don't have yet any figures to back that up. We're currently, with 10% of the beta user base, using about 400MB of storage (this is for ~10K calls a day and represents ~9% of the space avail. in Redis).

	~~=== Data migration ===~~		The plan is to migrate automatically the data using the following process:

	~~This schedule is tied to~~ the ~~fact we would not need to write data migration scripts.~~		- Read from MySQL, if the keys aren't present, read them from redis and insert them into MySQL;
	~~We need to find out if that~~'~~s acceptable to drop the data we have for beta use, so we can start with a new empty database~~, and ~~what is the last~~ time ~~we can~~ drop the ~~beta~~ data.		- After some time, just drop the data in Redis because we don't need it anymore.

Ametaireau at 14:43, 23 October 2014

2014-10-23T14:43:19Z

← Older revision		Revision as of 14:43, 23 October 2014
Line 7:		Line 7:
	=== Tentative schedule ===		=== Tentative schedule ===

	We need to have this landed on our prod stack '''before''' we hit firefox 35 release. ~~The~~ schedule ~~for that~~ is ~~pretty though~~ and we ~~would need to have~~ something ~~like that~~:		We need to have this landed on our prod stack '''before''' we hit firefox 35 release if possible.
			Depending if we can drop the data or not, the schedule would change.

			In case the data can be dropped but not after a certain point in time, we need to define what this time is and it would mean that we deploy something early on:

	* '''28 of october''': Code ready, we are ready for deployment;		* '''28 of october''': Code ready, we are ready for deployment;
Line 13:		Line 16:
	* Once it's deployed to staging and verified, we can deploy to prod.		* Once it's deployed to staging and verified, we can deploy to prod.
	* Once this is done, we plan to have some feature-freeze time when we have time to investigate how MySQL is behaving and to any bugfix that's needed.		* Once this is done, we plan to have some feature-freeze time when we have time to investigate how MySQL is behaving and to any bugfix that's needed.

			In case we need data migration, it means that we need to write the scripts to do this data migration, decide on how to deal with that and make it happen before the firefox 35 release.

	=== Data migration ===		=== Data migration ===

Ametaireau at 14:28, 23 October 2014

2014-10-23T14:28:19Z

← Older revision		Revision as of 14:28, 23 October 2014
Line 12:		Line 12:
	* '''29 of october''': MySQL deployed on the staging stack, QA can verify;		* '''29 of october''': MySQL deployed on the staging stack, QA can verify;
	* Once it's deployed to staging and verified, we can deploy to prod.		* Once it's deployed to staging and verified, we can deploy to prod.
			* Once this is done, we plan to have some feature-freeze time when we have time to investigate how MySQL is behaving and to any bugfix that's needed.

	=== Data migration ===		=== Data migration ===

	This schedule is tied to the fact we would not need to write data migration scripts.		This schedule is tied to the fact we would not need to write data migration scripts.
	We need to find out if that's acceptable to drop the data we have for beta use, so we can start with a new empty database.		We need to find out if that's acceptable to drop the data we have for beta use, so we can start with a new empty database, and what is the last time we can drop the beta data.

Ametaireau at 14:26, 23 October 2014

2014-10-23T14:26:40Z

← Older revision		Revision as of 14:26, 23 October 2014
Line 9:		Line 9:
	We need to have this landed on our prod stack '''before''' we hit firefox 35 release. The schedule for that is pretty though and we would need to have something like that:		We need to have this landed on our prod stack '''before''' we hit firefox 35 release. The schedule for that is pretty though and we would need to have something like that:

	- '''28 of october''': Code ready, we are ready for deployment;		* '''28 of october''': Code ready, we are ready for deployment;
	- '''29 of october''': MySQL deployed on the staging stack, QA can verify;		* '''29 of october''': MySQL deployed on the staging stack, QA can verify;
	- Once it's deployed to staging and verified, we can deploy to prod.		* Once it's deployed to staging and verified, we can deploy to prod.

	=== Data migration ===		=== Data migration ===

Ametaireau: Created page with "== Loop migration to MySQL == The loop-server will be transistioning from using Redis for all its storage to using MySQL for the persistant data and Redis for the volatile da..."

2014-10-23T14:26:15Z

Created page with "== Loop migration to MySQL == The loop-server will be transistioning from using Redis for all its storage to using MySQL for the persistant data and Redis for the volatile da..."

New page

== Loop migration to MySQL ==

The loop-server will be transistioning from using Redis for all its storage to using MySQL for the persistant data and Redis for the volatile data.

This is because it is far easier from an OPs point of view to operate a MySQL database for non-volatile data.

=== Tentative schedule ===

We need to have this landed on our prod stack '''before''' we hit firefox 35 release. The schedule for that is pretty though and we would need to have something like that:

- '''28 of october''': Code ready, we are ready for deployment;
- '''29 of october''': MySQL deployed on the staging stack, QA can verify;
- Once it's deployed to staging and verified, we can deploy to prod.

=== Data migration ===

This schedule is tied to the fact we would not need to write data migration scripts.
We need to find out if that's acceptable to drop the data we have for beta use, so we can start with a new empty database.