CloudServices/Loop/MySQL

Loop migration to MySQL

The loop-server will be transistioning from using Redis for all its storage to using MySQL for the persistant data and Redis for the volatile data.

This is because it is far easier from an OPs point of view to operate a MySQL database for non-volatile data.

Tentative schedule

We would like to have this landed on our prod stack before we hit firefox 35 release if possible.

• 30 of october (next week): Finish the coding;
• 31st of october: Deploy it to the staging server and see if we're able to migrate data correctly (QA!);
• Once this is done, we plan to have some feature-freeze time when we have time to investigate how MySQL is behaving and to any bugfix that's needed.

Data migration

After a quick discussion with hello product folks, we *need* to actually migrate the redis data to the MySQL database (from Redis).

Good news is that it means we have more time than we expected to actually do the migration, the only limiting factor we have is the size of the redis storage (currently set to 4GB).

I don't think 4GB will be enough to handle the Firefox release user base, but I don't have yet any figures to back that up. We're currently, with 10% of the beta user base, using about 400MB of storage (this is for ~10K calls a day and represents ~9% of the space avail. in Redis).

The plan is to migrate automatically the data using the following process:

• Read from MySQL, if the keys aren't present, read them from redis and insert them into MySQL;
• After some time, just drop the data in Redis because we don't need it anymore.