Changes

Balrog's [https://mana.mozilla.org/wiki/display/SVCOPS/Firefox+Update+Service stage and production infrastructure] is managed by [https://mana.mozilla.org/wiki/display/SVCOPS/Contacting+Cloud+Operations the Cloud Operations team]. This section describes how to go from a reviewed patch to deploying it in production. You should generally begin this process '''at least 1 hour24 hours''' before you want the new code live in production. This gives CloudOps plenty of time the new code a chance to respond to the requestbake in stage.

The stage dev environment automatically deploys new code from the master branch of [https://github.com/mozilla/balrog the Balrog repository] (including any necessary schema changes). Before asking for a pushbeginning the deployment procedure, you should do some functional testing there. At the very least, you should do explicit testing of all the new code that would be included in the push. Eg: if you're changing the format of a blob, make sure that you can add a new blob of that type, and that the XML response looks correct. When you've completed testing take note of the "commit" on https://balrog-admin.stage.mozaws.net/__version__ - you'll need it when filing the production push bug.

'''If you have schema changes''' you must also ensure that the existing deployed code will work with the new schema. To do this, CloudOps will downgrade the stage dev apps. You should do some routine testing (make some changes to some objects, try some update requests) to ensure that everything works. If you have any issues you CANNOT proceed to production.

Pushing live requires CloudOps. For non-urgent pushes, please make sure you file the bug for them X should begin this procedure '''at least 24 hours in advance'''. For urgent pushes, file the bug immediately and [https://mana.mozilla.org/wiki/display/SVCOPS/Contacting+Cloud+Operations escalate if no action is taken quickly]. Either way, you must follow this procedure to push:

# [https://github.com/mozilla/balrog/releases Tag the repository] with a "vX.Y" tag. Eg: "git tag -s vX.Y&& git push --tags"

# Wait for CI jobs to complete. Unit tests must pass and a new Docker Image [https://hub.dockergithub.com/r/mozilla/balrog/tags/ for the webappsreleases Create a new Release on Github] . This create new Docker images tagged with your version, and [https://hubdeploy them to stage.dockerIt may take upwards of 30 minutes for the deployment to happen.com/r/mozilla# Let things bake on stage for at least 24 hours. You can do additional targeted testing here if you wish, or simply wait for nightlies/balrogagent/tags/ the Agent] must be pushed releases to Dockerhub before you proceedprod things along.# [https://bugzilla.mozilla.org/enter_bug.cgi?assigned_to=dthorn%40mozilla.com&bug_file_loc=http%3A%2F%2F&bug_ignored=0&bug_severity=normal&bug_status=NEW&cc=dthorn%40mozilla.com&cc=miles%40milescrabill.com&cf_blocking_fennec=---&cf_fx_iteration=---&cf_fx_points=---&cf_status_firefox56=---&cf_status_firefox57=---&cf_status_firefox58=---&cf_status_firefox_esr52=---&cf_tracking_firefox56=---&cf_tracking_firefox57=---&cf_tracking_firefox58=---&cf_tracking_firefox_esr52=---&cf_tracking_firefox_relnote=---&comment=Balrog%20version%20X.Y%20is%20ready%20to%20be%20pushed%20to%20prod.%20Please%20deploy%20the%20new%20Docker%20images%20%28master-%24COMMIT%2928vX.Y%20for%20admin%2C%20public%2C%20and%20the%20agent.%0D%0A%0D%0AWe%27d%20like%20the%20production%20push%20for%20this%20to%20happen%20sometime%20between%2011am%20and%201pm%20pacific%20on%20%28DATE%20GOES%20HERE%29.%0D%0A%0D%0AONE%20OF%3A%0D%0A%28NO%20SCHEMA%20CHANGE%20OR%20ADDITIVE%20SCHEMA%20CHANGE%29%20If%20anything%20goes%20wrong%20with%20the%20new%20version%20in%20production%2C%20we%20can%20safely%20rollback%20to%20the%20previous%20versions%20of%20admin%2C%20agent%2C%20public%2C%20and%20public%20in%20that%20order.%0D%0A%0D%0A%28DESTRUCTIVE%20SCHEMA%20CHANGE%29%20Because%20this%20push%20contains%20a%20destructive%20schema%20change%2C%20rollback%20to%20the%20previous%20version%20must%20be%20done%20as%20follows%3A%0D%0A%2A%20Revert%20the%20schema%20change%20%28run%20%22python%20%2Fapp%2Fscripts%2Fmanage-db.py%20-d%20%24DBURI%20--version%20X%22%20in%20the%20admin%20container%29%0D%0A%2A%20Rollback%20admin%2C%20agent%2C%20and%20public%20in%20that%20order.%0D%0A%0D%0A%28UNABLE%20TO%20COME%20UP%20WITH%20SAFE%20ROLLBACK%20PROCEDURE%20-%20THIS%20SHOULD%20NEVER%20HAPPEN%29%20This%20push%20contains%20changes%20that%20cannot%20be%20safely%20rolled%20back.%20If%20anything%20goes%20wrong%20with%20the%20new%20version%20in%20production%2C%20please%20escalate%20to%20me.&component=Operations%3A%20Deployment%20Requests&contenttypemethod=autodetect&contenttypeselection=text%2Fplain&defined_groups=1&flag_type-37=X&flag_type-4=X&flag_type-5=X&flag_type-607=X&flag_type-708=X&flag_type-721=X&flag_type-737=X&flag_type-787=X&flag_type-800=X&flag_type-803=X&flag_type-846=X&flag_type-864=X&flag_type-914=X&flag_type-916=X&form_name=enter_bug&maketemplate=Remember%20values%20as%20bookmarkable%20template&op_sys=Unspecified&priority=--&product=Cloud%20Services&rep_platform=Unspecified&short_desc=please%20deploy%20balrog%20X.Y%20to%20prod&target_milestone=---&version=unspecified File a bug] to have the new version pushed to production. Be sure to substitute the new version number, and commit SHA for the Docker image tags.

#* Make sure you substitute the version number and choose the correct rollback option from the bug template.