Changes

Jump to: navigation, search

Balrog

118 bytes removed, 17:38, 31 March 2017
Deploying Changes
== Deploying Changes ==
Balrog's [https://mana.mozilla.org/wiki/display/SVCOPS/Firefox+Update+Service stage and production infrastructure] is managed by [https://mana.mozilla.org/wiki/display/SVCOPS/Contacting+Cloud+Operations the Cloud Operations team]. This section describes how to go from a reviewed patch to deploying it in production. To ensure there's adequate time for stage deployment and testing, you You should generally begin this process '''at least 24 hours1 hour''' before you want the new code live in production. This gives CloudOps plenty of time to respond to the request.
=== Is now a good time? ===
* Do you have enough time to safely do a push? Most pushes take at most 60 minutes to complete after the stage push has been done. This time is mostly affected by how long it takes you to verify your changes in stage and production.
=== Landing ======= Schema Upgrades ====
If you need to do a schema change you '''must''' ensure that either the current production code can run with your schema change applied, or that your new code can run with the old schema. Code and schema changes '''cannot''' be done at the same instant, so you must be able to support one of these scenarios. Generally, additive changes (column or table additions) should do the schema change first, while destructive changes (column or table deletions) should do the schema change second. You can simulate the upgrade with your local Docker containers to verify which is right for you.
=== Testing ===
Before asking for a push, you should do some functional testing on your local machine with The stage environment automatically deploys new code from the [[#Docker | Docker]] images. You should do this against master branch of [https://github.com/mozilla/balrog the master branch of the upstream Balrog repository] to ensure (including any necessary schema changes). Before asking for a push, you're should do some functional testing the exact code that is to be deployedthere. At the very least, you should do explicit testing of all the new code that would be included in the push. Eg: if you're changing the format of a blob, make sure that you can add a new blob of that type, and that the XML response looks correct.When you've completed testing take note of the "commit" on https://balrog-admin.stage.mozaws.net/__version__ - you'll need it when filing the production push bug. === Pushing to production ===Pushing live requires CloudOps. For non-urgent pushes, please make sure you file the bug for them X hours in advance. For urgent pushes, file the bug immediately and [https://mana.mozilla.org/wiki/display/SVCOPS/Contacting+Cloud+Operations escalate if no action is taken quickly]. Either way, you must follow this procedure to push:
=== Pushing to stage and production ===
Pushing live is a two step process. First, you must push to the stage environment and ensure things are working there. Then, you can push live.
# Bump the [https://github.com/mozilla/balrog/blob/master/version.txt in-repo version].
# [https://github.com/mozilla/balrog/releases Tag the repository] with a "vX.Y" tag. Eg: "git tag -s vX.Y"
# Wait for CI jobs to complete. Unit tests must pass and a new Docker Image [https://hub.docker.com/r/mozilla/balrog/tags/ for the webapps] and [https://hub.docker.com/r/mozilla/balrogagent/tags/ the Agent] must be pushed to Dockerhub before you proceed.
# [https://bugzilla.mozilla.org/enter_bug.cgi?assigned_to=bwongdthorn%40mozilla.com&bug_file_loc=http%3A%2F%2F&bug_ignored=0&bug_severity=normal&bug_status=NEW&cc=bwongdthorn%40mozilla.com&cc=ckolosmiles%40mozilla.com&cc=bhearsum%40mozilla40milescrabill.com&cf_blocking_b2g=---&cf_blocking_fennec=---&cf_fx_iteration=---&cf_fx_points=---&cf_status_b2g_2_6=---&cf_status_firefox48cf_status_firefox52=---&cf_status_firefox49cf_status_firefox53=---&cf_status_firefox50cf_status_firefox54=---&cf_status_firefox51cf_status_firefox55=---&cf_status_firefox_esr45=---&cf_tracking_b2gcf_status_firefox_esr52=---&cf_tracking_e10scf_tracking_firefox52=---&cf_tracking_firefox48cf_tracking_firefox53=---&cf_tracking_firefox49cf_tracking_firefox54=---&cf_tracking_firefox50cf_tracking_firefox55=---&cf_tracking_firefox51cf_tracking_firefox_esr45=---&cf_tracking_firefox_esr45cf_tracking_firefox_esr52=---&cf_tracking_firefox_relnote=---&comment=Balrog%20version%20X.Y%20is%20ready%20to%20be%20pushed%20to%20stage.%20Please%20deploy%20the%20new%20Docker%20image20images%20%28TAG_NAME28master-%24COMMIT%29%20the20for%20admin%20and2C%20public%20webapps%2C%20and%20the%20new%20Docker%20image%20for%20the%20Agent%20%28TAG_NAME%2920agent.%0D%0A%0D%0AWe%27d%20like%20the%20production%20push%20for%20this%20to%20happen%20sometime%20between%2011am%20and%201pm%20pacific%20on%20%28DATE%20GOES%20HERE%29&component=Operations%3A%20Deployment%20Requests&contenttypemethod=autodetect&contenttypeselection=text%2Fplain&defined_groups=1&flag_type-37=X&flag_type-4=X&flag_type-5=X&flag_type-607=X&flag_type-708=X&flag_type-720=X&flag_type-721=X&flag_type-737=X&flag_type-781=X&flag_type-787=X&flag_type-800=X&flag_type-803=X&flag_type-846=X&flag_type-864=X&flag_type-875905=X&flag_type-892914=X&flag_type-901=X&flag_type-905=X&flag_type-908916=X&form_name=enter_bug&maketemplate=Remember%20values%20as%20bookmarkable%20template&op_sys=Unspecified&priority=--&product=Cloud%20Services&rep_platform=Unspecified&short_desc=please%20deploy%20balrog%20X.Y%20to%20stage%2Fprod&target_milestone=---&version=unspecified File a bug] to have the new version pushed to stageproduction. Be sure to include substitute the new version number, and commit SHA for the Docker image tags for the webapps and the Agent you want deployed.
#* This bug should generally be filed a about 24h in advance of the desired production push time to give adequate time for the stage deployment and testing.
#* Wednesdays are usually the best day to push to production, because they are generally free of release events. Unless you have a specific need to deploy on a different day, you should request the prod push for a Wednesday.
#* You should link any bugs being deployed is the "Blocks" field.
# Once stage the push has been updatedhappened, verify your the changes again. Even though you've tested locally, it's important to retest in stage to make sure there's no deployment-specific issues.# When stage looks good, you're ready to comment in the bug to ask for production to be updated.# Reverify in production. When production has been updated, verify your changes again there. If you need to tweak rules or releases to do so, be careful not to touch any live channels (create new rules or releases if necessary). This final verification is as more about making sure the right thing got deployed than whether or not your code is correct.# Bump the [https://github.com/mozilla/balrog/blob/master/version.txt in-repo version] to the next available one to ensure the next push gets a new version.
= Meeting Notes =
Canmove, confirm
6,438
edits

Navigation menu