ReleaseEngineering/How To/Restart Buildbot Masters: Difference between revisions

ReleaseEngineering/How To/Restart Buildbot Masters (view source)

Revision as of 22:02, 20 November 2016

58 bytes added , 20 November 2016

→‎By script: Streamline docs for running on buildduty-tools

NThomas

Confirmed users

4,293

edits

@@ Line 19: / Line 19: @@
 = By script =
-The above actions have been encapsulated into a script: https://hg.mozilla.org/build/tools/file/default/buildfarm/maintenance/restart_masters.py
+The above actions have been encapsulated into a script: https://hg.mozilla.org/build/tools/file/default/buildfarm/maintenance/restart_masters.py. The script is setup to run on <tt>buildduty-tools.srv.releng.usw2.mozilla.com</tt>, located at <tt>/home/buildduty/restart_masters/</tt> under buildduty's account.
-The script requires a [[ReleaseEngineering/Buildduty/Reconfigs#How_to_reconfig|bash-format config file like the one used by the end_to_end_reconfig.sh script]]. At the very least the config file must define values for LDAP_USERNAME, LDAP_PASSWORD, and CLTBLD_PASSWORD.
+The wrapper script <tt>/home/buildduty/restart_masters.sh</tt> is used to update repositories and call restart_masters.py. The latter will prompt for the usernames and passwords it needs (ldap for slavealloc, cltbld and root for ssh).
-The script is setup to run on buildduty-tools.srv.releng.usw2.mozilla.com in a venv under buildduty's account ,the venv is set on this path: /home/buildduty/restart_masters and the script  restart_masters.sh is located in /home/buildduty({{bug|1299421}}).
 Here is an example invocation:
-  # buildduty-tools
+  # buildduty@buildduty-tools
   $ screen -R restart_masters
-  $ cd ~buildduty/restart_masters
+  $ /home/buildduty/restart_masters.sh
- $ source bin/activate
- $ cd tools/buildfarm/maintenance/
+The logs go to [https://papertrailapp.com/systems/255918104/events?q=restart_masters papertrail], outputting a progress report every 60 minutes. You can send a SIGUSR1 to restart_masters.py to prompt an extra progress report; allow time for the current status check to complete first.
- $ ./restart_masters.py -v -m production-masters.json 2>&1
 = Automated =
 The above script requires sensitive credentials that shouldn't be stored on disk. For now, we're still running this script by hand.