Breakpad/Status Meetings/2015-10-14: Difference between revisions

Breakpad/Status Meetings/2015-10-14 (view source)

884 bytes added , 14 October 2015

Confirmed users

1,031

edits

@@ Line 18: / Line 18: @@
 == Operations Updates ==
+* stage has had some challenges and opportunities
+** deploy failed earlier this week, but exited 0
+*** fixed in pr https://github.com/mozilla/socorro-infra/pull/221
+* systemd was running stuff as the wrong user
+** race condition in our infra that we hadn't hit in the prior six months
+** crash mover happened to start sooner this time
+** monitoring failure. only discovered because we were looking at a change on stage in detail and saw nothing was running
+** alerts were firing, but dev team was not seeing them
+** looking for a way to connect them to irc
+* stage admin node had not been running crontabber, but it was all green
+** divergence between our consul config and the code
+** crontabber has in the past taken its config from code
+** consul was overriding with a different set of jobs in config
+** now have monitoring to check to ensure crontabber is running every so often
 == Project Updates ==