ReleaseEngineering/How To/Restart Redis: Difference between revisions

ReleaseEngineering/How To/Restart Redis (view source)

1,051 bytes added , 15 October 2013

no edit summary

Confirmed users

4,293

edits

@@ Line 1: / Line 1: @@
 {{Release Engineering How To|Restart Redis}}
+Redis is used to cache information about builds, as well as support the signing servers. The general way of restarting Redis is
+ service redis restart
+Note that we write the in-memory database to disk frequently so there is a low risk of data loss in doing this.
+= Redis hangs =
+{{Bug|905587}} tracks an issue with the version we're running where it stops functioning after opening too many file descriptors. This results in [https://secure.pub.build.mozilla.org/builddata/buildjson/builds-4hr.js.gz builds-4hr.js.gz] getting stale, and TBPL stops getting information about builds that are recently finished. This is a tree closing problem.
+== Fix ==
+* Connect:
+ ssh root@redis01.build.mozilla.org
+* Diagnosis - look for a result >= 1000
+ ls /proc/`pidof redis-server`/fd | wc -l
+* Resolution
+ service redis restart
+* Verification:
+ telnet localhost 6379
+ # say 'MONITOR'
+You should see a lot of lines fly past once builds-4hrs generation gets going again. If it doesn't <tt>root@buildapi01.build.mozilla.org</tt> and look at buildapi processes