Snappy Symbolication Server: Difference between revisions

Snappy Symbolication Server (view source)

1,075 bytes added , 31 July 2015

47

edits

@@ Line 58: / Line 58: @@
 * Too bad we don't have unit tests.
+= Symbol file cache proposal =
+Snappy searches for symbol files on S3 when it doesn't find them locally. The proposal is to cache these files locally, with a LRU eviction policy. Instead of storing them in the original sym file, we store in the parsed format using [https://docs.python.org/2/library/pickle.html Pickle] module.
+The cache size is counted by the number of files stored, and must be accounted to keep some extra free space, since the eviction code runs in a timely fashion. Here is some pseudo Python code for file cache:
+<code>
+def Initialize():
+:for each file in the cache directory:
+::accessTime = AccessTime(filePath)
+::ListofFiles.append(FilePath)
+:LRU = ListOfFiles reverse sorted by access time
+</code>
+<code>
+def StoreCache(SymbolData, path):
+:writeFile(SymbolData, path)
+:LRU.append(path)
+</code>
+<code>
+def FetchCache(path):
+:SymbolData = readFile(path)
+:LRU.remove(path)
+:LRU.append(path)
+</code>
+<code>
+def Evict():
+:lruSize = len(LRU)
+:idealSize = maxSize * 0.6
+:while lruSize >= idealSize:
+::path = LRU.popleft()
+::removeFile(path)
+</code>
 == Regressions tests ==