Snappy Symbolication Server: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Snappy regression tests)
(Rewritten page for updated information)
Line 1: Line 1:
{{FeatureStatus
The Snappy Symbolication Server is a Web server for symbolicating Firefox stacks. It matches PC addresses to modules in memory and looks up the corresponding function names in server-side symbol files (.SYM files).
|Feature name=Snappy Symbolication Server
|Feature stage=Feature Inbox
|Feature status=In progress
|Feature version=Nightly 14
|Feature health=OK
}}
{{FeatureTeam
|Feature feature manager=bsmedberg
|Feature lead engineer=vladan
|Feature additional members=benwa, ehsan
}}
{{FeaturePageBody
|Feature overview=In nightly profiling builds (and perhaps later in regular nightlies and aurora) we want to give users the ability to profile their own slowness via an about:snappy page. We also want to submit significant chrome hangs to telemetry.
|Feature users and use cases=In order to make this useful, the reports need to be able to map stack traces to symbols, similarly to how crash-stats does this. However, we're not taking full minidumps for privacy reasons, so we want to expose a webservice based on crash-stats symbol data that can convert address/offset information to a symbolicated name.


There will be two users of this data:
== Running ==


1) the telemetry server will use this to convert numeric stacks into symbols
The source code for Snappy lives in the [https://github.com/mozilla/Snappy-Symbolication-Server Mozilla Github repository]. Snappy runs at [http://www.python.org Python] 2.7 and depends on [http://www.tornadoweb.org Tornado] and
2) the client page about:snappy will use this to convert numeric stacks into symbols
[https://pypi.python.org/pypi/futures concurrent.futures] packages. The best way to install them is through [https://pip.pypa.io pip]:
|Feature functional spec=Stack data will be submitted to the server in JSON format and will be returned in JSON format. Details TBD as we iterate.


|Feature implementation plan=
:<code>pip install tornado futures</code>
# get a VM for experimentation/development with access to the symbol data and node COMPLETE
# develop the webservice behind the firewall INITIAL CODE COMPLETE, NEEDS REVISION FOR DIRECT SYMBOL ACCESS - The code is currently hosted at https://github.com/bgirard/ProfilerSymbolServer
# After development is complete, open up the webservice via a public URL


Note that the plan has changed and the server has now been rewritten in Python with extended functionality: https://github.com/vdjeric/Snappy-Symbolication-Server/
To run snappy on your machine, just type:


|Feature security review=
:<code>python symbolicationWebService.py <configuration-file></code>
Hopefully minimal security review will be required. The webservice can run with readonly mounts in an isolated environment and should not have any private or persistent data.
https://wiki.mozilla.org/Security/Reviews/SnappySymbolSrv


|Feature privacy review=
The snappy repository contains two sample configuration files for Linux and Windows. Here is a summary for the config fields:
No private information will be processed.
 
https://wiki.mozilla.org/Privacy/Reviews/SnappySymbolicServer
{| class="wikitable"
}}
! Section !! colspan=2 | Fields
{{FeatureInfo
|-
|Feature priority=Unprioritized
! !! Field !! Description
|Feature theme=Performance
|-
|Feature roadmap=Platform
| rowspan=6 | General || hostname || Server address or name
|Feature list=Platform
|-
|Feature project=Responsiveness
| portNumber || Server port number
}}
|-
{{FeatureTeamStatus
| remoteSymbolServer || The address of a secondary remote server to forward requests to
|Feature security status=sec-review-sched
|-
|Feature security notes=2012.03.02
| maxCacheEntries || Number of entries for RAM symbol cache
}}
|-
| mruSymbolStateFile || Json file with RAM cache info
|-
| maxMRUSymbolPersist || Maximum number of symbols for cache
|-
| rowspan=2 | DiskCache || cachePath || Path to cached symbol files
|-
| maxCacheFiles || Maximum number of files in the cache dir
|-
| rowspan=4 | Log || MaxFiles || Maximum number of log files
|-
| maxFileSize || Maximum size for a log file
|-
| logPath || Path to log files
|-
| logLevel || NOTSET, DEBUG, INFO, WARNING, ERROR, CRITICAL
|-
| SymbolPaths || || Each entry represents a path to search for symbols in the local disk
|-
| SymbolURLs ||  || Each entry represents a remote path to search for symbols
|}
 
== Contributing ==
 
It is a [https://github.com Github ] project, just fork it and send a PR. If you want to ask something, you can find people involved with Snappy Server in the #perf channel at irc.mozilla.org.
 
== Project ideas ==
 
TODO


== Regressions tests ==
== Regressions tests ==
Some regressions tests to perform before sending a PR:


* Perform a get request to the server
* Perform a get request to the server

Revision as of 20:37, 29 July 2015

The Snappy Symbolication Server is a Web server for symbolicating Firefox stacks. It matches PC addresses to modules in memory and looks up the corresponding function names in server-side symbol files (.SYM files).

Running

The source code for Snappy lives in the Mozilla Github repository. Snappy runs at Python 2.7 and depends on Tornado and concurrent.futures packages. The best way to install them is through pip:

pip install tornado futures

To run snappy on your machine, just type:

python symbolicationWebService.py <configuration-file>

The snappy repository contains two sample configuration files for Linux and Windows. Here is a summary for the config fields:

Section Fields
Field Description
General hostname Server address or name
portNumber Server port number
remoteSymbolServer The address of a secondary remote server to forward requests to
maxCacheEntries Number of entries for RAM symbol cache
mruSymbolStateFile Json file with RAM cache info
maxMRUSymbolPersist Maximum number of symbols for cache
DiskCache cachePath Path to cached symbol files
maxCacheFiles Maximum number of files in the cache dir
Log MaxFiles Maximum number of log files
maxFileSize Maximum size for a log file
logPath Path to log files
logLevel NOTSET, DEBUG, INFO, WARNING, ERROR, CRITICAL
SymbolPaths Each entry represents a path to search for symbols in the local disk
SymbolURLs Each entry represents a remote path to search for symbols

Contributing

It is a Github project, just fork it and send a PR. If you want to ask something, you can find people involved with Snappy Server in the #perf channel at irc.mozilla.org.

Project ideas

TODO

Regressions tests

Some regressions tests to perform before sending a PR:

  • Perform a get request to the server
  • Check if the IP in X-Forward-For is logged
  • Test requests with /gecko-profiler/ path
  • Test for /debug and /nodebug special paths
  • Test with compressed symbols files
  • Exiting with Control-C should kill the server smoothly
  • Check if the host handles ill-formed requests
  • Check if symbol server can forward requests
  • Parse a symbol file, exit and start the server again. The server should fetch the sym file from cache
  • Check if the server writes logs to files
Retrieved from "https://wiki.mozilla.org/index.php?title=Snappy_Symbolication_Server&oldid=1087192"