Confirm
125
edits
Changes
no edit summary
* Rob Miller
* Victor Ng
== User Requirements ==
* Message processing system must be able to distinguish between different message types, so the various types can be routed to the appropriate back end(s) for effective analysis and reporting.
* Service app owners must have access to an interface (or interfaces) that will provide reporting and querying capabilities appropriate to the various types of messages that have been sent into the system.
== Proposed API ==
* metadata: Arbitrary set of key/value pairs that indicates the type of message that is being generated and includes any additional data that may be useful for back end reporting or analysis.
We will provide a "MetLogmetlog" library that will both ease generation of these messages and that will handle packaging them up and delivering them (via UDP) into the message processing infrastructure. Implementations of this library will likely be available in both Python and Javascript, but the Python library will be available first and this document will, for now, only describe the Python API. The Javascript API will be similar, modulo syntactic sugar that is available in Python but not in JS (e.g. decorators, context managers), and will be documented in detail in the future. The proposed Python API is as follows:
'''set_metlog_dest(host, port)'''
value to be updated to contain the flavor's metadata; defaults to an
empty list
'''timer(name, timestamp=None, logger=None, severity=6, metadata=None, flavors=None, rate=1)'''
Can be used as either a context manager or a decorator. Will calculate the
time required to execute the enclosed code, and will generate and send a
metlog message containing the timing information upon completion. ''name''
is a required string label for the timer that will be added to the message
metadata. ''rate'' represents what fraction of these invocations should
actually be timed, so a value of 0.3 would mean that the code would be
timed and the results sent off approximately 30% of the time it was
executed.
'''incr(name, timestamp=None, logger=None, severity=6, metadata=None, flavors=None)'''
Sends an "increment counter" message to metlog. ''name'' is a required
string label for the counter that will be added to the message metadata.
== Use Cases ==
=== Python App Framework performance metrics ===
The Python framework that underlies the Services Apps will be annotated w/ ''timer'' calls to automatically generate performance metrics for such key activities as authentication and execution of the actual view callable. The sample rate of these calls will be able to be specified in the app configuration, where a value of 0 can be entered to turn off the timers altogether. These will ultimately feed into a [https://github.com/etsy/statsd statsd] / [http://graphite.wikidot.com/ graphite] back end provided by Services Ops, where app owners will be able to see graphs of the captured data.
=== Python App Framework exception logging ===
In addition to timing information, the Python framework for services apps can automatically capture exceptions, sending a full traceback and some amount of local variable information as part of the message payload. This can ultimately be delivered to a [https://github.com/dcramer/django-sentry Sentry] installation for developer introspection and debugging.
=== Ad-Hoc service app metrics gathering ===
Any service app will have the ability to easily generate arbitrary message data and metadata for delivery into the services metrics system. Any messages not specifically recognized as being intended for statsd or sentry will be delivered to a Hadoop cluster provided by the Metrics team, allowing for later analysis via custom map-reduce jobs or Hive queries.
