Changes

The first version of the Metrics Metlog system will focus on providing an easy mechanism for the [[Services/Sync| Sync]] and [https://browserid.org/ BrowserID] projects (and any other internal Mozilla services) to efficiently send profiling data and any other arbitrary metrics information that may be desired into one or more backend back end storage locations. Once the data has made it to its final destination, there should be available to those w/ appropriate access the ability to do analytics queries and report generation on the accumulated data.

The proposed Services Metrics Metlog architecture will consist of 3 layers:

; router : The router is what will be listening for the messages sent out by the provided libraries. It will deserialize these messages and examine the metadata to determine the appropriate back end(s) to which the message should be delivered. The format and protocol for delivering these messages to the endpoints will vary from back end to back end. We plan on initially using [http://logstash.net/ logstash] as the message router, because it is already planned to be installed on every services server machine, and it is built specifically for this type of event-based messager message routing.

* [https://github.com/etsyfetep/ruby-statsd ruby-statsd]: '''(Phase 1)''' ruby-statsd is already in the pipeline to be running on every Services machine.* [https://githubhadoop.comapache.org/mozilla-metricshdfs/bagheera BagheeraHDFS]: '''(Phase 1)''' Bagheera is a REST service provided by the Mozilla Metrics team that Some data will insert data be inserted into the Mozilla Metrics team's Hadoop HDFS infrastructure, where it will be available for later processing[https://hive.apache.org/ Hive] and/or map reduce based queries.

The atomic unit for the Services Metrics Metlog system is the "message". The structure of a message is inspired by that of the well known syslog message standard, with some slight extensions to allow for more rich metadata. Each message will consist of the following fields:

* ''env_version'': API version number of the "message envelope", i.e. any changes to the message data structure (exclusive of message-type-specific changes that may be embedded within the fields or the payload) must increment the env_version value. The structure described in this document is envelope version 0.8.

We will provide "metlog" Metlog client libraries that will both ease generation of these messages and that will handle packaging them up and delivering them into the message processing infrastructure. Implementations of this library are be available in both Python and Node[https://github.jscom/mozilla-services/metlog-style Javascript. The py Python API is as follows: ; '''MetlogClient(sender, logger="", severity=6)''' : Primary metlog client class which can accept metlog messages, format ] and serialize them, and pass them along for delivery to the message processor. * ''sender''[https: A "sender" object, i//github.e. any object that exposes a ''send_message'' method accepting a single ''msg'' argument. All actual message transport and delivery is the responsibility of the sender. We will provide a ZeroMQ Publisher sender implementation.* ''logger'': Default for all subsequent ''com/mozilla-services/metlog'' calls which do not explicitly pass this value-node Node.* ''severity'': Default for all subsequent ''metlog'' calls which do not explicitly pass this value. ; '''MetlogClientjs style Javascript].metlog(type, timestamp=None, logger=None, severity=None, message="", fields=None)''' : Sends a single log message along to Please see the metlog listener(s). Most of the arguments correspond to the message fields described above. Only ''type'' is strictly required, the rest will be populated by reasonable defaults if they aren't provided: * ''timestamp'': Defaults to current system time* ''logger'': Defaults to the current value of MetlogClient.logger* ''severity'': Defaults to the current value of MetlogClient.severity* ''message'': Defaults to an empty string* ''fields'': Defaults to an empty dictionary ; '''MetlogClient.timer(name, timestamp=None, logger=None, severity=None, fields=None, rate=1)''' : Can be used as either a context manager or a decorator. Will calculate the time required to execute the enclosed code, and will generate and send a metlog message (of type "timer") containing the timing information upon completion. * ''name'': A required string label documentation for the timer that will be added to the message fields* ''timestamp'': Defaults to current system time* ''logger'': Defaults to the current value of MetlogClient.logger* ''severity'': Defaults to the current value of MetlogClient.severity* ''fields'': Defaults to an empty dictionary* ''rate'' Represents what fraction of these invocations should actually be timed; a value of 0.3 would mean that the code would be timed and the results sent off approximately 30% of the time it was executed ; '''MetlogClient.incr(name, timestamp=None, logger=None, severity=None, fields=None)''' : Sends an "increment counter" message client libraries to metlog. ''name'' is a required string label for learn more about the counter that will be added to the message metadata. * ''name'': A required string label for the counter that will be added to the message fields* ''timestamp'': Defaults to current system time* ''logger'': Defaults to the current value of MetlogClient.logger* ''severity'': Defaults to the current value of MetlogClientspecific APIs available in each environment.severity* ''fields'': Defaults to an empty dictionary

The Python framework that underlies the Services Apps will be annotated w/ ''timer'' calls to automatically generate performance metrics for such key activities as authentication and execution of the actual view callable. The sample rate of these calls will be able to be specified in the app configuration, where a value of 0 can be entered to turn off the timers altogether. These will ultimately feed into a [https://github.com/etsyfetep/ruby-statsd ruby-statsd] / [http://graphite.wikidot.com/ graphite] back end provided by Services Ops, where app owners will be able to see graphs of the captured data.

Any service app will have the ability to easily generate arbitrary message data and metadata for delivery into the services metrics Metlog system. Any messages not specifically recognized as being intended for statsd or sentry another back end will be delivered to a Hadoop an HDFS cluster provided by the Metrics team, allowing for later analysis via custom map-reduce jobs or [https://hive.apache.org/ Hive] queries.