Confirm
125
edits
Changes
no edit summary
* [https://github.com/dcramer/django-sentry Sentry]: '''(Phase 1)''' Sentry is an exception logging infrastructure that provides useful debugging tools to service app developers. Sentry is not yet planned on being provided by any Mozilla operations team, using it would require buy-in from and coordination with a Mozilla internal service provider (probably the Services Ops team).
* [http://esper.codehaus.org/ Esper]: '''(Phase 2)''' System for "complex event processing", i.e. one which will watch various statistic streams in real time looking for anomalous behavior.
* [http://opentsdb.net/ OpenTSDB] '''(Phase 2)''' A "Time Series Database" providing fine grained real time monitoring and graphing.
* [http://www.arcsight.com/products/products-esm/ ArcSight ESM] '''(Phase 2)''' A "security correlation engine" already in use throughout the Mozilla organization.
== Proposed API ==
Any service app will have the ability to easily generate arbitrary message data and metadata for delivery into the services metrics system. Any messages not specifically recognized as being intended for statsd or sentry will be delivered to a Hadoop cluster provided by the Metrics team, allowing for later analysis via custom map-reduce jobs or [https://hive.apache.org/ Hive] queries.
=== CEF security logging ===
Several groups in Mozilla are already using ArcSight ESM to track events and to evaluate them looking for patterns that may indicate attempts at security or abuse violations. ArcSight expects messages in the [http://www.arcsight.com/solutions/solutions-cef/ "Common Event Format"]. Rather than talking to ArcSight directly, services developers could send messages of type "cef" through metlog, decoupling service applications from a vendor-specific back end.
