Auto-tools/Projects/Structured Logging: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
Line 4: Line 4:


Goals:
Goals:
* Support a wider range of output formats
* Support alternative output formats for test results
* Eliminate complex and brittle regex based log parsing
* Eliminate complex and brittle regex based log parsing
* Consolidate output processing between harnesses
* Consolidate output processing and logging configuration between harnesses


Approach:
Approach:
* Establish a common data format for test results
* Establish a common data format for test results
* Populate data format in-harness with a standardized logging API
* Populate data format in-harness with a logging API designed to output this format
* Log data from test harnesses as data structures rather than formatting them into strings
* Log data from test harnesses as data structures


== Implementation ==
== Implementation ==

Revision as of 21:53, 24 August 2014

This page is a high level account of the structured logging project providing links to many resources about the project. This page is a work in progress.

Background and Motivation

Goals:

  • Support alternative output formats for test results
  • Eliminate complex and brittle regex based log parsing
  • Consolidate output processing and logging configuration between harnesses

Approach:

  • Establish a common data format for test results
  • Populate data format in-harness with a logging API designed to output this format
  • Log data from test harnesses as data structures

Implementation

Participant Systems

  • System under test (Javascript, Java, C++)
  • Python test harness
  • Mozharness/Buildbot
  • Tbpl/Treeherder
  • Blobber

Logging APIs

The primary logging api for our data format is mozlog.structured readthedocs

For data originating from JavaScript or Java systems, StructuredLog.jsm and StructuredLogger.java have been implemented to produce this data format. The python library supports logging JSON directly from these producers with log_raw.

Status

  • The tracking bug for this effort is bug916295.
  • An etherpad with high-level status updates per test harness is maintained for the time being.
Retrieved from "https://wiki.mozilla.org/index.php?title=Auto-tools/Projects/Structured_Logging&oldid=1008704"