Event Telemetry: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(new page)
 
(First pass on page contents)
Line 2: Line 2:


= Overview =
= Overview =
In 2015, we migrated [[Firefox Health Report]] data collection to the [[Telemetry]] system. At the same time, we made changes to Telemetry so that pings would be sent more frequently. We also updated the [[CloudServices/DataPipeline|Data Pipeline]] that ingests and processes the data.
There is a common need across teams (fx-team, mobile, test-pilot, heartbeat, …) to have a mechanism for recording, storing, sending & analysing application usage in an event-oriented format.
The Data Platform team wants to support this with a common API and mechanisms for dealing with the collected data, without owning the individual measurements.
The solution here is to provide common client code, a standard data format, so we can come up with common processes and tooling for data pipeline & analysis work.
Historically we already send a form of UITelemetry data, but the current format is too complicated to work with and to maintain.


=== Dates ===
=== Dates ===


* '''Fx41''' (2015-09-22): Started sending opt-out telemetry (base set) for 5% of the release population
* ...: Event data explorable in re:dash (from pre-release channels)
* '''Fx42''' (2015-11-03): Started sending opt-out telemetry (base set) for 100% of the release population
* '''Fx52''' (~2017-03-07): Event data collection implemented in Firefox Telemetry
* '''Fx43''' (2015-12-15): Stopped sending FHR v2 data


=== Goals for Unified Telemetry ===
=== Goals for Event Telemetry ===
* On the client, unify the telemetry and FHR measurement systems so that measurements do not have to be implemented more than once in different systems.
* Enable exploratory usage behavior analysis
* Reduce the latency from the time a measurement occurs until it can be analyzed on the server.
* Enable event data collection from Firefox and addons
* Increase the accuracy of measurements so that they can be better correlated with factors in the user environment such as the specific build, enabled addons, and other hardware or software factors.
* Use a common data pipeline for client telemetry and service log data.


=== Documentation ===
=== Documentation ===
* [https://gecko.readthedocs.org/en/latest/toolkit/components/telemetry/telemetry/index.html Client pings (tree documentation)]
* [https://docs.google.com/document/d/1hNuS9lUJMvMqgntZXbFA6xZBU9zBpQgo7x73-sXKRpI/ Event Telemetry draft]
* [https://docs.google.com/spreadsheets/d/1bqamxVskDF7kQ6xL7S2BqY8TpngL-w41v6keiX_qByg/edit?usp=sharing V2 - V4 mappings]
* [https://gecko.readthedocs.org/en/latest/toolkit/components/telemetry/telemetry/index.html In-tree docs (to be written)]


=== Analysis and Reporting ===
=== Analysis and Reporting ===
* Telemetry Dashboard (now using v4 unified telemetry data!): https://telemetry.mozilla.org/
* Raw data using a spark cluster: https://telemetry-dash.mozilla.org/
* Launch a spark cluster: https://telemetry-dash.mozilla.org/
* re:dash event data tables
* Stream processing, heka reporting: [https://mana.mozilla.org/wiki/display/CLOUDSERVICES/Exploring+with+the+Mozilla+Data+Pipeline+Demo Exploring with the Mozilla Data Pipeline Demo]


= Project =
= Project =

Revision as of 16:25, 27 September 2016

The Telemetry wiki page has more information about using Telemetry -- this page describes the 2015 project.

Overview

There is a common need across teams (fx-team, mobile, test-pilot, heartbeat, …) to have a mechanism for recording, storing, sending & analysing application usage in an event-oriented format. The Data Platform team wants to support this with a common API and mechanisms for dealing with the collected data, without owning the individual measurements. The solution here is to provide common client code, a standard data format, so we can come up with common processes and tooling for data pipeline & analysis work. Historically we already send a form of UITelemetry data, but the current format is too complicated to work with and to maintain.

Dates

  • ...: Event data explorable in re:dash (from pre-release channels)
  • Fx52 (~2017-03-07): Event data collection implemented in Firefox Telemetry

Goals for Event Telemetry

  • Enable exploratory usage behavior analysis
  • Enable event data collection from Firefox and addons

Documentation

Analysis and Reporting

Project

Deliverables

  • Monitoring and alerting about pipeline health
  • Basic tool support
    • Telemetry Dashboard works against new pipeline data
    • Telemetry-dash (or new equivalent) can launch spark, heka reporting jobs
  • Derived data sets
    • Executive dashboard rollup
    • 1% sample of clientIds for longitudinal analysis
  • v2-v4 Data Continuity
    • Executive dashboard continues to work
    • Search analysis continues to work

Client work

Pipeline work

Client Testing

Communication

Resources

  • Kickoff document
    • "Query Requirements" section has list of sample queries/questions that get asked frequently of FHR data

People and Roles

  • Georg Fritzsche (client data collection)
  • Alessio Placitelli, :Dexter (client data collection)
  • Mark Reid (data pipeline, telemetry server)
  • Michael Trinkala, :trink (data pipeline, heka)
  • Wesley Dawson, :whd (data pipeline operations)
  • Daniel Thornton, :relud (data pipeline operations)
  • Stuart Philp (test automation)
  • Anthony Zhang (Telemetry dashboard)
  • Roberto Vitillo (Spark analysis tool, telemetry data validation)
  • Brendan Colloran (metrics team, data validation)
  • Sam Penrose (metrics team, data validation)
  • Thomas Huelbert (project management)
  • Katie Parlante (eng manager)
  • Benjamin Smedberg (project sponsor, data steward)
Retrieved from "https://wiki.mozilla.org/index.php?title=Event_Telemetry&oldid=1149429"