Event Telemetry: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(MOAR infos)
(Update in-tree docs link)
 
(28 intermediate revisions by 3 users not shown)
Line 6: Line 6:
The solution here is to provide common client code, a standard data format, so we can come up with common processes and tooling for data pipeline & analysis work.
The solution here is to provide common client code, a standard data format, so we can come up with common processes and tooling for data pipeline & analysis work.
Historically we already send a form of UITelemetry data, but the current format is too complicated to work with and to maintain.
Historically we already send a form of UITelemetry data, but the current format is too complicated to work with and to maintain.
=== Dates ===
* ...: Event data explorable in re:dash (from pre-release channels)
* '''Fx52''' (~2017-03-07): Event data collection implemented in Firefox Telemetry


=== Goals for Event Telemetry ===
=== Goals for Event Telemetry ===
* Enable exploratory usage behavior analysis
* Standardized event format for all pings collecting events (Main Ping, Test Pilot, Shield, Sync, etc.)
* Enable event data collection from Firefox and addons
* Expand collection of UI Telemetry to support Product teams
* Enable top line metrics for 2017
* Standardized support for analysis of events
* API support in client for collecting event telemetry


=== Documentation ===
=== Documentation ===
* [https://docs.google.com/document/d/1hNuS9lUJMvMqgntZXbFA6xZBU9zBpQgo7x73-sXKRpI/ Event Telemetry draft]
* [https://docs.google.com/document/d/1hNuS9lUJMvMqgntZXbFA6xZBU9zBpQgo7x73-sXKRpI/ Event Telemetry draft]
* [https://gecko.readthedocs.org/en/latest/toolkit/components/telemetry/telemetry/index.html In-tree docs (to be written)]
* [https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/collection/events.html In-tree docs]
* [https://docs.google.com/document/d/1cFCymhLQE7qI-p_czzz9-KexCMMhnf9ezLTMkGAKj58/edit#heading=h.w4fgaxpswo Table of projects that will be collecting Event telemetry]


=== Analysis and Reporting ===
=== Analysis and Reporting ===
* Raw data using a spark cluster: https://telemetry-dash.mozilla.org/
* Raw data using a spark cluster (ATMO): https://analysis.telemetry.mozilla.org/
* re:dash event data tables
* re:dash event data tables (STMO): https://sql.telemetry.mozilla.org/


= Project =
= Project =


=== Deliverables ===
=== Deliverables ===
*  
* '''2016 Q3'''
** Project kickoff
** Common Event Format design
** Event registration mechanism design
** Dataset strategy ([https://docs.google.com/document/d/1FI-jvzE4nVdas3e0o3QauNXUf5aQg9BGuXyVz0cDC1I/edit Event Telemetry Data sets discussion])
* '''2016 Q4'''
** Telemetry events on main pings implemented in Firefox
** Activity Stream submitting events (using common format, in main ping)
** Event data accessible in Spark
** Generic Event data table available in re:dash
** Test Pilot submitting events (using common format)
** Custom sync ping is submitting events (using common format)
* '''2017 Q1'''
** More custom pings submitting events (using common format)
** Non-specialized, analysis-oriented datasets available
** Mobile plan (Q1/TBD)
** Some project (TBD) is on the main ping


=== Client work ===
=== Client work ===
* Backlog as [https://docs.google.com/a/mozilla.com/spreadsheets/d/1yAJmgCGYyk1d7A41DZa653Z3u2AbH-kDWsO1vPSgbfE/edit?usp=sharing spreadsheet], with estimates
* [https://bugzilla.mozilla.org/showdependencytree.cgi?id=1286606&hide_resolved=1 client bug tree]
* Bug tree, phase 4: https://bugzilla.mozilla.org/show_bug.cgi?id=1122482
* [http://georgf.github.io/measurements-dash/ current sprint]
* Bug tree, phase 3: https://bugzilla.mozilla.org/show_bug.cgi?id=1120356 (Done)
* Bug tree, phase 2: https://bugzilla.mozilla.org/show_bug.cgi?id=1069869 (Done)
* Bug tree, phase 1: https://bugzilla.mozilla.org/show_bug.cgi?id=1040800 (Done)
 
=== Pipeline work ===
* Bugzilla: http://mzl.la/1KWiNST
 
=== Event Implementation Plan ===
* [https://docs.google.com/document/d/1cFCymhLQE7qI-p_czzz9-KexCMMhnf9ezLTMkGAKj58/edit#heading=h.w4fgaxpswo List of Event groups, events, owners etc]


=== Client Testing ===
=== Client Testing ===
* [https://docs.google.com/document/d/10sZICCbsfcSTF3RPyeVDskSI9-I2E4iApmShmIWSLfg/edit#heading=h.a6hfij6xookn Test cases document]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=1302670 Bugzilla tracking]
* [https://docs.google.com/a/mozilla.com/spreadsheets/d/1YxqvjRJuuIPRegNXAFCLHA7_56vhQ6leaZLaLeFqyxY/edit#gid=0 Spreadsheet to track testing]


= Communication =
= Communication =
* Conversation about Event telemetry on fhr-dev: https://mail.mozilla.org/listinfo/fhr-dev
* Conversation about Event telemetry on fhr-dev: https://mail.mozilla.org/listinfo/fhr-dev
* Data verification meeting notes: https://etherpad.mozilla.org/fhr-v4-status
* IRC: #telemetry
* IRC: #telemetry, #datapipeline, #metrics
* Slack: #fx-metrics
* [https://docs.google.com/document/d/1P0BmMRLSglX9G53-j5udU5CnrwDaqcHKP5fFjU5hEwo/edit Weekly Meeting notes]
* [https://docs.google.com/document/d/1P0BmMRLSglX9G53-j5udU5CnrwDaqcHKP5fFjU5hEwo/edit Weekly Meeting notes]
* [[Unified_Telemetry/Status_reports|EPM reports]]
* [[Unified_Telemetry/Status_reports|EPM reports]]
= Resources / Notes =
* [https://docs.google.com/document/d/1IGpzsYGi_sq3YFQDAPyKOkU_BKvXAC95fZYA2i4ceVs/edit?usp=sharing Kickoff document]
** "Query Requirements" section has list of sample queries/questions that get asked frequently of FHR data
* [https://docs.google.com/document/d/1FI-jvzE4nVdas3e0o3QauNXUf5aQg9BGuXyVz0cDC1I/edit Event Telemetry Data sets discussion]


= People and Roles =
= People and Roles =
* Georg Fritzsche  
* Georg Fritzsche (Data Platform, lead)
* Alessio Placitelli, :Dexter (client data collection)
* Alessio Placitelli, :Dexter (Data Platform)
* Mark Reid (data pipeline, telemetry server)
* Mark Reid (Data Platform)
* Roberto Vitillo
* Roberto Vitillo (Data Platform)
* Sunah Suh (Analysis)
* Sunah Suh (Data Platform)
* Rebecca Weiss (PM)
* Rebecca Weiss (PM)
* Ilana Segall (Analysis)
* Ilana Segall (Analysis)
* John Dorlus (Quality Engineering)
* John Dorlus (Quality Engineering)
* Roberto Vitillo (Spark analysis tool, telemetry data validation)
* Thomas Huelbert (project management)
* Thomas Huelbert (project management)

Latest revision as of 20:54, 19 May 2020

The Telemetry wiki page has more information about using Telemetry -- this page describes the Event Telemetry project.

Overview

There is a common need across teams (fx-team, mobile, test-pilot, heartbeat, …) to have a mechanism for recording, storing, sending & analysing application usage in an event-oriented format. The Data Platform team wants to support this with a common API and mechanisms for dealing with the collected data, without owning the individual measurements. The solution here is to provide common client code, a standard data format, so we can come up with common processes and tooling for data pipeline & analysis work. Historically we already send a form of UITelemetry data, but the current format is too complicated to work with and to maintain.

Goals for Event Telemetry

  • Standardized event format for all pings collecting events (Main Ping, Test Pilot, Shield, Sync, etc.)
  • Expand collection of UI Telemetry to support Product teams
  • Enable top line metrics for 2017
  • Standardized support for analysis of events
  • API support in client for collecting event telemetry

Documentation

Analysis and Reporting

Project

Deliverables

  • 2016 Q3
  • 2016 Q4
    • Telemetry events on main pings implemented in Firefox
    • Activity Stream submitting events (using common format, in main ping)
    • Event data accessible in Spark
    • Generic Event data table available in re:dash
    • Test Pilot submitting events (using common format)
    • Custom sync ping is submitting events (using common format)
  • 2017 Q1
    • More custom pings submitting events (using common format)
    • Non-specialized, analysis-oriented datasets available
    • Mobile plan (Q1/TBD)
    • Some project (TBD) is on the main ping

Client work

Client Testing

Communication

People and Roles

  • Georg Fritzsche (Data Platform, lead)
  • Alessio Placitelli, :Dexter (Data Platform)
  • Mark Reid (Data Platform)
  • Roberto Vitillo (Data Platform)
  • Sunah Suh (Data Platform)
  • Rebecca Weiss (PM)
  • Ilana Segall (Analysis)
  • John Dorlus (Quality Engineering)
  • Thomas Huelbert (project management)
Retrieved from "https://wiki.mozilla.org/index.php?title=Event_Telemetry&oldid=1227364"