The Telemetry wiki page has more information about using Telemetry -- this page describes the 2015 project.

Overview

There is a common need across teams (fx-team, mobile, test-pilot, heartbeat, …) to have a mechanism for recording, storing, sending & analysing application usage in an event-oriented format. The Data Platform team wants to support this with a common API and mechanisms for dealing with the collected data, without owning the individual measurements. The solution here is to provide common client code, a standard data format, so we can come up with common processes and tooling for data pipeline & analysis work. Historically we already send a form of UITelemetry data, but the current format is too complicated to work with and to maintain.

Dates

...: Event data explorable in re:dash (from pre-release channels)
Fx52 (~2017-03-07): Event data collection implemented in Firefox Telemetry

Goals for Event Telemetry

Enable exploratory usage behavior analysis
Enable event data collection from Firefox and addons

Documentation

Analysis and Reporting

Raw data using a spark cluster: https://telemetry-dash.mozilla.org/
re:dash event data tables

Project

Deliverables

Monitoring and alerting about pipeline health
Basic tool support
- Telemetry Dashboard works against new pipeline data
- Telemetry-dash (or new equivalent) can launch spark, heka reporting jobs
Derived data sets
- Executive dashboard rollup
- 1% sample of clientIds for longitudinal analysis
v2-v4 Data Continuity
- Executive dashboard continues to work
- Search analysis continues to work

Client work

Backlog as spreadsheet, with estimates
Bug tree, phase 4: https://bugzilla.mozilla.org/show_bug.cgi?id=1122482
Bug tree, phase 3: https://bugzilla.mozilla.org/show_bug.cgi?id=1120356 (Done)
Bug tree, phase 2: https://bugzilla.mozilla.org/show_bug.cgi?id=1069869 (Done)
Bug tree, phase 1: https://bugzilla.mozilla.org/show_bug.cgi?id=1040800 (Done)

Pipeline work

Bugzilla: http://mzl.la/1KWiNST

Client Testing

Communication

Conversation about unified telemetry on fhr-dev: https://mail.mozilla.org/listinfo/fhr-dev
Data verification meeting notes: https://etherpad.mozilla.org/fhr-v4-status
IRC: #telemetry, #datapipeline, #metrics
Unified Telemetry/Status reports
Unified Telemetry/Data Continuity

Resources

Kickoff document
- "Query Requirements" section has list of sample queries/questions that get asked frequently of FHR data

People and Roles

Georg Fritzsche (client data collection)
Alessio Placitelli, :Dexter (client data collection)
Mark Reid (data pipeline, telemetry server)
Michael Trinkala, :trink (data pipeline, heka)
Wesley Dawson, :whd (data pipeline operations)
Daniel Thornton, :relud (data pipeline operations)
Stuart Philp (test automation)
Anthony Zhang (Telemetry dashboard)
Roberto Vitillo (Spark analysis tool, telemetry data validation)
Brendan Colloran (metrics team, data validation)
Sam Penrose (metrics team, data validation)
Thomas Huelbert (project management)
Katie Parlante (eng manager)
Benjamin Smedberg (project sponsor, data steward)