Unified Telemetry: Difference between revisions
Jump to navigation
Jump to search
(→Acceptance Criteria: Add automation bug) |
(→Data validation: added bug titles and acceptance criteria) |
||
| Line 23: | Line 23: | ||
= Data validation = | = Data validation = | ||
=== Acceptance Criteria === | === Acceptance Criteria === | ||
* https://bugzilla.mozilla.org/show_bug.cgi?id=1134661 | * https://bugzilla.mozilla.org/show_bug.cgi?id=1134661 (An automated script to compare FHR v2 results and FHR-v4 for a sample of users) | ||
* rollup fields compare reasonably to v2 | |||
** # of sessions | |||
** session lengths | |||
** searches | |||
** default browser status | |||
** places counts | |||
=== Monitoring === | === Monitoring === | ||
* https://bugzilla.mozilla.org/show_bug.cgi?id=1147395 (Compare a few telemetry measurements between "saved-session" and "main" pings) | * https://bugzilla.mozilla.org/show_bug.cgi?id=1147395 (Compare a few telemetry measurements between "saved-session" and "main" pings) | ||
* https://bugzilla.mozilla.org/show_bug.cgi?id=1129185 | * https://bugzilla.mozilla.org/show_bug.cgi?id=1129185 (Reporting to make sure we don't have broken or incomplete session fragment chains) | ||
* https://bugzilla.mozilla.org/show_bug.cgi?id=1134669 (unified-FHR quality report: activity latency) | |||
=== Client Testing === | === Client Testing === | ||
* [https://docs.google.com/document/d/10sZICCbsfcSTF3RPyeVDskSI9-I2E4iApmShmIWSLfg/edit#heading=h.a6hfij6xookn Test cases document] | * [https://docs.google.com/document/d/10sZICCbsfcSTF3RPyeVDskSI9-I2E4iApmShmIWSLfg/edit#heading=h.a6hfij6xookn Test cases document] | ||
* [https://docs.google.com/a/mozilla.com/spreadsheets/d/1YxqvjRJuuIPRegNXAFCLHA7_56vhQ6leaZLaLeFqyxY/edit#gid=0 Spreadsheet to track testing] | * [https://docs.google.com/a/mozilla.com/spreadsheets/d/1YxqvjRJuuIPRegNXAFCLHA7_56vhQ6leaZLaLeFqyxY/edit#gid=0 Spreadsheet to track testing] | ||
Revision as of 22:55, 4 May 2015
Overview
We're unifying the Telemetry and Firefox Health Report collection systems on the client, and sending them through one Data Pipeline.
Goals
- On the client, unify the telemetry and FHR measurement systems so that measurements do not have to be implemented more than once in different systems.
- Reduce the latency from the time a measurement occurs until it can be analyzed on the server.
- Increase the accuracy of measurements so that they can be better correlated with factors in the user environment such as the specific build, enabled addons, and other hardware or software factors.
Resources
- Kickoff document
- Format documentation
Timeline
Client work
- Backlog as spreadsheet, with estimates
- Bug tree, phase 3: https://bugzilla.mozilla.org/show_bug.cgi?id=1120356
- Bug tree, phase 2: https://bugzilla.mozilla.org/show_bug.cgi?id=1069869 (Done)
- Bug tree, phase 1: https://bugzilla.mozilla.org/show_bug.cgi?id=1040800 (Done)
Data validation
Acceptance Criteria
- https://bugzilla.mozilla.org/show_bug.cgi?id=1134661 (An automated script to compare FHR v2 results and FHR-v4 for a sample of users)
- rollup fields compare reasonably to v2
- # of sessions
- session lengths
- searches
- default browser status
- places counts
Monitoring
- https://bugzilla.mozilla.org/show_bug.cgi?id=1147395 (Compare a few telemetry measurements between "saved-session" and "main" pings)
- https://bugzilla.mozilla.org/show_bug.cgi?id=1129185 (Reporting to make sure we don't have broken or incomplete session fragment chains)
- https://bugzilla.mozilla.org/show_bug.cgi?id=1134669 (unified-FHR quality report: activity latency)