8
edits
CloudServices/Sync/ReDash: Difference between revisions
Jump to navigation
Jump to search
Document current data format more extensively.
(added validation) |
(Document current data format more extensively.) |
||
| Line 1: | Line 1: | ||
=== Table Described === | === Table Described === | ||
The Sync team's telemetry goes through our data pipeline and lands in Presto to be explored in [https://sql.telemetry.mozilla.org/ Re:Dash]. Below is a description of the data available in the '''sync_summary''' table. | The Sync team's telemetry goes through our data pipeline and lands in Presto to be explored in [https://sql.telemetry.mozilla.org/ Re:Dash]. Below is a description of the data available in the '''sync_summary''' table. Further documentation on this data is available [https://github.com/mozilla/telemetry-batch-view/blob/master/docs/SyncSummary.md in the telemetry-batch-view repo], and in the [http://gecko.readthedocs.io/en/latest/toolkit/components/telemetry/telemetry/data/sync-ping.html general sync ping documentation]. | ||
{| class="wikitable" | {| class="wikitable" | ||
| Line 14: | Line 14: | ||
|- | |- | ||
| app_name || varchar || Will always be the string "Firefox" in desktop submitted pings. | | app_name || varchar || Will always be the string "Firefox" in desktop submitted pings. | ||
|- | |||
| app_channel || varchar || The update channel (e.g. "release", "nightly", etc) - corresponds to the MOZ_UPDATE_CHANNEL config variable. | |||
|- | |- | ||
| uid || varchar || Hashed Sync/FxA ID | | uid || varchar || Hashed Sync/FxA ID | ||
| Line 28: | Line 30: | ||
|- | |- | ||
| why || varchar || Currently always null, but eventually should be the reason the sync was performed (eg, timer, button press, score update, etc) | | why || varchar || Currently always null, but eventually should be the reason the sync was performed (eg, timer, button press, score update, etc) | ||
|- | |||
| devices || array(row(id varchar, os varchar, version varchar)) || Array of the other devices in this user's device constellation. | |||
|- | |- | ||
| engines || array(engine_record) || A record of the engines that synced. Each element of the array is in the format of an [[#Engine Record|engine record]]. | | engines || array(engine_record) || A record of the engines that synced. Each element of the array is in the format of an [[#Engine Record|engine record]]. | ||
| Line 53: | Line 57: | ||
|- | |- | ||
| outgoing || array(row(sent bigint, failed bigint))) || For each batch of records uploaded as part of a Sync, how many records were sent in that batch and how many failed. | | outgoing || array(row(sent bigint, failed bigint))) || For each batch of records uploaded as part of a Sync, how many records were sent in that batch and how many failed. | ||
|- | |- | ||
|validation || | | validation || validation_record || Validation information for this engine. Null if validation cannot or did not run during this sync (common). If present, it's format is of a [[#Validation Record|validation record]] | ||
|} | |||
===== Validation Record ===== | |||
A validation record is defined as: | |||
{| class="wikitable" | |||
|- | |||
! Field Name!! Data Type !! Description | |||
|- | |||
| version || bigint || Version of the validator used to get this data. | |||
|- | |||
| checked || bigint || Number of records the validator checked in this engine. | |||
|- | |||
| took || bigint || How long validation took for this engine. | |||
|- | |||
| problems || array(row(name varchar, count varchar)) || The problems identified. Problems with a count of 0 are excluded. Null on failure or if no problems occurred. | |||
|- | |||
| failureReason || row(name varchar, value varchar) || Details of any errors recorded during the validation, or null if validation succeeded. | |||
|} | |} | ||
| Line 68: | Line 91: | ||
A2: I don't think that's true - eg, "select * from sync_summary where engines is not null and status is null limit 10" shows records. | A2: I don't think that's true - eg, "select * from sync_summary where engines is not null and status is null limit 10" shows records. | ||
--- | |||
Q3: What values are valid in the validation record's '''name''' field? | |||
A3: It's up to the engine and to the client. For desktop bookmarks, they're documented [https://dxr.mozilla.org/mozilla-central/source/services/sync/modules/bookmark_validator.js here] (or [https://dxr.mozilla.org/mozilla-central/rev/1d025ac534a6333a8170a59a95a8a3673d4028ee/services/sync/modules/bookmark_validator.js#27-68 here] if that link breaks). | |||
=== Query Examples === | === Query Examples === | ||