MozillaWiki

CloudServices/Sync/ReDash: Difference between revisions

Page Discussion

CloudServices/Sync/ReDash (view source)

Revision as of 17:58, 22 June 2017

3,171 bytes added ,  22 June 2017
failureReason was renamed to failure_reason as of https://github.com/mozilla/telemetry-batch-view/pull/205
(Added validation data table)
(failureReason was renamed to failure_reason as of https://github.com/mozilla/telemetry-batch-view/pull/205)
 
(4 intermediate revisions by one other user not shown)
Line 25: Line 25:
| took || bigint || Number of milli-seconds it took to Sync.
| took || bigint || Number of milli-seconds it took to Sync.
|-
|-
| failurereason || row(name varchar, value varchar) || Sync failure reason, or null if no failure.
| failure_reason || row(name varchar, value varchar) || Sync failure reason, or null if no failure.
|-
|-
| status || row(sync varchar, service varchar) || The status of sync after completion, or null is both statuses record success.
| status || row(sync varchar, service varchar) || The status of sync after completion, or null is both statuses record success.
Line 52: Line 52:
| status || varchar || The status of this engine after sync completed, or null is the status reflects success.
| status || varchar || The status of this engine after sync completed, or null is the status reflects success.
|-
|-
| failureReason || row(name varchar, value varchar) || Details of any errors recorded during the sync of this engine, or null on success.
| failure_reason || row(name varchar, value varchar) || Details of any errors recorded during the sync of this engine, or null on success.
|-
|-
| incoming || row(applied bigint, failed bigint, newFailed bigint, reconciled bigint) || Details of how many incoming records were applied, failed, failed for the first time in this sync, and reconciled.
| incoming || row(applied bigint, failed bigint, newFailed bigint, reconciled bigint) || Details of how many incoming records were applied, failed, failed for the first time in this sync, and reconciled.
Line 77: Line 77:
| problems || array(row(name varchar, count varchar)) || The problems identified. Problems with a count of 0 are excluded. Null on failure or if no problems occurred.
| problems || array(row(name varchar, count varchar)) || The problems identified. Problems with a count of 0 are excluded. Null on failure or if no problems occurred.
|-
|-
| failureReason || row(name varchar, value varchar) || Details of any errors recorded during the validation, or null if validation succeeded.
| failure_reason || row(name varchar, value varchar) || Details of any errors recorded during the validation, or null if validation succeeded.
|}
|}


Line 181: Line 181:
* There may be any number of repairResponse events written (including zero events if there are no suitable devices). Each of these should result in a event_object="repairResponse", event_method="finished/aborted"
* There may be any number of repairResponse events written (including zero events if there are no suitable devices). Each of these should result in a event_object="repairResponse", event_method="finished/aborted"


=== Validation Data Roll-ups ===
=== Bookmark Validation Data Roll-ups ===
==== telemetry.sync_bmk_total_per_day ====
==== telemetry.sync_bmk_total_per_day ====


This table is a roll-up of all bookmark validations (success and failures).
This table is a roll-up of all bookmark validations (success and failures).
This table is useful to calculate rates in conjunction with the ''telemetry.sync_bmk_validation_problems'' table.


{| class="wikitable"
{| class="wikitable"
Line 201: Line 203:
|}
|}


==== telemetry.sync_bmk_validation_problems ====
This table was rolled-up using the following workbook:
https://gist.github.com/kitcambridge/364f56182f3e96fb3131bf38ff648609
This table has a row for every validation problem detected during validation.
'''Example:'''
If a user has 4 different validation problems, we will log each problem in their own row.
'''Logic used for format:'''
This choice was made to avoid having endless amounts of columns representing each validation problem checked (which would mostly all be null) and to avoid having to add columns as we detect new types of validation problems.
{| class="wikitable"
|-
! Field name !! Data type !! Description
|-
| app_build_id || varchar || Firefox build ID (e.g. 20160801085519)
|-
| app_version || varchar || Firefox version (e.g. 50.0a2) - corresponds to the MOZ_APP_VERSION configure value
|-
| app_display_version || varchar || The application version as shown in the about dialog. Almost always identical to app_version. Corresponds to the  MOZ_APP_VERSION_DISPLAY configure value.
|-
| app_name || varchar || Will always be the string "Firefox" in desktop submitted pings.
|-
| app_channel || varchar || The update channel (e.g. "release", "nightly", etc) - corresponds to the  MOZ_UPDATE_CHANNEL config variable.
|-
| uid || varchar || Hashed Sync/FxA ID
|-
| deviceid || varchar || Hashed FxA device ID.
|-
| submission_day || integer || Date data was added to Presto (e.g. 20170324)
|-
| sync_day || integer || Date of sync. (e.g. 20170320)
|-
| when || bigint || Unix timestamp of when sync occurred. Make sure to put in "quotes" since when is a reserved SQL word. Note that because this is taken from the client's clock, the time may be wildly inaccurate.
|-
| status || row(sync varchar, service varchar) || The status of sync after completion, or null is both statuses record success.
|-
| engine_name || varchar || Should always be '''bookmarks''' in this table.
|-
| engine_status || varchar || The status of this engine after sync completed, or null if the status reflects success.
|-
| engine_failure_reason || row(name varchar, value varchar) || Details of any errors recorded during the sync of this engine, or null on success.
|-
| engine_has_problems || true/false || engine_has_problems is always going to be true in sync_bmk_validation_problems. Used to filter results out of all_engine_validation_results
|-
| engine_validation_version || integer || Version of the validator used to get this data.
|-
| engine_validation_checked || bigint || Number of records the validator checked in this engine.
|-
| engine_validation_took || bigint || How long validation took for this engine.
|-
| engine_validation_problem_name || varchar || Name of error recorded during the validation. Will never be null since this table only includes validation problems.
|-
| engine_validation_problem_count || integer || Number of bookmarks afflicted by the problem during the validation check.
|-
| run_start_date || integer || Date added to presto. (e.g. 20170331)
|}


=== Query Examples ===
=== Query Examples ===
Tcsc
8

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1174251...1174428"