Unified Telemetry/Status reports/dataprogram/datapipeline

From MozillaWiki
Jump to: navigation, search

Data Pipeline, desktop client, SvcOps report July 10, 2015

Status for desktop client, data pipeline and service ops.

Overall Project Health

  • Pipeline: Green
  • desktop client: Yellow
  • SvcOps: Green

Exec Summary

  • Issue with TestPilot are being investigated at priority
  • Investigating multi-backend data sources for presto, to allow querying both server side and client data together.
  • V2 edge provisioning
  • V3 of main_summary is impending
  • Intern started this week (w00t)
  • Scalar measurement work begins


Desktop Telemetry
  • scalar measurements work kicked off
  • Planning keyed histogram hardening after recent events
  • Also planning analysis to find more crazy keyed histograms
  • HTTP Date header uplift requested
Infrastructure and Tools
  • Reordered the sprint slightly since the popen bug is no longer blocking
  • Moved the S3 tooling up into this sprint
  • Rob + Roberto looking into multi-backend data sources for presto (for querying server- and client-data together)
  • working on v2 edge provisioning
  • rpms for librdkafka and luasandbox (in lib64)
  • building nginx/openresty 1.9.15rc1 for nginx_moz_ingest
  • testing in stage
    • Blake PTO, but working on presto (hive centralization)
  • Investigations / discussion about latency w.r.t. MAU/DAU
  • Server-data analysis proceeding nicely
  • Decoder Pull request pending
  • Seems this was merged, deploy going out today
  • Bunch of pipeline PRs landed
Other Reporting
  • V3 of main_summary is impending. Please let mreid know about any fields that should be added asap. * So far, planning to address bugs 1270482 (push metrics), 1270505 (client date), 1265874 (default search engine id)
  • fxa uid hmacification for email retention analysis for rfkelly
  • Loop - trink implemented loop retention reports. To be reportified?


Job description: https://careers.mozilla.org/position/ocIi2fwY

Important Links/References