Telemetry/Reboot: Difference between revisions

Latest revision as of 23:19, 23 October 2013

Goal: Fast, Robust server & frontend operational by Sep 30th able to accept and graph telemetry data. This means that as of Oct 1 http://metrics.mozilla.com/ will stop updating. Metrics should be able to retire telemetry portion of mango cluster in October.

Telemetry Backend

Migration

Nov 1: Start porting dashboard generation script to hadoop
Nov 7: Start running ^ script
Nov 30: Turn off hadoop

Telemetry Dashboard

Dashboard(https://github.com/mozilla/telemetry-dashboard/) Milestones:

Sep 1: Add median, percentiles, 90% ready
Sep 15: ^ finished
Sep 15: Produce a second set of dashboards(same codebase, probably different http server) that are being fed data from AWS

Open bugs: https://bugzilla.mozilla.org/buglist.cgi?product=Webtools&component=Telemetry%20Dashboard&list_id=7702794

Misc

Notes:

Original etherpad https://etherpad.mozilla.org/telemetry-reboot

@@ Line 1: / Line 1: @@
-Goal: Fast, Robust server & frontend operational by '''Sep 31st''' able to accept and graph telemetry data. This means that as of Oct 1 http://metrics.mozilla.com/ will stop updating. Metrics should be able to retire telemetry portion of mango cluster in October.
+Goal: Fast, Robust server & frontend operational by '''Sep 30th''' able to accept and graph telemetry data. This means that as of Oct 1 http://metrics.mozilla.com/ will stop updating. Metrics should be able to retire telemetry portion of mango cluster in October.
-Server requirements:
+==Telemetry Backend==
-* Ability to process 10x incoming packet rates of metrics telemetry infrastructure on a single AWS instance: 2400req/s with 30K HTTP POST packets. Fall
+Migration
-* Server should be bandwidth-limited, not CPU.
+* Nov 1: Start porting dashboard generation script to hadoop
-* Server should make data available for map/reduce immediately. Fallback goal: 5min latency
+* Nov 7: Start running ^ script
+* Nov 30: Turn off hadoop
-Server Milestones:
+==Telemetry Dashboard==
-* '''Sep 2''': Ability temporarily(1 hour?) point telemetry dns at AWS, forwarding telemetry to metrics cluster. Ideally we'd be able to do this at bouncer level so no changes are needed to accomplish forwarding
+Dashboard(https://github.com/mozilla/telemetry-dashboard/) Milestones:
-* '''Sep 16''': ^ should be feeding both servers (metrics and AWS) until cutover date.
+*Sep 1: Add median, percentiles, 90% ready
+*Sep 15: ^ finished
+*Sep 15: Produce a second set of dashboards(same codebase, probably different http server) that are being fed data from AWS
-Dashboard Milestones
+Open bugs:  https://bugzilla.mozilla.org/buglist.cgi?product=Webtools&component=Telemetry%20Dashboard&list_id=7702794
+==Misc==
+Notes:
 * Original etherpad https://etherpad.mozilla.org/telemetry-reboot