106
edits
Telemetry/Alerts: Difference between revisions
linkification and name-checking Bhattacharyya
(A guide to triaging Telemetry Alert Emails.) |
(linkification and name-checking Bhattacharyya) |
||
| Line 2: | Line 2: | ||
Many Telemetry probes were created to show performance trends over time. Sudden changes happening in Nightly could be the sign of an unintentional performance regression, so we introduced a system to automatically detect and alert developers about such changes. | Many Telemetry probes were created to show performance trends over time. Sudden changes happening in Nightly could be the sign of an unintentional performance regression, so we introduced a system to automatically detect and alert developers about such changes. | ||
Thus we created Telemetry Alerts. It comes in two pieces: Cerberus the Detector and Medusa the Front-end. | Thus we created Telemetry Alerts. It comes in two pieces: [https://github.com/mozilla/cerberus/ Cerberus] the Detector and [https://github.com/mozilla/medusa/ Medusa] the Front-end. | ||
===Cerberus=== | ===Cerberus=== | ||
Every day Cerberus grabs the latest aggregated information about all non-keyed Telemetry probes from aggregates.telemetry.mozilla.org and compares the distribution of values from the '''Nightly''' builds of the past two days to the distribution of values from the Nightly builds of the past seven days. | Every day Cerberus grabs the latest aggregated information about all non-keyed Telemetry probes from aggregates.telemetry.mozilla.org and compares the distribution of values from the '''Nightly''' builds of the past two days to the distribution of values from the Nightly builds of the past seven days. | ||
It does this by calculating the [https://en.wikipedia.org/wiki/Bhattacharyya_distance Bhattacharyya distance] between the two distributions and guessing whether or not they are [https://github.com/mozilla/cerberus/blob/master/alert/alert.py#L72 significant and narrow]. | |||
It places all detected changes in a file for ingestion by Medusa. | It places all detected changes in a file for ingestion by Medusa. | ||