13
edits
Data/Platform/Airflow Runbook: Difference between revisions
Jump to navigation
Jump to search
Add airflow version
(Whitespace) |
(Add airflow version) |
||
| Line 1: | Line 1: | ||
[https://github.com/mozilla/telemetry-airflow Airflow] is our workflow management system for telemetry batch jobs. The main project docs are [https://airflow.incubator.apache.org/ here]. This document describes the process for resolving issues when things go sideways. | [https://github.com/mozilla/telemetry-airflow Airflow] is our workflow management system for telemetry batch jobs. The main project docs are [https://airflow.incubator.apache.org/ here]. We're currently running airflow version 1.7.1.3 (the latest version available on PyPI.) This document describes the process for resolving issues when things go sideways. | ||
| Line 8: | Line 8: | ||
If you accidentally start DAG runs for dates that are either already processed or you're not interested in, the best course is often to mark the task(s) as `Success` from the web UI. To do this, click on the root task and, in the resulting modal dialog, click "Downstream" and then "Mark Success" to turn those task runs green. | If you accidentally start DAG runs for dates that are either already processed or you're not interested in, the best course is often to mark the task(s) as `Success` from the web UI. To do this, click on the root task and, in the resulting modal dialog, click "Downstream" and then "Mark Success" to turn those task runs green. | ||
[[File:Airflow task popup. | [[File:Airflow task popup.png|frameless|Click "Downstream" and then "Mark Sucess" in the task modal dialog]] | ||
This doesn't stop any actually currently running clusters, however, so find those running clusters on EMR and kill them. | This doesn't stop any actually currently running clusters, however, so find those running clusters on EMR and kill them. | ||