Change Details

==== Migrate existing Airflow DAGs exporting metrics to Graphite to export to Prometheus. Some Airflow DAGs seem to be configured to send metrics to Graphite. e.g. https://gitlab.wikimedia.org/search?search=graphite&nav_source=navbar&project_id=93&group_id=189&search_code=true&repository_ref=main This is a request to review and migrate any graphite metric-producing DAGs to Prometheus. [] Review all in-use Airflow DAGs that emit metrics to Graphite and migrate the needed [] Modify the Airflow configuration to redirect metric collection from Graphite to Prometheus using StatsD. [] Verify that metrics are correctly collected and visible in Prometheus. [] Ensure that all key metrics previously captured in Graphite are correctly reflected in Prometheus. [] Test existing or new Grafana dashboards to confirm accurate metric visualization. Resources: - [[ https://wikitech.wikimedia.org/wiki/Prometheus#Statsd | prometheus statsd documentation ]] - [[ https://wikitech.wikimedia.org/wiki/Prometheus/statsd_k8s | prometheus statsd documentation for kubernetes]] - {T359056}

==== Disable existing Airflow DAGs exporting metrics to Graphite. Some Airflow DAGs seem to be configured to send metrics to Graphite. e.g. https://gitlab.wikimedia.org/search?search=graphite&nav_source=navbar&project_id=93&group_id=189&search_code=true&repository_ref=main Prometheus does not support historical data, so we cannot migrate these pipelines to produced Prometheus. [] Review all in-use Airflow DAGs that emit metrics to Graphite [] Contact teams/owners of these dags and work with them to use different means of viewing these metrics [] Disable and remove all Graphite generating airflow dags [] Where possible, delete relevant Grafana dashboards that use the relevant Graphite metrics. Resources: - [[ https://wikitech.wikimedia.org/wiki/Prometheus#Statsd | prometheus statsd documentation ]] - [[ https://wikitech.wikimedia.org/wiki/Prometheus/statsd_k8s | prometheus statsd documentation for kubernetes]] - {T359056}

==== MigrateDisable existing Airflow DAGs exporting metrics to Graphite to export to Prometheus. Some Airflow DAGs seem to be configured to send metrics to Graphite. e.g. https://gitlab.wikimedia.org/search?search=graphite&nav_source=navbar&project_id=93&group_id=189&search_code=true&repository_ref=main This is a request to review andPrometheus does not support historical data, so we cannot migrate any graphite metric-these pipelines to producing DAGs toed Prometheus. [] Review all in-use Airflow DAGs that emit metrics to Graphite and migrate the needed [] Modify the Airflow configuration to redirect metric collection from Graphite to Prometheus using StatsD.Contact teams/owners of these dags and work with them to use different means of viewing these metrics [] Verify that metrics are correctly collected and visible in Prometheus.Disable and remove all Graphite generating airflow dags [] Ensure that all key metrics previously captured in Graphite are correctly reflected in Prometheus. [] Test existing or new Grafana dashboards to confirm accurate metric visualizationWhere possible, delete relevant Grafana dashboards that use the relevant Graphite metrics. Resources: - [[ https://wikitech.wikimedia.org/wiki/Prometheus#Statsd | prometheus statsd documentation ]] - [[ https://wikitech.wikimedia.org/wiki/Prometheus/statsd_k8s | prometheus statsd documentation for kubernetes]] - {T359056}