Page MenuHomePhabricator

refine_to_hive dag optimizations
Open, MediumPublic

Description

Refine currently generates a high overhead for relatively small work units:

  • Many templated calls generating individual requests/jobs.
  • Small streams processed as separate tasks instead of being grouped.
  • Precomputation of all configuration in configuration step and storage in xcoms

This leads to:

  • Large numbers of task instances (200*3=600 tasks per run).
  • pressure on DB (large rows + number of rows)
  • Increased dependency check times.

First improvements:

  • Run small jobs within pod (local spark) as it's done in analytics-test
  • Only store in xcoms the minimal information needed to launch downstream tasks T375064
  • Compute configuration at execution time instead of precomputing all in configuration step

We’ll start by testing medium and low resource profiles and adjust based on observations from:
Kubernetes Resources dashboard
Airflow Instance dashboard
• Pod-level stats

Eventually, later, we can consider batching. We would lose some visibility on individual items but
we would reduce DB pressure and number of task instances.

Huge thanks to @brouberol for the guidance!

Event Timeline

Change #1149617 had a related patch set uploaded (by Brouberol; author: Brouberol):

[operations/deployment-charts@master] airflow: grant analytics airflow instances access to schema.deployment.wmnet

https://gerrit.wikimedia.org/r/1149617

Change #1149617 merged by Brouberol:

[operations/deployment-charts@master] airflow: grant airflow instances access to schema.deployment.wmnet

https://gerrit.wikimedia.org/r/1149617

Change #1149628 had a related patch set uploaded (by Brouberol; author: Brouberol):

[operations/deployment-charts@master] airflow: deploy a tiny toolbox allowing users to test task networking

https://gerrit.wikimedia.org/r/1149628

Change #1149628 merged by Brouberol:

[operations/deployment-charts@master] airflow: deploy a tiny toolbox allowing users to test task networking

https://gerrit.wikimedia.org/r/1149628

Change #1149638 had a related patch set uploaded (by Brouberol; author: Brouberol):

[operations/deployment-charts@master] Enable talking to schema.discovery.wmnet via the service mesh

https://gerrit.wikimedia.org/r/1149638

Change #1149639 had a related patch set uploaded (by Brouberol; author: Brouberol):

[operations/deployment-charts@master] airflow: disable hardcoded networkpolicy in favor of the service mesh

https://gerrit.wikimedia.org/r/1149639

Change #1149638 merged by Brouberol:

[operations/deployment-charts@master] Enable talking to schema.discovery.wmnet via the service mesh

https://gerrit.wikimedia.org/r/1149638

Change #1149639 merged by Brouberol:

[operations/deployment-charts@master] airflow: disable hardcoded networkpolicy in favor of the service mesh

https://gerrit.wikimedia.org/r/1149639

Next steps are:

  • analytics_test: skipping skein and using Spark local in pod.
  • main: running small tasks with LocalExecutor. It's limited to a parallelization of ~13 but the improvement on airflow task warmup makes it ~4 time more efficient than KubernetesExecutor. (already running in analytics_test). Plays well with sensors, hdfs touch operator or pure jvm jobs.
    • refine_to_hive_dag
    • canary_events_dag
  • main: skipping skein
Antoine_Quhen renamed this task from Refine to Hive with Airflow – Kubernetes Resource Optimization to refine_to_hive dag optimizations.Dec 8 2025, 9:22 AM
Antoine_Quhen updated the task description. (Show Details)
Ahoelzl changed the task status from In Progress to Open.Jan 12 2026, 9:27 PM
Ahoelzl triaged this task as Medium priority.

I think we can push more config setup at execution time. This would be a big win and not huge work.