The Airflow backend Postgres instance shows signs of performance pressure, particularly during busy periods (e.g. beginning of the month on the main instance). We need a systematic investigation of where time is spent and what the main bottlenecks are.
- Identify the top offending queries that consume most of the DB resources (e.g. with pg_stat_statements, or slow-query logs). Leverage existing Airflow Postgres dashboards.
- Determine whether the main bottleneck is CPU, IO, locks, external back pressure (Ceph), or N+1 patterns between core Airflow tables. Focus on beginning of the month load patterns.
- Propose and implement targeted optimizations (new indexes, query changes, config tuning).