Airflow scheduler currently runs in its own readonly conda environment. This is good. But jobs will need various dependencies installed. We should make it possible and easy to install those dependencies for use by jobs run in the Airflow executor.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | odimitrijevic | T282033 Airflow collaborations | |||
| Resolved | Ottomata | T294024 [Airflow] Automate sync'ing archiva packages to HDFS | |||
| Declined | None | T288271 Make it possible to use anaconda + stacked conda envs for Airflow executors |
Event Timeline
Comment Actions
We need to resolve computational resources for Airflow and discuss depending on the executors that are implemented.
Comment Actions
going to decline this, i don't think this is quite right. Discussion in T296543: Tooling for Deploying Conda Environments