Folks are looking into job manager/scheduler software for use with analytics, search, dumps, CI. Can we find a solution good enough for us all? We won't know until we discuss our common needs.
Let's each summarize in a couple paragraphs our basic needs. An IRC discussion will follow, the content of which will also be summarized on this task.
- T217059: Spike [2019-2020 work] Oozie Replacement. Airflow Study / Argo Study
- T218812: RFC: Provide the ability to have time-delayed or time-offset jobs in the job queue
- T128520: What already available software can we build on for a job scheduler?
- T48643: [Story] Dispatching via delayed jobs (instead of cron script)
- T270613: [SPIKE] can we orchestrate (parts of) ETL with airflow?
Also: CI Seakeeper Proposal