We currently hardcode --date last in the worker command, but we should infer the date value from the DAG scheduling date, and format it as YYYYMMDD.
Description
Description
Details
Details
Related Changes in GitLab:
| Title | Reference | Author | Source Branch | Dest Branch | |
|---|---|---|---|---|---|
| test_k8s/dumpsv1: format the --date flag from the DAG run start date | repos/data-engineering/airflow-dags!1224 | brouberol | T391492 | main |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T88728 Improve Wikimedia dumping infrastructure | |||
| Resolved | BTullis | T352650 WE 5.4 KR - Hypothesis 5.4.4 - Q3 FY24/25 - Migrate current-generation dumps to run on kubernetes | |||
| Resolved | brouberol | T388378 Orchestrate dumps v1 from an airflow instance | |||
| Resolved | brouberol | T391492 Infer the --date parameter from the DAG scheduling date |
Event Timeline
Comment Actions
brouberol merged https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/1224
test_k8s/dumpsv1: format the --date flag from the DAG run start date