Once we have a reasonable baseline for the resources consumed by the scheduler while the dumps run, we should make sure the adjust the resource requests/limits in the deployment values.
Description
Description
Details
Details
Related Changes in Gerrit:
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T88728 Improve Wikimedia dumping infrastructure | |||
| Resolved | BTullis | T352650 WE 5.4 KR - Hypothesis 5.4.4 - Q3 FY24/25 - Migrate current-generation dumps to run on kubernetes | |||
| Resolved | brouberol | T388378 Orchestrate dumps v1 from an airflow instance | |||
| Resolved | brouberol | T390945 Run an experimental dump of 200 regular sized wikis | |||
| Resolved | brouberol | T391556 Adjust the scheduler resource requirements |
Event Timeline
Comment Actions
Change #1135724 had a related patch set uploaded (by Brouberol; author: Brouberol):
[operations/deployment-charts@master] airflow-test-k8s: increase the reserved resources for the airflow-test-k8s scheduler
Comment Actions
Change #1135724 merged by Brouberol:
[operations/deployment-charts@master] airflow-test-k8s: increase the reserved resources for the airflow-test-k8s scheduler
Comment Actions
Change #1135891 had a related patch set uploaded (by Brouberol; author: Brouberol):
[operations/deployment-charts@master] airflow-test-k8s: increase the limit ranges to support deploying a bigger scheduler
Comment Actions
Change #1135891 merged by Brouberol:
[operations/deployment-charts@master] airflow-test-k8s: increase the limit ranges to support deploying a bigger scheduler