| brouberol | |
| Jan 12 2024, 8:17 AM |
| F41667358: image.png | |
| Jan 12 2024, 8:17 AM |
We currently request 4GB of RAM, with a limit set to 8GB, but we currently sit at 2GB RAM usage due to SPARK_DAEMON_MEMORY being hardcoded to 2g.
We should probably set Xms to a value related to $.Values.app.requests.memory and Xmx to a value related to $.Values.app.limits.memory instead, to make sure that the JVM memory constraint is tied to what we request the scheduler and set in the cgroup, while keeping space for off-heap data.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | brouberol | T330176 [Data Platform] Deploy Spark History Service | |||
| Resolved | brouberol | T354929 Tweak Spark History memory settings |
Change 990033 had a related patch set uploaded (by Brouberol; author: Brouberol):
[operations/deployment-charts@master] spark-history: align Xmx/Xms valuea with amount of requested memory
Change 990033 merged by Brouberol:
[operations/deployment-charts@master] spark-history: align Xmx/Xms values with amount of requested memory