Summary
As an MLE running LLM services on the MI300X nodes, I want to know how much GPU each deployed model is using (utilisation, VRAM, power, temperature), so I can tell which model is
driving GPU load and debug performance issues for a specific model.
Technical notes
GPU metrics and vLLM metrics are both already collected in Thanos, but today GPU usage is only visible per physical card — there is no way to tell which model is running on which
card. This task is to connect GPU usage to the model using it; the exact approach to be figured out as part of the work.
Acceptance criteria
- GPU usage can be viewed and filtered by model/deployment, not just by physical card.
- Wikitech documentation on how this works