In T429597, SREs are working on creating grafana dashboards that show LLM inference services metrics.
Our LLM inference services run vLLM via the embedded vllm.LLM engine inside a KServe predictor (not vllm serve). On that offline path, engine stats are off by default and they are not emitted as shown in: P94723
When we set disable_log_stats=False, vLLM metrics are emitted on the same /metrics endpoint that KServe already exposes as shown in: P94724
Below are the vLLM model-servers where we are going to enable metrics:
- cope_b_a4b
- cope_a_9b
- gpt_oss_safeguard_20b
- embeddings
- llm-qwen3-14b
NB: vLLM labels its metrics with served_model_name, which defaults to the HF model path e.g model_name="zentropi-ai/cope-b-a4b", not cope-b-a4b. Our own KServe metrics and the legends shown in: https://grafana-rw.wikimedia.org/goto/ffqvoolrpfrwgf?orgId=default use the short isvc name. We'll align this on our side by passing served_model_name=<kserve model name> so vLLM emits model_name="cope-b-a4b", matching the rest as shown in: P94725