| dcaro | |
| Aug 12 2025, 12:34 PM |
| F65741762: image.png | |
| Aug 12 2025, 2:16 PM |
| F65741739: image.png | |
| Aug 12 2025, 2:14 PM |
It seems that due to a change in the default configuration:
https://github.com/kyverno/kyverno/commit/eb72b04d2c130fabd39bdd0eec0ea36fd8a08c70
The metric kyverno_policy_rule_info_total is being exported without the namespace, and that causes it to not report per-tool metrics. Given that it's what we use for alerts and graphs (https://grafana-rw.wmcloud.org/d/kyverno/kyverno), we should either fix it or stop using that metric.
The alert Toolforge Kyverno has low amount of policy resources.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T414596 [infra,k8s] Upgrade Toolforge Kubernetes to version 1.35 | |||
| Open | None | T414595 [infra,k8s] Upgrade Toolforge Kubernetes to version 1.34 | |||
| Open | None | T408785 [infra,k8s] Upgrade Toolforge Kubernetes to version 1.33 | |||
| Open | None | T379047 [infra,k8s] Upgrade Toolforge Kubernetes to version 1.32 | |||
| Resolved | taavi | T372697 [infra,k8s] Upgrade Toolforge Kubernetes to version 1.31 | |||
| Open | None | T335131 [infra,k8s] replace admission controllers with an existing policy admin project | |||
| Open | None | T364293 [infra,k8s] Move to kubernetes VAPs and drop kyverno | |||
| Resolved | dcaro | T362869 [k8s,infra] Upgrade Toolforge to Uwubernetes (1.30) | |||
| Resolved | dcaro | T394787 [kyverno] Upgrade to `3.3.9` chart (`1.13` app) for k8s 1.30 support | |||
| Resolved | dcaro | T401681 [kyverno] policy countres stopped showing correctly in grafana |
Updated the config to not delete the namespace related labels from that metric, and they are showing up now on grafana:
I also improved the graphs to pick up the correct datasource (it was hard-coded, ignoring the selected one).
Hmm... the alert is summing by pod, without caring if there's any old pods:
So it will take some time to go away, let me try to improve the query instead