Page MenuHomePhabricator

Develop periodic health report for Kubernetes cluster objects
Open, LowPublic

Description

During the final stages of the legacy Kubernetes cluster migration a non-trivial number of orphan ReplicaSets, Services, and Pods were found to be running. These generally looked to be objects leaked over time due to various bugs in the webservice command.

It would be nice to have a deep state check across all namespaces periodically (weekly? daily?) which produced a report of suspicious looking objects that should be investigated further. Any Pod that is not tied to a ReplicaSet or similar control object, any ReplicaSet not tied to a Deployment, any Service or Ingress pointing at non-existent ports, any Pods in CrashLoopBackoff or Scheduling state, etc.

This could probably be added as a feature in the k8s-status tool to leverage its ability to look at objects across namespaces.