Page MenuHomePhabricator

OpenSearch on K8s: Figure out whether secrets should be destroyed by helmfile/fix or accept
Closed, ResolvedPublic

Description

While working through T408012, I noticed the helmfile isn't completely cleaning up all resources:

bking@deploy2002:~/deployment-charts/helmfile.d/dse-k8s-services/opensearch-ipoid-test$ helmfile -e dse-k8s-eqiad -i destroy
Listing releases matching ^opensearch-ipoid-test$
opensearch-ipoid-test   opensearch-ipoid-test   1               2025-10-29 20:52:48.40303069 +0000 UTC  deployed        opensearch-cluster-0.0.8     2.7.0

Affected releases are:
  opensearch-ipoid-test (wmf-stable/opensearch-cluster)

Do you really want to delete?
  Helmfile will delete all your releases, as shown above.

 [y/n]: y
Deleting opensearch-ipoid-test
release "opensearch-ipoid-test" uninstalled


DELETED RELEASES:
NAME                    NAMESPACE               DURATION
opensearch-ipoid-test   opensearch-ipoid-test         1s

Post-delete, we are still seeing some resources:

bking@deploy2002:~/deployment-charts/helmfile.d/dse-k8s-services/opensearch-ipoid-test$ kubectl get secret
NAME                                   TYPE                                  DATA   AGE
default-token-mjfjm                    kubernetes.io/service-account-token   3      54d
opensearch-ipoid-test-admin-password   Opaque                                2      7m5s
opensearch-wmf                         kubernetes.io/tls                     3      7m3s
opensearch-wmf-admin                   kubernetes.io/tls                     3      7m4s

Creating this ticket to:

  • Figure out whether this is expected behavior or not
  • If not, ensure the secrets resources are properly removed

Event Timeline

bking claimed this task.
bking added a subscriber: brouberol.

Per Slack conversation with @brouberol , this is expected behavior, as helm does not manage the certificate resources (cert-manager does). As such, I'm closing out this ticket.