Page MenuHomePhabricator

Delete datahub WikiKube release references
Closed, ResolvedPublic

Description

We have deployed the datahub service on dse-k8s both providing a staging environment and the production environment. The last steps are :

  • deleting the release on WikiKube as per delete_a_release
  • deleting the helmfiles
  • deleting the namespaces
  • deleting the kubeconfigs

Event Timeline

Gehel triaged this task as Medium priority.May 31 2024, 8:30 AM

Change #1038773 had a related patch set uploaded (by Stevemunene; author: Stevemunene):

[operations/deployment-charts@master] Clean up datahub from main cluster

https://gerrit.wikimedia.org/r/1038773

Ran the delete on the wikikube clusters with

cd /srv/deployment-charts/helmfile.d/services/datahub/

helmfile -e staging --interactive destroy

helmfile -e eqiad --interactive destroy

helmfile -e codfw --interactive destroy

Next whould be cleaning up the helmfiles and namespaces

Change #1038773 merged by jenkins-bot:

[operations/deployment-charts@master] Clean up datahub from main cluster

https://gerrit.wikimedia.org/r/1038773

There were some remaining jobs on the eqiad cluster,

stevemunene@deploy1002:~$ kubectl get jobs
NAME                                   COMPLETIONS   DURATION   AGE
datahub-main-elasticsearch-setup-job   1/1           7s         34d
datahub-main-mysql-setup-job           1/1           10s        34d
datahub-main-nocode-migration-job      1/1           81s        34d
datahub-main-system-update-job         1/1           91s        34d

Deleted them as below

kubectl delete jobs datahub-main-elasticsearch-setup-job datahub-main-mysql-setup-job datahub-main-nocode-migration-job datahub-main-system-update-job -n datahub
job.batch "datahub-main-elasticsearch-setup-job" deleted
job.batch "datahub-main-mysql-setup-job" deleted
job.batch "datahub-main-nocode-migration-job" deleted

Verified no remaining jobs and pods in all environments

stevemunene@deploy1002:~$ kube_env datahub eqiad
stevemunene@deploy1002:~$ kubectl get pods
No resources found in datahub namespace.
stevemunene@deploy1002:~$ kube_env datahub codfw
stevemunene@deploy1002:~$ kubectl get pods
No resources found in datahub namespace.
stevemunene@deploy1002:~$ kube_env datahub staging
stevemunene@deploy1002:~$ kubectl get pods
No resources found in datahub namespace.

Then deleted the helmfiles, values and namespace from main

Change #1039618 had a related patch set uploaded (by Stevemunene; author: Stevemunene):

[operations/puppet@production] Delete datahub kubeconfigs on main

https://gerrit.wikimedia.org/r/1039618

Stevemunene renamed this task from Delete datahub WikiKube release to Delete datahub WikiKube release references.Jun 6 2024, 9:16 AM
Stevemunene updated the task description. (Show Details)

Change #1039618 merged by Stevemunene:

[operations/puppet@production] Delete datahub kubeconfigs on main

https://gerrit.wikimedia.org/r/1039618

Namespaces were deleted with the below after merging https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1038773

cd /srv/deployment-charts/helmfile.d/admin_ng
helmfile -e staging-eqiad -i apply --context 5
helmfile -e staging-codfw -i apply --context 5
helmfile -e eqiad -i apply --context 5
helmfile -e codfw -i apply --context 5