Page MenuHomePhabricator

OpenSearch on K8s: Deploy an OpenSearch cluster in dse-k8s-codfw
Closed, ResolvedPublic

Description

Creating this ticket to:

Event Timeline

bking renamed this task from OpenSEarch on K8s: Deploy an OpenSearch cluster in dse-k8s-codfw to OpenSearch on K8s: Deploy an OpenSearch cluster in dse-k8s-codfw.
bking updated the task description. (Show Details)
bking removed bking as the assignee of this task.Oct 29 2025, 1:55 PM

Change #1200010 had a related patch set uploaded (by Stevemunene; author: Stevemunene):

[operations/puppet@production] Add an opensearch-test-codfw namespace

https://gerrit.wikimedia.org/r/1200010

Change #1200010 merged by Bking:

[operations/puppet@production] Add an opensearch-test-codfw namespace

https://gerrit.wikimedia.org/r/1200010

Gehel added a subscriber: Stevemunene.
bking changed the task status from Open to In Progress.Nov 18 2025, 7:44 PM
bking claimed this task.
bking triaged this task as Medium priority.

Change #1206969 had a related patch set uploaded (by Bking; author: Bking):

[operations/deployment-charts@master] dse-k8s-codfw: set minimum resources for opensearch namespaces

https://gerrit.wikimedia.org/r/1206969

Change #1206973 had a related patch set uploaded (by Bking; author: Bking):

[operations/deployment-charts@master] opensearch on k8s: Add CODFW environment to helmfiles

https://gerrit.wikimedia.org/r/1206973

Change #1206969 merged by Bking:

[operations/deployment-charts@master] dse-k8s-codfw: set minimum resources for opensearch namespaces

https://gerrit.wikimedia.org/r/1206969

Change #1206973 merged by jenkins-bot:

[operations/deployment-charts@master] opensearch on k8s: Add CODFW environment to helmfiles

https://gerrit.wikimedia.org/r/1206973

Update: I was able to successfully deploy in CODFW, although I rolled it back because it was setting off alerts.

This proves that we can deploy the cluster in CODFW. For a real production deploy, we'll need to set watchNamespace to the correct namespaces for the CODFW opensearch-operator helmfile deployment and redeploy admin_ng.

Change #1214562 had a related patch set uploaded (by Bking; author: Bking):

[operations/deployment-charts@master] opensearch-operator: push dummy chart update

https://gerrit.wikimedia.org/r/1214562

Change #1214562 merged by jenkins-bot:

[operations/deployment-charts@master] opensearch-operator: watch the correct namespaces in CODFW

https://gerrit.wikimedia.org/r/1214562

Cookbook cookbooks.sre.k8s.pool-depool-node started by bking@cumin2002 depool for host dse-k8s-worker2003.codfw.wmnet completed:

  • dse-k8s-worker2003.codfw.wmnet (PASS)
    • Host dse-k8s-worker2003.codfw.wmnet depooled from dse-codfw

Cookbook cookbooks.sre.k8s.pool-depool-node started by bking@cumin2002 pool for host dse-k8s-worker2003.codfw.wmnet completed:

  • dse-k8s-worker2003.codfw.wmnet (PASS)
    • Host dse-k8s-worker2003.codfw.wmnet pooled in dse-codfw

Change #1216832 had a related patch set uploaded (by Brouberol; author: Brouberol):

[operations/deployment-charts@master] dse-k8s-codfw: enable pod-to-pod traffic cluster-wide

https://gerrit.wikimedia.org/r/1216832

Change #1216832 merged by Bking:

[operations/deployment-charts@master] dse-k8s-codfw: enable pod-to-pod traffic cluster-wide

https://gerrit.wikimedia.org/r/1216832

We now have working clusters in CODFW, and have for some time!

kube_env opensearch-test dse-k8s-codfw
bking@deploy2002:~$ k get po
NAME                                          READY   STATUS      RESTARTS   AGE
opensearch-test-masters-0                     1/1     Running     0          31d
opensearch-test-masters-1                     1/1     Running     0          31d
opensearch-test-masters-2                     1/1     Running     0          31d
opensearch-test-securityconfig-update-gdmws   0/1     Completed   0          31d

As such, I'm resolving this ticket.