Page MenuHomePhabricator

Configure routing for dse-k8s cluster
Closed, ResolvedPublic5 Estimated Story Points

Event Timeline

Change 826525 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/homer/public@master] Add BGP neighbor data for the new dse-k8s cluster

https://gerrit.wikimedia.org/r/826525

Change 826525 merged by jenkins-bot:

[operations/homer/public@master] Add BGP neighbor data for the new dse-k8s cluster

https://gerrit.wikimedia.org/r/826525

Change 826572 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/homer/public@master] Add btullis to users to allow for router configuration

https://gerrit.wikimedia.org/r/826572

Change 826579 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/homer/public@master] Add BGP neighbor data for the new dse-k8s cluster

https://gerrit.wikimedia.org/r/826579

Change 826579 merged by jenkins-bot:

[operations/homer/public@master] Add BGP neighbor data for the new dse-k8s cluster

https://gerrit.wikimedia.org/r/826579

Pushing out this configuration to all eqiad switches/routers now.

btullis@cumin1001:~$ homer "*eqiad*" commit "T310174 enabling calico for dse-k8s cluster"

Change 827979 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/homer/public@master] Fix policy name discrepancy between dse_k8s and kubedse

https://gerrit.wikimedia.org/r/827979

Change 827979 merged by jenkins-bot:

[operations/homer/public@master] Fix policy name discrepancy between dse_k8s and kubedse

https://gerrit.wikimedia.org/r/827979

BTullis removed a project: Patch-For-Review.

It took one more patch, but these changes to the switches and routers have all now been deployed.
Tentatively moving this ticket to done.

Change 826572 merged by jenkins-bot:

[operations/homer/public@master] Add btullis to users to allow for router configuration

https://gerrit.wikimedia.org/r/826572

Change 829037 had a related patch set uploaded (by Cathal Mooney; author: Cathal Mooney):

[operations/homer/public@master] Change CR template to only include Kubedse in BGP_Switch_in for eqiad

https://gerrit.wikimedia.org/r/829037

Change 829037 merged by jenkins-bot:

[operations/homer/public@master] Change CR template to only include Kubedse in BGP_Switch_in for eqiad

https://gerrit.wikimedia.org/r/829037

Thanks @BTullis, looks ok from my side too. I had to submit the above patch, but don't worry it's really just to do with the circular logic - I would have also failed to catch it. Longer term we probably need to look at that, the "switch_in" policy is probably us being a little too cautious (dates from eqiad expansion initial go-live when we were maybe more cautious about filtering what the switches sent, given it was a new setup).

Anyway thanks for looking at this. I'm hopeful as we move forward we can simplify all this somewhat but for this particular scenario hopefully we are now good!

Mentioned in SAL (#wikimedia-operations) [2022-09-05T09:25:40Z] <btullis> deployed calico to dse-k8s cluster T310174

Thanks @cmooney for the additional patch.
I had to make a small modification to /srv/private/hieradata/common/profile/kubernetes.yaml in the private repo, because I had accidentally omitted the calico user tokens.

Now I've deployed calico and everything seems fine.

root@deploy1002:~# kubectl -n kube-system get deployment,daemonset
NAME                                      READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/calico-kube-controllers   1/1     1            1           82s
deployment.apps/calico-typha              2/2     2            2           82s

NAME                         DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
daemonset.apps/calico-node   10        10        10      10           10          kubernetes.io/os=linux   82s
EChetty claimed this task.