Change 824161 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/puppet@production] Add a new intermediate CA for kubernetes

https://gerrit.wikimedia.org/r/824161

I have discussed with @jbond and we would like to take this opportunity, if possible, to allow Kubernetes clusters to use the cfssl based PKI instead of Puppet CA certificates.

The first step will require creating a new intermediate CA, since we cannot use the discovery CA for this purpose. We need client authentication, which the discovery CA doesn't allow.

Ahh this ticket already confirms my first question on the CR i.e. the CA is just for the control plane and not the pods.

In T310172#8161105, @BTullis wrote:

I have discussed with @jbond and we would like to take this opportunity, if possible, to allow Kubernetes clusters to use the cfssl based PKI instead of Puppet CA certificates.

The first step will require creating a new intermediate CA, since we cannot use the discovery CA for this purpose. We need client authentication, which the discovery CA doesn't allow.

i think we should also consider if we should have one intermediate per k8 cluster or if we just have a generic Kubernetes CA for all clusters· Personally i would go with the former as it would by design prevent k8 cluster components talking to the wrong cluster but perhaps that is overly cautious (cc @akosiaris @elukey @JMeybohm )

Change 824161 abandoned by Btullis:

[operations/puppet@production] Add a new intermediate CA for kubernetes

Reason:

Decided not to go about it this way right now.

https://gerrit.wikimedia.org/r/824161

Change 824705 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/puppet@production] Add etcd data for dse-k8s kubeserver-api backend selection.

https://gerrit.wikimedia.org/r/824705

Change 824705 merged by Btullis:

[operations/puppet@production] Add etcd data for dse-k8s kubeserver-api backend selection.

https://gerrit.wikimedia.org/r/824705

Change 825348 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/puppet@production] Add an extry for dse-k8s-ctrl to the service catalog

https://gerrit.wikimedia.org/r/825348

Change 825348 merged by Btullis:

[operations/puppet@production] Add an entry for dse-k8s-ctrl to the service catalog

https://gerrit.wikimedia.org/r/825348

I have enabled the kubemaster backend servers for the dse-k8s service with conftool.

btullis@puppetmaster1001:~$ sudo -i confctl select cluster=dse-k8s get
{"dse-k8s-ctrl1001.eqiad.wmnet": {"weight": 0, "pooled": "inactive"}, "tags": "dc=eqiad,cluster=dse-k8s,service=kubemaster"}
{"dse-k8s-ctrl1002.eqiad.wmnet": {"weight": 0, "pooled": "inactive"}, "tags": "dc=eqiad,cluster=dse-k8s,service=kubemaster"}
btullis@puppetmaster1001:~$ sudo -i confctl select 'cluster=dse-k8s,service=kubemaster' set/pooled=yes:weight=1
The selector you chose has selected the following objects:
{"/eqiad/dse-k8s/kubemaster": ["dse-k8s-ctrl1001.eqiad.wmnet", "dse-k8s-ctrl1002.eqiad.wmnet"]}
Ok to continue? [y/N]
confctl>y
eqiad/dse-k8s/kubemaster/dse-k8s-ctrl1001.eqiad.wmnet: pooled changed inactive => yes
eqiad/dse-k8s/kubemaster/dse-k8s-ctrl1001.eqiad.wmnet: weight changed 0 => 1
eqiad/dse-k8s/kubemaster/dse-k8s-ctrl1002.eqiad.wmnet: pooled changed inactive => yes
eqiad/dse-k8s/kubemaster/dse-k8s-ctrl1002.eqiad.wmnet: weight changed 0 => 1
WARNING:conftool.announce:conftool action : set/pooled=yes:weight=1; selector: cluster=dse-k8s,service=kubemaster
btullis@puppetmaster1001:~$ sudo -i confctl select cluster=dse-k8s get
{"dse-k8s-ctrl1001.eqiad.wmnet": {"weight": 1, "pooled": "yes"}, "tags": "dc=eqiad,cluster=dse-k8s,service=kubemaster"}
{"dse-k8s-ctrl1002.eqiad.wmnet": {"weight": 1, "pooled": "yes"}, "tags": "dc=eqiad,cluster=dse-k8s,service=kubemaster"}

Change 825726 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/puppet@production] Enable the LVS realserver profile for dse-k8s-ctrl

https://gerrit.wikimedia.org/r/825726

Change 825726 merged by Btullis:

[operations/puppet@production] Enable the LVS realserver profile for dse-k8s-ctrl

https://gerrit.wikimedia.org/r/825726

Change 826296 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/puppet@production] Configure the load-balancers for dse-k8s-ctrl

https://gerrit.wikimedia.org/r/826296

Change 826296 merged by Btullis:

[operations/puppet@production] Configure the load-balancers for dse-k8s-ctrl

https://gerrit.wikimedia.org/r/826296

I believe that this step of the process is now broadly complete.

I have labelled the master nodes as follows:

root@deploy1002:/srv/deployment-charts/helmfile.d/admin_ng# kubectl label nodes dse-k8s-ctrl1001.eqiad.wmnet node-role.kubernetes.io/master=""
node/dse-k8s-ctrl1001.eqiad.wmnet labeled
root@deploy1002:/srv/deployment-charts/helmfile.d/admin_ng# kubectl label nodes dse-k8s-ctrl1002.eqiad.wmnet node-role.kubernetes.io/master=""
node/dse-k8s-ctrl1002.eqiad.wmnet labeled

Change 828049 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/puppet@production] Add a kublet node_label to each master of the dse-k8s cluster

https://gerrit.wikimedia.org/r/828049

Change 828049 merged by Btullis:

[operations/puppet@production] Add a kublet node_label to each master of the dse-k8s cluster

https://gerrit.wikimedia.org/r/828049