Page MenuHomePhabricator

Toolforge: new k8s: upload internal docker images to our registry
Closed, ResolvedPublic

Description

We have a policy in k8s to only allow docker images from our registry. We need to upload them to the registry.

At least these ones:

  • nginx-ingress
  • custom admission controllers
  • other images used in the kube-system namespace? otherwise we might have issues scaling the cluster

Event Timeline

aborrero triaged this task as Medium priority.Oct 23 2019, 12:22 PM
aborrero created this task.
aborrero moved this task from Inbox to Soon! on the cloud-services-team (Kanban) board.

Mentioned in SAL (#wikimedia-cloud) [2019-10-28T11:47:17Z] <arturo> upload image nginx-ingress-controller v0.25.1 (0439eb3e11f1) to docker registry (T236249)

I had no idea about how to do this. After some investigation, I created some docs on how to do this at https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Docker-registry#Uploading_custom_docker_images

Mentioned in SAL (#wikimedia-cloud) [2019-10-28T11:58:22Z] <arturo> upload image calico/kube-controllers v3.8.0 (df5ff96cd966) to docker registry (T236249)

Mentioned in SAL (#wikimedia-cloud) [2019-10-28T12:01:11Z] <arturo> upload image calico/cni v3.8.0 (539ca36a4c13) to docker registry (T236249)

Mentioned in SAL (#wikimedia-cloud) [2019-10-28T12:03:24Z] <arturo> upload image calico/calico/pod2daemon-flexvol v3.8.0 (f68c8f870a03) to docker registry (T236249)

Mentioned in SAL (#wikimedia-cloud) [2019-10-28T12:04:51Z] <arturo> upload image calico/node v3.8.0 (cd3efa20ff37) to docker registry (T236249)

Mentioned in SAL (#wikimedia-cloud) [2019-10-28T12:18:57Z] <arturo> upload image kube-scheduler v1.15.1 (b0b3c4c404da) to docker registry (T236249)

Mentioned in SAL (#wikimedia-cloud) [2019-10-28T12:20:15Z] <arturo> upload image kube-proxy v1.15.1 (89a062da739d) to docker registry (T236249)

Mentioned in SAL (#wikimedia-cloud) [2019-10-28T12:22:00Z] <arturo> upload image kube-controller-manager v1.15.1 (d75082f1d121) to docker registry (T236249)

Mentioned in SAL (#wikimedia-cloud) [2019-10-28T12:23:25Z] <arturo> upload image kube-apiserver v1.15.1 (68c3eb07bfc3) to docker registry (T236249)

Mentioned in SAL (#wikimedia-cloud) [2019-10-28T12:24:59Z] <arturo> upload image coredns v1.3.1 (eb516548c180) to docker registry (T236249)

Change 546459 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] toolforge: k8s: use docker image from internal registry

https://gerrit.wikimedia.org/r/546459

@aborrero: I have deliberately whitelisted the kube-system namespace for registry purposes. Otherwise, upgrades will be very problematic and broken for kubeadm and calico. We do not need to have those images there.

Kubeadm likely won't get them from there anyway. Ingress images must be in the local registry (with yaml updated to match) or you won't be able to deploy image upgrades--anything outside the kube-system namespace.

Oh, well. Good to know anyway.

Sorry I didn't call that out sooner!

aborrero claimed this task.

No problem! Closing task now, we already have all the images we need in the registry.

Change 546459 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] toolforge: k8s: ingress: use docker image from internal registry

https://gerrit.wikimedia.org/r/546459