Page MenuHomePhabricator

k8s 1.23 magnum template for PAWS
Closed, ResolvedPublic

Description

Pull down newer image (Tested in codfw1dev in T325538) (actually don't do this part, coredns doesn't work on this image):

wget https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/37.20221127.3.0/x86_64/fedora-coreos-37.20221127.3.0-openstack.x86_64.qcow2.xz
xz --decompress fedora-coreos-37.20221127.3.0-openstack.x86_64.qcow2.xz
openstack image create magnum-fcos-37-20221127 --file=fedora-coreos-37.20221127.3.0-openstack.x86_64.qcow2 --disk-format=qcow2 --container-format=bare --property os_distro='fedora-coreos' --public
openstack image set --property visibility=shared --project testlabs magnum-fcos-37-20221127
openstack image set --activate magnum-fcos-37-20221127
openstack coe cluster template create paws-k8s23 \
--image Fedora-CoreOS-34 \
--external-network wan-transport-eqiad \
--fixed-network lan-flat-cloudinstances2b \
--fixed-subnet cloud-instances2-b-eqiad \
--dns-nameserver 8.8.8.8 \
--network-driver flannel \
--docker-storage-driver overlay2 \
--docker-volume-size 50 \
--master-flavor g3.cores2.ram4.disk20 \
--flavor g3.cores8.ram32.disk20 \
--coe kubernetes \
--labels kube_tag=v1.23.15-rancher1-linux-amd64,hyperkube_prefix=docker.io/rancher/,cloud_provider_enabled=true \
--floating-ip-disabled

For codfw1dev:

openstack coe cluster template create paws-dev-k8s23-34 \
--image Fedora-CoreOS-34 \
--external-network wan-transport-codfw \
--fixed-subnet cloud-instances2-b-codfw \
--fixed-network lan-flat-cloudinstances2b \
--dns-nameserver 8.8.8.8 \
--network-driver flannel \
--docker-storage-driver overlay2 \
--docker-volume-size 30 \
--master-flavor g2.cores1.ram2.disk20 \
--flavor g2.cores1.ram2.disk20 \
--coe kubernetes \
--labels kube_tag=v1.23.15-rancher1-linux-amd64,hyperkube_prefix=docker.io/rancher/,cloud_provider_enabled=true \
--floating-ip-disabled

Event Timeline

Changed image name in codfw1dev as original was in use, and attached to a magnum cluster that cannot be deleted.

openstack coe cluster template create paws-dev-k8s23 \
--image magnum-fcos-37-20221127-2 \
--external-network wan-transport-codfw \
--fixed-subnet cloud-instances2-b-codfw \
--fixed-network lan-flat-cloudinstances2b \
--dns-nameserver 8.8.8.8 \
--network-driver flannel \
--docker-storage-driver overlay2 \
--docker-volume-size 30 \
--master-flavor g2.cores1.ram2.disk20 \
--flavor g2.cores1.ram2.disk20 \
--coe kubernetes \
--labels kube_tag=v1.23.15-rancher1-linux-amd64,hyperkube_prefix=docker.io/rancher/,cloud_provider_enabled=true \
--floating-ip-disabled

coredns fails with the above deployment.

$ kubectl logs -n kube-system pod/coredns-56448757b9-kzjdf
.:53
[INFO] plugin/reload: Running configuration MD5 = 58ca7856bba97d0fdab33b000518a2c9
CoreDNS-1.6.6
linux/amd64, go1.13.5, 6a7a75e
[FATAL] plugin/loop: Loop (127.0.0.1:52285 -> :53) detected for zone ".", see https://coredns.io/plugins/loop#troubleshooting. Query: "HINFO 789671922418294263.2416897044245653832."

Seems to be working with the Fedora-CoreOS-34 image. So we, presumably, cannot upgrade fcos to 37 yet.

rook changed the task status from Open to Stalled.Feb 1 2023, 2:27 PM

Created in codfw1dev, eqiad1 stalled on T328560

This did work, perhaps a bad character in the command?

openstack coe cluster template create test-remove3 --image magnum-fedora-coreos-34 --external-network wan-transport-eqiad --fixed-network lan-flat-cloudinstances2b --fixed-subnet cloud-instances2-b-eqiad --dns-nameserver 8.8.8.8 --network-driver flannel --docker-storage-driver overlay2 --docker-volume-size 80 --master-flavor g3.cores2.ram4.disk20 --flavor g3.cores8.ram32.disk20 --coe kubernetes --labels kube_tag=v1.23.15-rancher1-linux-amd64,hyperkube_prefix=docker.io/rancher/,cloud_provider_enabled=true --floating-ip-disabled

Settled on:

openstack coe cluster template create paws-k8s23 --image magnum-fedora-coreos-34  --external-network wan-transport-eqiad --fixed-network lan-flat-cloudinstances2b --fixed-subnet cloud-instances2-b-eqiad --dns-nameserver 8.8.8.8 --network-driver flannel --docker-storage-driver overlay2 --docker-volume-size 80 --master-flavor g3.cores2.ram4.disk20 --flavor g3.cores8.ram32.disk20 --coe kubernetes --labels kube_tag=v1.23.15-rancher1-linux-amd64,hyperkube_prefix=docker.io/rancher/,cloud_provider_enabled=true --floating-ip-disabled

for production