Page MenuHomePhabricator

Something is up with the kubeadm component on stretch VMs
Closed, ResolvedPublic

Description

On bastions and several other stretch VMs reply with this kind of stuff on apt update:

E: Failed to fetch http://apt.wikimedia.org/wikimedia/dists/stretch-wikimedia/InRelease  Unable to find expected entry 'thirdparty/kubeadm-k8s-1-19/source/Sources' in Release file (Wrong sources.list entry or malformed file)
E: Some index files failed to download. They have been ignored, or old ones used instead.

Not sure why yet. The bastions do seem to be running the latest-installed kubectl, so this would be a recent issue.

Event Timeline

@aborrero I haven't looked much at what's up with that so far. I can say that deleting the sources file and letting puppet put it back didn't work. This just a quick report of the problem.

If I had to guess, the component exists for Buster only, not on stretch

/me plugs https://apt-browser.toolforge.org/ which lists all suites, components, and their packages in apt.wm.o.

As @Majavah said, thirdparty/kubeadm-k8s-1-19 currently only exists in buster-wikimedia, not stretch.

We need it in stretch for bastions...and it clearly used to be there. :)

As is whatever changed is going to interfere with the upgrade to 1.20 for T280402

Toolsbeta was just upgraded and the bastion cannot upgrade the client. I can just download a binary, but that'd be out of step with the puppet setup. I can try to figure out what changed today.

As far as I'm aware all K8s upgrades I've done only added repositories for buster. I'm not at all sure how they made to the stretch bastions in the first place.

Change 725041 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] aprepo: Import kubeadm to stretch too

https://gerrit.wikimedia.org/r/725041

Change 725041 merged by Bstorm:

[operations/puppet@production] aprepo: Import kubeadm to stretch too

https://gerrit.wikimedia.org/r/725041

Mentioned in SAL (#wikimedia-operations) [2021-09-30T17:42:19Z] <bstorm> updating packages for thirdparty/kubeadm-k8s-1-20 and thirdparty/kubeadm-k8s-1-19 in stretch-wikimedia on apt1001 T292131

Bstorm claimed this task.

All better now! Thanks @Majavah