Page MenuHomePhabricator

Odd error when creating a new session on the new PAWS cluster occasionally
Closed, ResolvedPublic

Description

Every now and then (especially after upgrades) I see an error like:

020-07-24 19:29:42+00:00 [Warning] error killing pod: failed to "KillPodSandbox" for "0a6f9721-b383-49be-9229-52fca881189d" with KillPodSandboxError: "rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \"jupyter--42-53torm-5f-28-57-4d-46-29_prod\" network: WorkloadEndpoint.crd.projectcalico.org \"WorkloadEndpoint(prod/paws--k8s--worker--5-k8s-jupyter----42--53torm--5f--28--57--4d--46--29-eth0)\" not found"

Everything recovers and it keeps moving. However, this could be a race-condition/bug in calico or similar. It is typical that the solutions involve upgrading k8s CNI, calico or similar. It isn't a fatal error, but it is ugly to see and then have to wait for it to figure things out.

Event Timeline

Bstorm created this task.

I'm not seeing this error, though am not exactly sure what was giving it. If it was calico we aren't using that at present so it may well have resolved itself in that case. I'm going to close this but please re-open if this is seen again.