Page MenuHomePhabricator

Toolforge kubernetes kube-proxy broken by recent production changes
Closed, ResolvedPublic


Toolforge is not ready for full upgrade of the Kubernetes cluster, and we need the old cluster live while we migrate. has introduced an error that prevents any new web services being launched:
Dec 02 18:21:11 tools-proxy-05 kube-proxy[30558]: unknown flag: --metrics-bind-address

Event Timeline

Bstorm triaged this task as Unbreak Now! priority.Dec 2 2019, 10:51 PM
Bstorm created this task.
Restricted Application added subscribers: Liuxinyu970226, Aklapper. · View Herald TranscriptDec 2 2019, 10:51 PM

Change 554178 had a related patch set uploaded (by Bstorm; owner: Bstorm):
[operations/puppet@production] kube-proxy: Fix toolforge kube-proxy

Change 554178 merged by Bstorm:
[operations/puppet@production] kube-proxy: Fix toolforge kube-proxy

Checked that my patch had no effect on production hosts (cc @akosiaris )

So at this point, the proxy seems to work right, but the worker nodes are unable to disable this flag still.

Fixed on the nodes now.

Bstorm closed this task as Resolved.Dec 3 2019, 12:42 AM
Joutbis removed a subscriber: Joutbis.Dec 3 2019, 1:35 AM

Thanks @Bstorm

My two tools came up nicely yesterday evening/night.