Page MenuHomePhabricator

Setup and verify authentication for Kubernetes
Closed, ResolvedPublic


Using token or basic auth to begin with. Since the users who will be allowed to use this is whitelist based and limited to begin with, this is ok. Anyone without a user account should be denied.

Event Timeline

yuvipanda raised the priority of this task from to Needs Triage.
yuvipanda updated the task description. (Show Details)
yuvipanda added subscribers: Aklapper, yuvipanda.

So I'm passing --token-auth-file and it is pointing to a file in the valid format, but there's no acknowledgement from kubernetes that it even saw this. Unauthenticated calls succeed just as well from kubectl.

Had same problem with the basic auth provider as well.

there is an ABAC policy file restricting the one account I have to only one namespace, but that also seems to have no effect.

After reading through the source for a bit I'm convinced that this is just because I was using the insecure port. I'll verify shortly.

Insecure port is likely reason for this.

You are setting " --insecure-bind-address=<%= @master_ip %> ".
unless master_ip is, that's probably not doing what you want.

@Etune yup, that was the issue. Now at F0909 20:26:59.469630 29907 helpers.go:71] Error in configuration: more than one authentication method found for ; found [token basicAuth], only one is allowed despite the fact that I'm speciying only --token-auth-file.

I had previously used basic auth, and had the same user account there. I wonder if that is 'cached' somewhere (etcd perhaps?) despite the restarts? Doesn't look obvious from the code.

You probably need to edit your .kube/config

Ok, that theory is bunk now. Still trying to figure out why it thinks basica uth is still on.

my local user's .kube/config? doesn't exist...

you probably have a users[i].user.password and a users[i].user.token in that file.

This is client side error, not apiserver side.

Am using:

kubectl --insecure-skip-tls-verify -s  --namespace='lolrrit-wm'  --token=='<password>'   get pods

(and had tried with --username and --user as well)

Are you seeing that fatal log message in the apiserver log file?

Nope in the client, when I try out kubectl.

Ok, so if I set the token in .kube/config and then select it with --user... I get:

yuvipanda@tools-k8s-master-01:~$ kubectl --insecure-skip-tls-verify -s  --namespace='lolrrit-wm' --user='lolrrit-wm'  get pods
error: couldn't read version from server: the server does not allow access to the requested resource

where the ABAC file is:

yuvipanda@tools-k8s-master-01:~$ sudo cat /etc/kube/abac
{"user": "lolrrit-wm", "namespace": "lolrrit-wm"}

So this should techncially work...

I also couldn't really find any documentation on .kube/config

so if I add another line:

{"user": "lolrrit-wm"}

to the ABAC file it works. I now have to check if the namespace filter is available in the verison of kubernetes we're running (v1.1.0-alpha.1)

Ok, so the code definitely exists (and so do the docs) for namespace field ABAC in the version we are running.

Adding just a `{"namespace": "lolrrit-wm"}' doesn't help things

I've cherry-picked a fix to ^ and it's all good now. Ongoing discussions on the issue and pull request associated with it.

Now to figure out the appropriate bits of ABAC required to let kubelet etc do their jobs :)

valhallasw triaged this task as Medium priority.Oct 4 2015, 11:50 AM
valhallasw moved this task from Triage to Backlog on the Toolforge board.
valhallasw added a subscriber: valhallasw.

It's all good now! We have a 'client-infrastructure' account for kubelet / kube-proxy, and 'admin' and 'namespaced' accounts. This is all controlled from hieradata/common.yaml in the secret repository in /var/lib/git/labs/private on tools-puppetmaster-01. Needs backup, however...

yuvipanda claimed this task.