Page MenuHomePhabricator

Allow unauthenticated access to the dse-k8s OIDC discovery and JWKS endpoints
Closed, ResolvedPublic

Description

Summary

The RADOS Gateway checks workload tokens offline: it fetches the Kubernetes API server's OIDC discovery document and token-verification public keys (the JWKS) once, then validates token signatures itself without ever calling the Kubernetes API per request. Those two endpoints currently require authentication, so this task opens them to unauthenticated readers on the dse-k8s clusters only. They expose nothing sensitive: only public keys and standard discovery metadata.

Technical notes

The fragment below is an illustrative starting point from the implementation notes accompanying the security model.

  • The change is a single RBAC binding, added to the rbac-rules release's resources in helmfile.d/admin_ng/helmfile_rbac.yaml in operations/deployment-charts, gated per cluster like the existing conditionals:
{{- if hasKey .Environment.Values "expose_oidc_discovery" }}
## Allow unauthenticated reads of the OIDC discovery + JWKS endpoints so Ceph RGW
## can verify projected ServiceAccount tokens. Exposes only public signing keys and
## non-sensitive metadata; dse-k8s only.
- apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRoleBinding
  metadata:
    name: service-account-issuer-discovery-unauthenticated
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: system:service-account-issuer-discovery
  subjects:
    - apiGroup: rbac.authorization.k8s.io
      kind: Group
      name: system:unauthenticated
{{- end }}
  • The gate is set once for both dse-k8s clusters, in helmfile.d/admin_ng/values/dse-k8s.yaml, leaving wikikube and every other cluster untouched:
expose_oidc_discovery: true
  • Prerequisite to confirm: the API server must permit anonymous requests (the Kubernetes default, not overridden in the manifests checked); if anonymous auth were ever disabled, Kubernetes 1.31's AuthenticationConfiguration can re-enable it narrowly for just the discovery paths.
  • The RADOS Gateway hosts also need network access to the API server address, which may need a firewall allowance.
  • Apply in codfw first, eqiad when the validation promotes.

Acceptance criteria

  • From a RADOS Gateway host in codfw, the discovery document and JWKS can be fetched without credentials.
  • No cluster other than dse-k8s is affected.

Event Timeline

BTullis triaged this task as High priority.

Change #1344034 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/deployment-charts@master] dse-k8s: allow unauthenticated reads of the OIDC discovery endpoints

https://gerrit.wikimedia.org/r/1344034

Change #1344034 merged by jenkins-bot:

[operations/deployment-charts@master] dse-k8s: allow unauthenticated reads of the OIDC discovery endpoints

https://gerrit.wikimedia.org/r/1344034

I am content that this now works and can be queries from a Ceph server, where radogw runs and will need it.

btullis@cephosd2001:~$ curl -s https://dse-k8s-ctrl.svc.codfw.wmnet:6443/.well-known/openid-configuration|jq .
{
  "issuer": "https://dse-k8s-ctrl.svc.codfw.wmnet:6443",
  "jwks_uri": "https://10.192.48.13:6443/openid/v1/jwks",
  "response_types_supported": [
    "id_token"
  ],
  "subject_types_supported": [
    "public"
  ],
  "id_token_signing_alg_values_supported": [
    "ES256"
  ]
}
btullis@cephosd2001:~$ curl -s https://dse-k8s-ctrl.svc.codfw.wmnet:6443/openid/v1/jwks|jq .
{
  "keys": [
    {
      "use": "sig",
      "kty": "EC",
      "kid": "3GLiiqgwc62G02D1H7xGsRsHAzcMPFb3eHLQH0piTxY",
      "crv": "P-256",
      "alg": "ES256",
      "x": "BbygNsZg3np84nfGn8I4J1Bz-MKhOpnaKKCzibibYGE",
      "y": "MKefuUBFZC5b1FEB2uyg2OOuGqhxRsjvdVWqdv9U5j4"
    },
    {
      "use": "sig",
      "kty": "EC",
      "kid": "3GLiiqgwc62G02D1H7xGsRsHAzcMPFb3eHLQH0piTxY",
      "crv": "P-256",
      "alg": "ES256",
      "x": "BbygNsZg3np84nfGn8I4J1Bz-MKhOpnaKKCzibibYGE",
      "y": "MKefuUBFZC5b1FEB2uyg2OOuGqhxRsjvdVWqdv9U5j4"
    },
    {
      "use": "sig",
      "kty": "EC",
      "kid": "3GLiiqgwc62G02D1H7xGsRsHAzcMPFb3eHLQH0piTxY",
      "crv": "P-256",
      "alg": "ES256",
      "x": "BbygNsZg3np84nfGn8I4J1Bz-MKhOpnaKKCzibibYGE",
      "y": "MKefuUBFZC5b1FEB2uyg2OOuGqhxRsjvdVWqdv9U5j4"
    },
    {
      "use": "sig",
      "kty": "EC",
      "kid": "a2wgmI0zzcHHZRhPfm_kgVBB2C0_L3T1DC4SKz4l9rI",
      "crv": "P-256",
      "alg": "ES256",
      "x": "OLx8J6FvZ8mG8aWy8PZHx3bwsNYhZqzEjJR7-JSt-Ts",
      "y": "alUXldRQQy9kKxVN4mJqjNXco9JoX3alOwG2c5U9j80"
    },
    {
      "use": "sig",
      "kty": "EC",
      "kid": "a2wgmI0zzcHHZRhPfm_kgVBB2C0_L3T1DC4SKz4l9rI",
      "crv": "P-256",
      "alg": "ES256",
      "x": "OLx8J6FvZ8mG8aWy8PZHx3bwsNYhZqzEjJR7-JSt-Ts",
      "y": "alUXldRQQy9kKxVN4mJqjNXco9JoX3alOwG2c5U9j80"
    }
  ]
}