Page MenuHomePhabricator

Toolforge ingress: decide on how ingress configuration objects will be managed
Closed, ResolvedPublic

Description

We have several options for doing the ingress object configuration in kubernetes. We can namespace the ingress objects and allow users to manage them via the API (mostly via the webservice command) or maintain the config in a daemon under our control.

Open question. How do we prevent tool2 maintainer from adding this config?:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: tool2-ingress
  namespace: tool2
spec:
  rules:
  - host: tool1.toolsbeta.wmflabs.org    <---- wrong!
    http:
      paths:
      - backend:
          serviceName: tool2-svc
          servicePort: 8081

It seems we have 3 options:

  • a daemon detecting which tools are online and generating the ingress config automagically. The webservice command does not generate the ingress config. Management of ingress objects in the API is forbidden for end users.
  • a custom admission controller to enforce correct ingress config, and have the webservice command generate it. The API allows users to manage ingress objects, because we are enforcing a valid config in the API.
  • some mixed thing using a CRD.

Details

Related Gerrit Patches:
cloud/toolforge/ingress-admission-controller : masteringress-admission: Complete pieces for delivery
cloud/toolforge/ingress-admission-controller : masteringress: adding initial files as they come together

Event Timeline

aborrero triaged this task as High priority.Sep 30 2019, 3:33 PM
aborrero created this task.
aborrero moved this task from Inbox to Important on the cloud-services-team (Kanban) board.
This comment was removed by bd808.

We had a team meeting on 2019-10-10 and we decided to try the custom admission controller to enforce correctness in ingress objects.

@Bstorm already has some code bootstrapped to handle this, and she will follow-up.

Bstorm claimed this task.Oct 11 2019, 10:36 PM

Requested the repo cloud/toolforge/ingress-admission-controller after getting the cloud and toolforge placed above that level.

Change 543727 had a related patch set uploaded (by Bstorm; owner: Bstorm):
[cloud/toolforge/ingress-admission-controller@master] ingress: adding initial files as they come together

https://gerrit.wikimedia.org/r/543727

Change 543727 merged by Bstorm:
[cloud/toolforge/ingress-admission-controller@master] ingress: adding initial files as they come together

https://gerrit.wikimedia.org/r/543727

Change 545116 had a related patch set uploaded (by Bstorm; owner: Bstorm):
[cloud/toolforge/ingress-admission-controller@master] ingress-admission: Complete pieces for delivery

https://gerrit.wikimedia.org/r/545116

Change 545116 merged by Bstorm:
[cloud/toolforge/ingress-admission-controller@master] ingress-admission: Complete pieces for delivery

https://gerrit.wikimedia.org/r/545116

Bstorm closed this task as Resolved.Oct 23 2019, 3:19 PM

The image needs upload and all that, but this is ready with some minor touches tracked in other tickets.

Mentioned in SAL (#wikimedia-cloud) [2019-10-25T23:41:32Z] <bstorm_> Deployed custom webhook controllers for registry and ingress checking to toolsbeta-test kubernetes cluster T215531 T215678 T234231