When https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/984819 is deployed, we will get a kubernetes Service and Endpoints resource for each external service "cluster" we need to configure egress to.
The next step becomes making it easy to define a Calico NetworkPolicy with egress rules pointing at such Service resources, to authorize egress traffic to the associated IPs.
We currently achieve with the following data structure:
# Add here the list of kafka-clusters (by name) that the service will need to reach. kafka: allowed_clusters: [] # Add here the list of zookeeper-clusters (by name) that the service will need to reach. zookeeper: allowed_clusters: []
An example of this would be
kafka: allowed_clusters: - logging-eqiad - logging-codfw
This data would then generate the following resource:
apiVersion: crd.projectcalico.org/v1 kind: NetworkPolicy metadata: name: {{ template "base.meta.name" . }}-egress-external-services namespace: {{ .Release.Namespace }} spec: selector: name == '{{ template "base.meta.name" . }}' types: - Egress egress: - action: Allow destination: services: name: logging-eqiad namespace: kafka - action: Allow destination: services: name: logging-codfw namespace: kafka
As the kafka namespace already contains the logging-eqiad/codfw Services, managed by the external-services chart, the generation of such template would be pretty trivial.
The question becomes: how can we guarantee a smooth migration for all apps?
My thinking is: we don't want to deprecate the current networkpolicy.yaml.skel file, as it is useful to be able to generate ingress policies, as well as custom egress policies as well. We'd probably want to remove this block from it:
egress: {{- include "base.networkpolicy.egress-basic" .Values }} - {{- include "base.networkpolicy.egress.kafka" . | indent 4 }} - {{- include "base.networkpolicy.egress.mariadb" . | indent 4 }} - {{- include "base.networkpolicy.egress.zookeeper" . | indent 4 }}
and define a base.networkpolicy.egress-external-services template in modules/base/networkpolicy_1.3.0.tpl, that would loop over all allowed egresses and create the associated Calico NetworkPolicy resources.
Note: I wonder if we should refactor the
kafka: allowed_clusters: [] zookeeper: allowed_clusters: []
structure into something like:
allowed_clusters: kafka: [] zookeeper: []
to make the definition of allowed egresses more centralized?