At the moment when we need access an "external domain" like en.wikipedia.org in a Lift Wing's mesh model server we need to explicitly proxy traffic to api-ro.discovery.wmnet. It would be nice to make this proxy action transparent, so that calling en.wikipedia.org inside an isvc would work without any specific mention of api-ro (via WIKI_URL for example).
Istio seems to be capable of doing it, but we never made it work. There are some things that I discovered today that could help, writing them down as reference for the future.
The idea would be to enable the Istio DNS proxy resolution, for example testing it on a specific isvc/pod via these annotations:
proxy.istio.io/config: |
proxyMetadata:
ISTIO_META_DNS_CAPTURE: "true"
ISTIO_META_DNS_AUTO_ALLOCATE: "true"Then we could add something like the following to the mesh config:
apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: external-svc-proxy
spec:
hosts:
- wikipedia.org
- "*.wikipedia.org"
location: MESH_EXTERNAL
ports:
- number: 80
name: http
protocol: HTTP
resolution: NONEWhen DNS proxying is enabled, all DNS traffic generated by a pod goes through a special Istio resolver (within the pod) that is capable of adding extra rules in place, for example from the Service Entry config. In the above example we specify that a DNS resolution for all the wikipedia domains should not be performed, leaving to Virtual Services and Destination rules the duty of handling that traffic. In our current config (Dec 2023) the VS and DR configs are already supporting this, but due to the DNS resolution being unaware of Service Entries, the code tries to resolve Wikipedia's domains ending up in trying to connect to external domains (sometimes they may fail due to IPv6 resolution etc..).
Why don't we enable DNS proxy now? Due to the following:
The Kserve's storage-initializer container doesn't work anymore if we enable it, but it may if we move its Docker image to uid 1337.