As discussed in this slack thread, the DSE apps currently do not possess a discovery DNS record, meaning we only use the datacenter-local svc record, such as superset.svc.eqiad.wmnet. In the event that we deploy the analytics/data-products stack in a multi-DC fashion, it'd be easier to have everything point to a discovery DNS record, and let it resolve to the ingress of the k8s cluster local to the DC.
Description
Description
Details
Details
Customize query in gerrit
| Title | Reference | Author | Source Branch | Dest Branch | |
|---|---|---|---|---|---|
| datahub: rely on the datahub GMS discovery URL | repos/data-engineering/airflow-dags!736 | brouberol | T367768-discovery-hostnames | main |
Event Timeline
Comment Actions
Change #1046699 had a related patch set uploaded (by Brouberol; author: Brouberol):
[operations/dns@master] dse-k8s: setup a discovery record for all deployed applications
Comment Actions
Change #1046699 merged by Brouberol:
[operations/dns@master] dse-k8s: setup a discovery record for all deployed applications
Comment Actions
brouberol@cumin2002:~$ for svc in "datahub-frontend" "datahub-frontend-next" "datahub-gms" "datahub-gms-next" "datasets-config" "datasets-config-next" "mpic" "mpic-next" "superset" "superset-next"; do host ${svc}.discovery.wmnet; done
datahub-frontend.discovery.wmnet is an alias for k8s-ingress-dse.discovery.wmnet.
k8s-ingress-dse.discovery.wmnet has address 10.2.2.91
datahub-frontend-next.discovery.wmnet is an alias for k8s-ingress-dse.discovery.wmnet.
k8s-ingress-dse.discovery.wmnet has address 10.2.2.91
datahub-gms.discovery.wmnet is an alias for k8s-ingress-dse.discovery.wmnet.
k8s-ingress-dse.discovery.wmnet has address 10.2.2.91
datahub-gms-next.discovery.wmnet is an alias for k8s-ingress-dse.discovery.wmnet.
k8s-ingress-dse.discovery.wmnet has address 10.2.2.91
datasets-config.discovery.wmnet is an alias for k8s-ingress-dse.discovery.wmnet.
k8s-ingress-dse.discovery.wmnet has address 10.2.2.91
datasets-config-next.discovery.wmnet is an alias for k8s-ingress-dse.discovery.wmnet.
k8s-ingress-dse.discovery.wmnet has address 10.2.2.91
mpic.discovery.wmnet is an alias for k8s-ingress-dse.discovery.wmnet.
k8s-ingress-dse.discovery.wmnet has address 10.2.2.91
mpic-next.discovery.wmnet is an alias for k8s-ingress-dse.discovery.wmnet.
k8s-ingress-dse.discovery.wmnet has address 10.2.2.91
superset.discovery.wmnet is an alias for k8s-ingress-dse.discovery.wmnet.
k8s-ingress-dse.discovery.wmnet has address 10.2.2.91
superset-next.discovery.wmnet is an alias for k8s-ingress-dse.discovery.wmnet.
k8s-ingress-dse.discovery.wmnet has address 10.2.2.91Both ${service_name}.svc.eqiad.wmnet and ${service_name}.discovery.wmnet hostnames work. Feel free to use the discovery one for private requests to any DSE service.
brouberol@cumin2002:~$ curl https://superset-next.svc.eqiad.wmnet:30443 <!doctype html> <html lang=en> <title>Redirecting...</title> <h1>Redirecting...</h1> <p>You should be redirected automatically to the target URL: <a href="/superset/welcome/">/superset/welcome/</a>. If not, click the link. brouberol@cumin2002:~$ curl https://superset-next.discovery.wmnet:30443 <!doctype html> <html lang=en> <title>Redirecting...</title> <h1>Redirecting...</h1> <p>You should be redirected automatically to the target URL: <a href="/superset/welcome/">/superset/welcome/</a>. If not, click the link.
Comment Actions
Change #1047040 had a related patch set uploaded (by Brouberol; author: Brouberol):
[operations/puppet@production] ATS: replace service by discovery record for datahub-next
Comment Actions
Change #1047041 had a related patch set uploaded (by Brouberol; author: Brouberol):
[operations/puppet@production] ATS: replace service by discovery record for all DSE services
Comment Actions
Change #1047040 merged by Brouberol:
[operations/puppet@production] ATS: replace service by discovery record for datahub-next
Comment Actions
Change #1047041 merged by Brouberol:
[operations/puppet@production] ATS: replace service by discovery record for all DSE services
Comment Actions
brouberol opened https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/736
datahub: rely on the datahub GMS discovery URL
Comment Actions
brouberol merged https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/736
datahub: rely on the datahub GMS discovery URL