The UI for wdqs (e.g. query.wikidata.org) lives on miscweb. We'll want to create slightly different UIs for the two types of graph hosts.
Note: I'm not sure if this should just be considered part of T364364, but for now I'll keep a separate task.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T335067 Epic: Wikidata Query Service stabilization | |||
| Open | None | T337013 [Epic] Splitting the graph in WDQS | |||
| Open | None | T364363 [Epic] Productionize federated wdqs graph-split endpoints | |||
| Resolved | Stevemunene | T364367 Create dedicated UIs for wdqs graph split endpoints |
Change #1046121 had a related patch set uploaded (by Stevemunene; author: Stevemunene):
[operations/puppet@production] wdqs: microsites for wdqs graph split
Hello! We are transitioning from the legacy Miscweb setup to Kubernetes. Most services have already been migrated (see main tracking task T300171). However, the migration for the query service is still pending (T350793).
I would like to add a brief disclaimer: all additions to the existing query service must be adapted for the Kubernetes environment as well. Depending on the architecture, this might result in duplicate work. If you are simply mapping multiple vhost/DNS names to the same service, this shouldn't be a big problem, as we can map multiple names in the Kubernetes ingress. However, if you are creating dedicated services, this will require additional work on the Kubernetes side as well.
The migration to k8s needs some coordination with WMDE. Let's move forward already on this work and back port as needed.
Change #1049238 had a related patch set uploaded (by Stevemunene; author: Stevemunene):
[wikidata/query/gui-deploy@production] wdqs: query main and query scholarly microsites
Change #1046121 merged by Dzahn:
[operations/puppet@production] wdqs: microsites for wdqs graph split
Change #1052826 had a related patch set uploaded (by Dzahn; author: Dzahn):
[operations/puppet@production] Revert "wdqs: microsites for wdqs graph split"
Change #1052826 merged by Dzahn:
[operations/puppet@production] Revert "wdqs: microsites for wdqs graph split"
Change #1053030 had a related patch set uploaded (by Dzahn; author: Dzahn):
[operations/dns@master] create query-main.wikidata.org and query-scholarly.wikidata.org
Change #1053030 abandoned by Dzahn:
[operations/dns@master] create query-main.wikidata.org and query-scholarly.wikidata.org
Reason:
duplicate of https://gerrit.wikimedia.org/r/c/operations/dns/+/1042160
Hi @Stevemunene
so.. I merged your change to add the apache sites yesterday but then reverted because we got a bunch of monitoring alerts.
Now was looking at how to fix that before re-reverting your change.
I didn't realize at first you did add them to DNS already meanwhile, hence my abandoned change above to add them.
So the next thing we would need here is a config change in the traffic layer, ATS, to map URLs to the new names.
Looking at the existing config I see it's not just one map/replacement like for most services but a couple different ones for different URLs under the subdomains:
- type: map target: http://query-main-experimental.wikidata.org/bigdata/ldf replacement: https://wdqs1024.eqiad.wmnet/bigdata/ldf - type: map target: http://query-main-experimental.wikidata.org/sparql replacement: https://wdqs1024.eqiad.wmnet/sparql - type: map target: http://query-main-experimental.wikidata.org/bigdata replacement: https://wdqs1024.eqiad.wmnet/sparql - type: map target: http://query-main-experimental.wikidata.org replacement: https://webserver-misc-eqiad.discovery.wmnet - type: map target: http://query-scholarly-experimental.wikidata.org/bigdata/ldf replacement: https://wdqs1023.eqiad.wmnet/bigdata/ldf - type: map target: http://query-scholarly-experimental.wikidata.org/sparql replacement: https://wdqs1023.eqiad.wmnet/sparql - type: map target: http://query-scholarly-experimental.wikidata.org/bigdata replacement: https://wdqs1023.eqiad.wmnet/sparql - type: map target: http://query-scholarly-experimental.wikidata.org replacement: https://webserver-misc-eqiad.discovery.wmnet
How should this look for the new "non-experimental" sites? Should they link to the same or other "replacement" URLs?
P.S. This is hieradata/common/profile/trafficserver/backend.yaml in the puppet repo. I am happy to make the patch or you can feel free to upload it and add me as reviewer.. up to you.
Change #1053756 had a related patch set uploaded (by Ryan Kemper; author: Ryan Kemper):
[operations/puppet@production] wdqs graph split: routing for new services
Change #1053765 had a related patch set uploaded (by Ryan Kemper; author: Ryan Kemper):
[operations/puppet@production] wdqs graph split: routing for wdqs backends
Change #1053773 had a related patch set uploaded (by Dzahn; author: Dzahn):
[operations/puppet@production] miscweb: add query-main and query-scholarly.wikidata.org to certs
Change #1053773 merged by Dzahn:
[operations/puppet@production] miscweb: add query-main and query-scholarly.wikidata.org to certs
merged and deployed https://gerrit.wikimedia.org/r/c/operations/puppet/+/1053773
This was needed to add the new sites to the envoy certs on the backends.
Without the traffic servers wouldn't be able to talk to the miscweb machines when requesting the new URLs.
Here are the update certs with the new names:
root@miscweb1003:/# openssl x509 -noout -ext subjectAltName -in /etc/envoy/ssl/discovery__webserver-misc-apps_discovery_wmnet_server.chained.pem
X509v3 Subject Alternative Name:
DNS:os-reports.wikimedia.org, DNS:commons-query.wikimedia.org, DNS:query-full-experimental.wikidata.org, DNS:query-main-experimental.wikidata.org, DNS:query-scholarly-experimental.wikidata.org, DNS:query.wikidata.org, DNS:query-main.wikidata.org, DNS:query-scholarly.wikidata.org, DNS:webserver-misc-apps.discovery.wmnetChange #1053756 merged by Dzahn:
[operations/puppet@production] wdqs graph split: route / to miscweb microsite
New URLs/microsites are up:
query-main.wikidata.org
query-scholarly.wikidata.org
This work is done. We still have the patch open for the routing to the backend hosts for the /sparql, /bigdata, /ldf that's associated to this ticket. Technically that's not a UI patch though. I'm marking this resolved.
Change #1049238 merged by Ryan Kemper:
[wikidata/query/gui-deploy@production] wdqs: query main and query scholarly microsites
Change #1057889 had a related patch set uploaded (by Ryan Kemper; author: DCausse):
[wikidata/query/gui-deploy@production] wdqs-main: set title to "WDQS main graph"
Change #1057889 merged by Ryan Kemper:
[wikidata/query/gui-deploy@production] wdqs-main: set title to "WDQS main graph"
Change #1053765 merged by Ryan Kemper:
[operations/puppet@production] wdqs graph split: routing for wdqs backends
Change #1066812 had a related patch set uploaded (by Ryan Kemper; author: Ryan Kemper):
[operations/puppet@production] wdqs graph split: routing for wdqs backends
Change #1066812 merged by Ryan Kemper:
[operations/puppet@production] wdqs graph split: routing for wdqs backends
Change #1078664 had a related patch set uploaded (by Jelto; author: Jelto):
[operations/deployment-charts@master] wikidata-query-gui: remove experimental endpoints
Change #1078664 merged by jenkins-bot:
[operations/deployment-charts@master] wikidata-query-gui: remove experimental endpoints