How to customize WDQS endpoint
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	lucamauri
	Jan 13 2025, 9:20 AM

Description

Preamble

As explained in the the documentation, the WDQS frontend endpoint is reachable at https://query.my-wikibase.org/proxy/wdqs/bigdata/namespace/wdq/sparql path.

The endpoint of Wikidata is reachable at a much shorter URL – https://query.wikidata.org/sparql – and the same is valid for the majority of publicly-available Wikibase on the internet; please see values of P5305 for examples.

Observation

I see the relevant path is referenced in the custom-config.json:

{
  "api": {
    "sparql": {
      "uri": "/proxy/wdqs/bigdata/namespace/wdq/sparql"
    },
    …

and then checked in docker-compose.yml:

…
healthcheck:
  test: curl --silent --fail localhost:9999/bigdata/namespace/wdq/sparql
  interval: 10s
  start_period: 2m
…

but I suppose this is only a small part of the big picture.

Question

How can we configure Wikibase Suite to customize our own frontend endpoint?
I thnk what I observed above is a couple of references to the configuration, not the configuration itself. Am I right?
If yes, where should we act in the WBS configuration to customize the endpoint URL?

Thanks in advance

Related Objects
Search...

Status	Subtype	Assigned	Task
Resolved	Spike	lojo_wmde	T377181 Release engineering and process follow-up to single-track release decision
Resolved		• roti_WMDE	T383022 Major Release: MediaWiki 1.43 based
Resolved		• roti_WMDE	T383542 How to customize WDQS endpoint

Event Timeline

lucamauri created this task.Jan 13 2025, 9:20 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 13 2025, 9:20 AM

If yes, where should we act in the WBS configuration to customize the endpoint URL?

I am answering my own question with a method I came up on myself after asking the question. I hope someone from the Wikibase Suite Team can validate it or propose something better.

I copied the file /templates/default.conf from the wdqs-frontend container into a new file named wdqs-frontend-default.conf, I added a location to the file to intercept the /sparql URL and send it to the wdqs-proxy container:

…
    location /sparql {
        rewrite /sparql /bigdata/namespace/wdq/sparql break;
        proxy_pass http://$WDQS_HOST:80;
    }
…

then I modified the docker-compose.yml to replace the original file in wdqs-frontend upon container creation:

…
   volumes:
      - ./config/wdqs-frontend-default.conf:/templates/default.conf
…

Thanks in advance for further comments or suggestions on this topic.

Olea awarded a token.Feb 6 2025, 10:37 AM

• roti_WMDE changed the task status from Open to In Progress.Feb 6 2025, 3:20 PM

• roti_WMDE claimed this task.

• roti_WMDE edited projects, added Wikibase Suite Team (Sprint-∞); removed Wikibase Suite Team.

• roti_WMDE moved this task from Sprint Backlog to Doing on the Wikibase Suite Team (Sprint-∞) board.

Hi @lucamauri,

Thanks a lot your message and documenting your findings. This is indeed the way to do it. I am now suggesting a very similar change for wikibase suite upstream: https://github.com/wmde/wikibase-release-pipeline/pull/832/commits/74465d7144a17e4c1b5dbe39979622845b12bf71

We still have wdqs-proxy in the stack, which makes the whole picture a bit more complex. We are looking into completely removing it for simplicity.

https://github.com/wmde/wikibase-release-pipeline/pull/767
https://github.com/wmde/wikibase-release-pipeline/pull/833

• roti_WMDE mentioned this in T365115: Use central proxy as WDQS proxy as well, remove WDQS proxy.Feb 7 2025, 10:24 AM

• roti_WMDE moved this task from Doing to In Review on the Wikibase Suite Team (Sprint-∞) board.Feb 18 2025, 9:49 AM

• roti_WMDE added a parent task: T383022: Major Release: MediaWiki 1.43 based.Mar 4 2025, 5:10 AM

• roti_WMDE closed this task as Resolved.Mar 10 2025, 8:27 AM

• roti_WMDE moved this task from In Review to Done on the Wikibase Suite Team (Sprint-∞) board.

• roti_WMDE moved this task from Sprint-∞ to Archive (closed) on the Wikibase Suite Team board.Mar 18 2025, 2:11 PM

• roti_WMDE edited projects, added Wikibase Suite Team; removed Wikibase Suite Team (Sprint-∞).