Federated queries with WDQS failing in Wikibase Cloud instances
Closed, ResolvedPublicBUG REPORT
Actions

Assigned To

Authored By

	Epidosis
	Wed, Apr 15, 8:11 AM

Description

Steps to replicate the issue (include links if applicable):

enter the SPARQL endpoint of any Wikibase Cloud instance
execute a federated query with WDQS (e.g. https://tinyurl.com/23gsjs3b)

What happens?:

the query immediately fails with this message: Unknown error: Service URI https://query.wikidata.org/sparql is not allowed

What should have happened instead?:

the query should work normally

Software version (on Special:Version page; skip for WMF-hosted wikis like Wikipedia):

Other information (browser name/version, screenshots, etc.):

In https://github.com/wbstack/queryservice/commit/5c0f82c933008a5d097352c345815fbad29af2c5 there is a statement "The allowlist determines which query services can be used for federated queries. This list is provided at runtime by mounting a newline-delimited text file of endpoints to /wdqs/allowlist.txt. It should include all instances hosted by Wikibase Cloud." Maybe this is not working correctly.

Related Objects

Mentioned In: T423614: Investigate helmfile/helm removes content of altered ConfigMap without warning
Mentioned Here: T418868: 🔬 Merge all of the GitHub Action Dependabot updates for Queryservice

Event Timeline

Epidosis created this task.Wed, Apr 15, 8:11 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptWed, Apr 15, 8:11 AM

Maintenance_bot added a project: Wikidata.Wed, Apr 15, 8:29 AM

Thanks for the report. I can replicate this issue. I looked under the hood and and I can see that indeed the allowlist seems to be empty. I guess we have a bug in building it:

apiVersion: v1
data:
  allowlist-static.txt: |
    https://query-scholarly.wikidata.org/sparql
    https://query.wikidata.org/subgraph/scholarly_articles
    https://query-main.wikidata.org/sparql
    https://query.wikidata.org/subgraph/wikidata_main
    https://query.wikidata.org/bigdata/namespace/categories/sparql
    https://query.wikidata.org/bigdata/namespace/dcatap/sparql
    https://query.wikidata.org/bigdata/namespace/wdq/sparql
    https://query.wikidata.org/sparql
    http://sparql.europeana.eu/
    http://nomisma.org/query
    http://data.cervantesvirtual.com/openrdf-sesame/repositories/data
    http://datos.bne.es/sparql
    http://edan.si.edu/saam/sparql
    http://data.bnf.fr/sparql
    http://dbpedia.org/sparql
    http://rdf.insee.fr/sparql
    https://sparql.wikipathways.org/sparql
    http://dati.camera.it/sparql
    http://dati.emilia-romagna.it/sparql
    http://data.ordnancesurvey.co.uk/datasets/os-linked-data/apis/sparql
    http://statistics.data.gov.uk/sparql
    http://opendatacommunities.org/sparql
    http://data.plan4all.eu/sparql
    http://vocab.getty.edu/sparql.json
    http://linkeddata.uriburner.com/sparql
    http://mw2sparql.toolforge.org/sparql
    http://linkedgeodata.org/sparql
    http://sparql.hegroup.org/sparql/
    http://lov.okfn.org/dataset/lov/sparql
    http://opencitations.net/sparql
    http://dati.beniculturali.it/sparql
    http://zbw.eu/beta/sparql/stw/query
    http://zbw.eu/beta/sparql/gnd/query
    http://zbw.eu/beta/sparql/econ_pers/query
    http://zbw.eu/beta/sparql/econ_corp/query
    http://id.nlm.nih.gov/mesh/sparql
    https://id.nlm.nih.gov/mesh/sparql
    http://collection.britishmuseum.org/sparql
    http://lod.opentransportdata.swiss/query
    http://data.bibliotheken.nl/sparql
    https://bag.basisregistraties.overheid.nl/sparql
    https://brt.basisregistraties.overheid.nl/sparql
    http://collection.britishart.yale.edu/openrdf-sesame/repositories/ycba
    http://bnb.data.bl.uk/sparql
    https://dati.quirinale.it/sparql
    http://sparql.uniprot.org/sparql
    https://sparql.uniprot.org/sparql
    https://ld.stadt-zuerich.ch/query
    http://data.nobelprize.org/sparql
    http://zbw.eu/beta/sparql/pm20/query
    http://libris.kb.se/sparql
    https://api.parliament.uk/sparql
    https://data.istex.fr/sparql/
    https://data.epo.org/linked-data/query
    https://data.niod.nl/PoolParty/sparql/WO2_Thesaurus
    https://www.ebi.ac.uk/rdf/services/sparql
    https://integbio.jp/rdf/ddbj/sparql
    https://integbio.jp/rdf/kero/sparql
    https://integbio.jp/rdf/sparql
    https://data.idref.fr/sparql
    https://sophox.org/sparql
    http://sciencesource-query.wmflabs.org/proxy/wdqs/bigdata/namespace/wdq/sparql
    https://agrovoc.fao.org/sparql
    https://api.druid.datalegend.net/datasets/nlgis/gemeentegeschiedenis/services/gemeentegeschiedenis/sparql
    https://data.niod.nl/PoolParty/sparql/WO2_biografieen
    http://data.muziekschatten.nl/sparql
    https://jpsearch.go.jp/rdf/sparql/
    https://slod.fiz-karlsruhe.de/sparql
    http://dbpedia.org/sparql*
    https://wdhqs.wmflabs.org/sparql
    http://dati.isprambiente.it/sparql
    https://linkeddata.cultureelerfgoed.nl/sparql
    https://tora.entryscape.net/store/sparql
    http://de.dbpedia.org/sparql*
    https://api.data.pldn.nl/datasets/cbs/wijken-buurten/services/wijken-buurten/sparql
    https://idsm.elixir-czech.cz/sparql/endpoint/wikidata
    https://lingualibre.org/sparql
    http://publications.europa.eu/webapi/rdf/sparql
    https://linked.ec-dataplatform.eu/sparql
    https://test-linked.ec-dataplatform.eu/sparql
    https://sweopendata.wikibase.cloud/query/sparql
    https://query.linkedopendata.eu/sparql
    https://data.europa.eu/sparql
    https://opendata.aragon.es/sparql
    https://vocabularies.unesco.org/sparql
    https://digitale.bncf.firenze.sbn.it/openrdf-workbench/repositories/NS/query
    https://lila-erc.eu/sparql/lila_knowledge_base/sparql
    https://qlever.cs.uni-freiburg.de/api/imdb
    https://qlever.cs.uni-freiburg.de/api/dblp
    https://qlever.cs.uni-freiburg.de/api/wikidata
    https://qlever.cs.uni-freiburg.de/api/wikimedia-commons
    https://data.nlg.gr/sparql
    https://nfdi4culture.de/sparql
    https://query.mimotext.uni-trier.de/proxy/wdqs/bigdata/namespace/wdq/sparql
    https://iconclass.org/sparql
    https://orkg.org/triplestore
    https://imagehash-sparql.wmcloud.org/sparql
    https://kaiko.getalp.org/sparql
    https://metabase.wikibase.cloud/query/sparql
    https://linkeddata.cultureelerfgoed.nl/_api/datasets/rce/cho/services/cho/sparql
    https://database.factgrid.de/sparql
    http://api.finto.fi/sparql
    https://kg.kunsten.be/query/proxy/wdqs/bigdata/namespace/wdq/sparql
    https://rkd.triply.cc/_api/datasets/rkd/RKD-Knowledge-Graph/services/SPARQL/sparql
    https://qlever.cs.uni-freiburg.de/api/osm-planet
    https://artsdata-trifid-production.herokuapp.com/query
    https://dbpedia.org/sparql
    https://snarc-llgc.wikibase.cloud/query/sparql
    https://yago-knowledge.org/sparql/query
    https://ddbkg.fiz-karlsruhe.de/sparql
    https://rkd.triply.cc/_api/datasets/rkd/RKD-SDO-Knowledge-Graph/sparql
    https://beta.sparql.swisslipids.org/sparql
    https://sparql.swisslipids.org
    https://query.portal.mardi4nfdi.de/sparql
    https://data.food.gov.uk/codes/system/query
    https://sparql.rhea-db.org/sparql
    http://sparql.archives-ouvertes.fr/sparql
    https://data.library.sh.cn/sparql
    https://triplestore.iccu.sbn.it/sparql
    https://id.ndl.go.jp/auth/ndla/sparql
    https://data.muziekweb.nl/_api/datasets/MuziekwebOrganization/Muziekweb/services/Muziekweb/sparql
    https://www.performing-arts.ch/sparql
    https://query.wikitrek.org/sparql
    https://ld.ncl.edu.tw/fuseki/lod/query
    https://dati.cultura.gov.it/sparql
    https://beta.sparql.swisslipids.org/sparql
    https://sparql.api.identifiers.org/sparql
    https://schema.gov.it/sparql
    https://virtuoso.virtualtreasury.ie/sparql
    https://sparql.dnb.de/api/gnd
    https://gtaa.apis.beeldengeluid.nl/sparql
    https://cat.apis.beeldengeluid.nl/sparql
  allowlist.txt: ""
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: queryservice
    meta.helm.sh/release-namespace: default
  creationTimestamp: "2024-10-24T13:06:43Z"
  labels:
    app.kubernetes.io/managed-by: Helm
  name: queryservice-allowlist
  namespace: default
  resourceVersion: "1776156885637439013"
  uid: aa405ae6-0b3e-4ffa-873c-3eab3b373c78

obtained using kubectl get configmap queryservice-allowlist -o yaml

Almost certainly you saw this at this exact time because deploying T418868 caused a restart and an allowlist reload.

Ollie.Shotton_WMDE moved this task from Backlog (incoming) to Kanban Board on the Wikibase Cloud board.Thu, Apr 16, 9:11 AM

Ollie.Shotton_WMDE edited projects, added Wikibase Cloud (Kanban Board); removed Wikibase Cloud.

Tarrow claimed this task.Thu, Apr 16, 9:14 AM

Tarrow moved this task from Waiting for Tech Breakdown to Doing on the Wikibase Cloud (Kanban Board) board.

The queryservice was probably last restarted with a node pool update around April 6th. We expect the Platform API Laravel job that failed to properly create the ConfigMap ran between April 6th and April 14th.

We looked in the logs to see if the job that makes the ConfigMap errored or had any hints. All we saw in the logs was:
2026-04-13 01:00:29 App\Jobs\UpdateQueryserviceAllowList ........... RUNNING
2026-04-13 01:00:29 App\Jobs\UpdateQueryserviceAllowList ..... 177.13ms DONE

We got here by querying the logs with:

SEARCH("UpdateQueryserviceAllowList")
-resource.labels.cluster_name="wbaas-2"
resource.labels.container_name="api-queue-default"

and then looking around this time.

We wonder if actually the deploy from T418868 actually caused the blanking of this ConfigMap. We saw in the GKE audit logs that a change was made to this ConfigMap when T418868 was deployed (patch) but we can't confirm the exact change from the logs.

What we now intent to do is:

manually re-run the laravel job to rebuild the ConfigMap
if the ConfigMap is rebuilt, restart the queryservice to restore functionality for the users
see if we can recreate (but not execute) the same deployment issue that blanked this ConfigMap

We tested that the same query fails on staging.
We ran the allowlist job with kubectl exec -it deployment/api-queue-default -- sh -c "php artisan job:dispatchNow UpdateQueryserviceAllowList" on staging and can confirm that the allowlist.txt: key of the queryservice-allowlist configmap is now populated.
We restarted the query service with k rollout restart deployment/queryservice.
Then we confirmed that federated query now works on staging.

Rosalie_WMDE moved this task from Waiting for Deploy or Action on Staging to Undergoing Deploy or Action on Staging on the Wikibase Cloud (Kanban Board) board.Thu, Apr 16, 10:28 AM

Rosalie_WMDE moved this task from Undergoing Deploy or Action on Staging to Undergoing Deploy or Action on Production on the Wikibase Cloud (Kanban Board) board.

Reproduced the above steps on production then tried the user's query and it all worked with some results.
Next step is for us to investigate why this happened.

Tarrow mentioned this in T423614: Investigate helmfile/helm removes content of altered ConfigMap without warning.Thu, Apr 16, 3:14 PM

Locally, I modified the image tag in k8s/helmfile/env/staging/queryservice.values.yaml.gotmpl from 0.3.135-wbstack.8 to 0.3.135-wbstack.7 in order to create a small diff between local and staging.
I ran make diff-staging to see what changes helmfile would make with this change.

Comparing release=queryservice, chart=wbstack/queryservice, namespace=default
default, queryservice, Deployment (apps) has changed:
...
          app.kubernetes.io/name: queryservice
          app.kubernetes.io/instance: queryservice
      spec:
        containers:
          - name: queryservice
-           image: "ghcr.io/wbstack/queryservice:0.3.135-wbstack.8"
+           image: "ghcr.io/wbstack/queryservice:0.3.135-wbstack.7"
            imagePullPolicy: Always
            volumeMounts:
            - mountPath: "/wdqs/data"
              name: data
            - mountPath: /wdqs/allowlist.txt
...

Unlike our hypothesis, I did not observe the queryservice-allowlist ConfigMap being modified.

I will deploy this to staging to see if the allowlist.txt field in the ConfigMap is cleared on deployment.

I confirmed that the allowlist.txt field in the ConfigMap is present on staging before doing the deployment.

$ kubectl --context=staging get configmap queryservice-allowlist -o yaml
apiVersion: v1
data:
  allowlist-static.txt: |
    https://query-scholarly.wikidata.org/sparql
    https://query.wikidata.org/subgraph/scholarly_articles
    https://query-main.wikidata.org/sparql
    https://query.wikidata.org/subgraph/wikidata_main
    https://query.wikidata.org/bigdata/namespace/categories/sparql
    https://query.wikidata.org/bigdata/namespace/dcatap/sparql
    https://query.wikidata.org/bigdata/namespace/wdq/sparql
    https://query.wikidata.org/sparql
    http://sparql.europeana.eu/
    http://nomisma.org/query
    http://data.cervantesvirtual.com/openrdf-sesame/repositories/data
    http://datos.bne.es/sparql
    http://edan.si.edu/saam/sparql
    http://data.bnf.fr/sparql
    http://dbpedia.org/sparql
    http://rdf.insee.fr/sparql
    https://sparql.wikipathways.org/sparql
    http://dati.camera.it/sparql
    http://dati.emilia-romagna.it/sparql
    http://data.ordnancesurvey.co.uk/datasets/os-linked-data/apis/sparql
    http://statistics.data.gov.uk/sparql
    http://opendatacommunities.org/sparql
    http://data.plan4all.eu/sparql
    http://vocab.getty.edu/sparql.json
    http://linkeddata.uriburner.com/sparql
    http://mw2sparql.toolforge.org/sparql
    http://linkedgeodata.org/sparql
    http://sparql.hegroup.org/sparql/
    http://lov.okfn.org/dataset/lov/sparql
    http://opencitations.net/sparql
    http://dati.beniculturali.it/sparql
    http://zbw.eu/beta/sparql/stw/query
    http://zbw.eu/beta/sparql/gnd/query
    http://zbw.eu/beta/sparql/econ_pers/query
    http://zbw.eu/beta/sparql/econ_corp/query
    http://id.nlm.nih.gov/mesh/sparql
    https://id.nlm.nih.gov/mesh/sparql
    http://collection.britishmuseum.org/sparql
    http://lod.opentransportdata.swiss/query
    http://data.bibliotheken.nl/sparql
    https://bag.basisregistraties.overheid.nl/sparql
    https://brt.basisregistraties.overheid.nl/sparql
    http://collection.britishart.yale.edu/openrdf-sesame/repositories/ycba
    http://bnb.data.bl.uk/sparql
    https://dati.quirinale.it/sparql
    http://sparql.uniprot.org/sparql
    https://sparql.uniprot.org/sparql
    https://ld.stadt-zuerich.ch/query
    http://data.nobelprize.org/sparql
    http://zbw.eu/beta/sparql/pm20/query
    http://libris.kb.se/sparql
    https://api.parliament.uk/sparql
    https://data.istex.fr/sparql/
    https://data.epo.org/linked-data/query
    https://data.niod.nl/PoolParty/sparql/WO2_Thesaurus
    https://www.ebi.ac.uk/rdf/services/sparql
    https://integbio.jp/rdf/ddbj/sparql
    https://integbio.jp/rdf/kero/sparql
    https://integbio.jp/rdf/sparql
    https://data.idref.fr/sparql
    https://sophox.org/sparql
    http://sciencesource-query.wmflabs.org/proxy/wdqs/bigdata/namespace/wdq/sparql
    https://agrovoc.fao.org/sparql
    https://api.druid.datalegend.net/datasets/nlgis/gemeentegeschiedenis/services/gemeentegeschiedenis/sparql
    https://data.niod.nl/PoolParty/sparql/WO2_biografieen
    http://data.muziekschatten.nl/sparql
    https://jpsearch.go.jp/rdf/sparql/
    https://slod.fiz-karlsruhe.de/sparql
    http://dbpedia.org/sparql*
    https://wdhqs.wmflabs.org/sparql
    http://dati.isprambiente.it/sparql
    https://linkeddata.cultureelerfgoed.nl/sparql
    https://tora.entryscape.net/store/sparql
    http://de.dbpedia.org/sparql*
    https://api.data.pldn.nl/datasets/cbs/wijken-buurten/services/wijken-buurten/sparql
    https://idsm.elixir-czech.cz/sparql/endpoint/wikidata
    https://lingualibre.org/sparql
    http://publications.europa.eu/webapi/rdf/sparql
    https://linked.ec-dataplatform.eu/sparql
    https://test-linked.ec-dataplatform.eu/sparql
    https://sweopendata.wikibase.cloud/query/sparql
    https://query.linkedopendata.eu/sparql
    https://data.europa.eu/sparql
    https://opendata.aragon.es/sparql
    https://vocabularies.unesco.org/sparql
    https://digitale.bncf.firenze.sbn.it/openrdf-workbench/repositories/NS/query
    https://lila-erc.eu/sparql/lila_knowledge_base/sparql
    https://qlever.cs.uni-freiburg.de/api/imdb
    https://qlever.cs.uni-freiburg.de/api/dblp
    https://qlever.cs.uni-freiburg.de/api/wikidata
    https://qlever.cs.uni-freiburg.de/api/wikimedia-commons
    https://data.nlg.gr/sparql
    https://nfdi4culture.de/sparql
    https://query.mimotext.uni-trier.de/proxy/wdqs/bigdata/namespace/wdq/sparql
    https://iconclass.org/sparql
    https://orkg.org/triplestore
    https://imagehash-sparql.wmcloud.org/sparql
    https://kaiko.getalp.org/sparql
    https://metabase.wikibase.cloud/query/sparql
    https://linkeddata.cultureelerfgoed.nl/_api/datasets/rce/cho/services/cho/sparql
    https://database.factgrid.de/sparql
    http://api.finto.fi/sparql
    https://kg.kunsten.be/query/proxy/wdqs/bigdata/namespace/wdq/sparql
    https://rkd.triply.cc/_api/datasets/rkd/RKD-Knowledge-Graph/services/SPARQL/sparql
    https://qlever.cs.uni-freiburg.de/api/osm-planet
    https://artsdata-trifid-production.herokuapp.com/query
    https://dbpedia.org/sparql
    https://snarc-llgc.wikibase.cloud/query/sparql
    https://yago-knowledge.org/sparql/query
    https://ddbkg.fiz-karlsruhe.de/sparql
    https://rkd.triply.cc/_api/datasets/rkd/RKD-SDO-Knowledge-Graph/sparql
    https://beta.sparql.swisslipids.org/sparql
    https://sparql.swisslipids.org
    https://query.portal.mardi4nfdi.de/sparql
    https://data.food.gov.uk/codes/system/query
    https://sparql.rhea-db.org/sparql
    http://sparql.archives-ouvertes.fr/sparql
    https://data.library.sh.cn/sparql
    https://triplestore.iccu.sbn.it/sparql
    https://id.ndl.go.jp/auth/ndla/sparql
    https://data.muziekweb.nl/_api/datasets/MuziekwebOrganization/Muziekweb/services/Muziekweb/sparql
    https://www.performing-arts.ch/sparql
    https://query.wikitrek.org/sparql
    https://ld.ncl.edu.tw/fuseki/lod/query
    https://dati.cultura.gov.it/sparql
    https://beta.sparql.swisslipids.org/sparql
    https://sparql.api.identifiers.org/sparql
    https://schema.gov.it/sparql
    https://virtuoso.virtualtreasury.ie/sparql
    https://sparql.dnb.de/api/gnd
    https://gtaa.apis.beeldengeluid.nl/sparql
    https://cat.apis.beeldengeluid.nl/sparql
  allowlist.txt: |-
    https://goathub.wikibase.dev/query/sparql
    https://deerbase.wikibase.dev/query/sparql
    https://coffeebase.wikibase.dev/query/sparql
    https://somethingwitty.wikibase.dev/query/sparql
    https://goat-136.wikibase.dev/query/sparql
    https://candy-collection.wikibase.dev/query/sparql
    https://potatobase.wikibase.dev/query/sparql
    https://potato.wikibase.dev/query/sparql
    https://asjhfkjsahdkjahs.wikibase.dev/query/sparql
    https://feddypropswiki.wikibase.dev/query/sparql
    https://addshore-alpha.wikibase.dev/query/sparql
    https://addshore-fake-cusstom-domain-wikibase.duckdns.org/query/sparql
    https://deerstack.wikibase.dev/query/sparql
    https://cacti.wikibase.dev/query/sparql
    https://biodiversity.wikibase.dev/query/sparql
    https://toms-test-wikibase.duckdns.org/query/sparql
    https://potatowiki3.wikibase.dev/query/sparql
    https://charlie-test-1.wikibase.dev/query/sparql
    https://deerdeerdeer.wikibase.dev/query/sparql
    https://test-lm-02.wikibase.dev/query/sparql
    https://testwikibase-jan.wikibase.dev/query/sparql
    https://aflex.wikibase.dev/query/sparql
    https://bh20subset1.wikibase.dev/query/sparql
    https://biomarker.wikibase.dev/query/sparql
    https://testa.wikibase.dev/query/sparql
    https://carrotwiki.wikibase.dev/query/sparql
    https://sweetcornbase.wikibase.dev/query/sparql
    https://mywayordatway.wikibase.dev/query/sparql
    https://redbull.wikibase.dev/query/sparql
    https://asdfg.wikibase.dev/query/sparql
    https://asfasfasfasfasf.wikibase.dev/query/sparql
    https://dev-test-wbaas-custom-deer.duckdns.org/query/sparql
    https://andrew.wikibase.dev/query/sparql
    https://mw138test.wikibase.dev/query/sparql
    https://23-06-06-test.wikibase.dev/query/sparql
    https://test-230801.wikibase.dev/query/sparql
    https://test-with-captcha.wikibase.dev/query/sparql
    https://fr-public-beta.wikibase.dev/query/sparql
    https://newaccount.wikibase.dev/query/sparql
    https://testing-wiki.wikibase.dev/query/sparql
    https://rosalie-collection.wikibase.dev/query/sparql
    https://cookiewiki.wikibase.dev/query/sparql
    https://testing-logo.wikibase.dev/query/sparql
    https://test1.wikibase.dev/query/sparql
    https://wbqc-2.wikibase.dev/query/sparql
    https://wbqc-2-alpha.wikibase.dev/query/sparql
    https://oauthtest.wikibase.dev/query/sparql
    https://metrics-test-2025-q1.wikibase.dev/query/sparql
    https://oauth2test.wikibase.dev/query/sparql
    https://newiwiki.wikibase.dev/query/sparql
    https://testwiki.wikibase.dev/query/sparql
    https://ollie-test1.wikibase.dev/query/sparql
    https://somewiki.wikibase.dev/query/sparql
    https://blablabla.wikibase.dev/query/sparql
    https://wikis.wikibase.dev/query/sparql
    https://testing-wiki4.wikibase.dev/query/sparql
    https://testing-wiki5.wikibase.dev/query/sparql
    https://tarrow-143-test1.wikibase.dev/query/sparql
    https://test4.wikibase.dev/query/sparql
    https://test-143-deer.wikibase.dev/query/sparql
    https://anton20.wikibase.dev/query/sparql
    https://test-143-251121-a.wikibase.dev/query/sparql
    https://ollie-test-2.wikibase.dev/query/sparql
    https://somewiki2.wikibase.dev/query/sparql
    https://test-varenya.wikibase.dev/query/sparql
    https://ilovedogs.wikibase.dev/query/sparql
    https://brat1.wikibase.dev/query/sparql
    https://wicked.wikibase.dev/query/sparql
    https://rosalia.wikibase.dev/query/sparql
    https://testprovision.wikibase.dev/query/sparql
    https://testing-wiki6.wikibase.dev/query/sparql
    https://verytemptest.wikibase.dev/query/sparql
    https://sdfgsdfg.wikibase.dev/query/sparql
    https://adsfasdfzxcv.wikibase.dev/query/sparql
    https://ollie-test-3.wbaas.localhost/query/sparql
    https://ollie-test-8.wikibase.dev/query/sparql
    https://ollie-test-9.wikibase.dev/query/sparql
    https://tom-test-20260402-2.wikibase.dev/query/sparql
    https://query-scholarly.wikidata.org/sparql
    https://query.wikidata.org/subgraph/scholarly_articles
    https://query-main.wikidata.org/sparql
    https://query.wikidata.org/subgraph/wikidata_main
    https://query.wikidata.org/bigdata/namespace/categories/sparql
    https://query.wikidata.org/bigdata/namespace/dcatap/sparql
    https://query.wikidata.org/bigdata/namespace/wdq/sparql
    https://query.wikidata.org/sparql
    http://sparql.europeana.eu/
    http://nomisma.org/query
    http://data.cervantesvirtual.com/openrdf-sesame/repositories/data
    http://datos.bne.es/sparql
    http://edan.si.edu/saam/sparql
    http://data.bnf.fr/sparql
    http://dbpedia.org/sparql
    http://rdf.insee.fr/sparql
    https://sparql.wikipathways.org/sparql
    http://dati.camera.it/sparql
    http://dati.emilia-romagna.it/sparql
    http://data.ordnancesurvey.co.uk/datasets/os-linked-data/apis/sparql
    http://statistics.data.gov.uk/sparql
    http://opendatacommunities.org/sparql
    http://data.plan4all.eu/sparql
    http://vocab.getty.edu/sparql.json
    http://linkeddata.uriburner.com/sparql
    http://mw2sparql.toolforge.org/sparql
    http://linkedgeodata.org/sparql
    http://sparql.hegroup.org/sparql/
    http://lov.okfn.org/dataset/lov/sparql
    http://opencitations.net/sparql
    http://dati.beniculturali.it/sparql
    http://zbw.eu/beta/sparql/stw/query
    http://zbw.eu/beta/sparql/gnd/query
    http://zbw.eu/beta/sparql/econ_pers/query
    http://zbw.eu/beta/sparql/econ_corp/query
    http://id.nlm.nih.gov/mesh/sparql
    https://id.nlm.nih.gov/mesh/sparql
    http://collection.britishmuseum.org/sparql
    http://lod.opentransportdata.swiss/query
    http://data.bibliotheken.nl/sparql
    https://bag.basisregistraties.overheid.nl/sparql
    https://brt.basisregistraties.overheid.nl/sparql
    http://collection.britishart.yale.edu/openrdf-sesame/repositories/ycba
    http://bnb.data.bl.uk/sparql
    https://dati.quirinale.it/sparql
    http://sparql.uniprot.org/sparql
    https://sparql.uniprot.org/sparql
    https://ld.stadt-zuerich.ch/query
    http://data.nobelprize.org/sparql
    http://zbw.eu/beta/sparql/pm20/query
    http://libris.kb.se/sparql
    https://api.parliament.uk/sparql
    https://data.istex.fr/sparql/
    https://data.epo.org/linked-data/query
    https://data.niod.nl/PoolParty/sparql/WO2_Thesaurus
    https://www.ebi.ac.uk/rdf/services/sparql
    https://integbio.jp/rdf/ddbj/sparql
    https://integbio.jp/rdf/kero/sparql
    https://integbio.jp/rdf/sparql
    https://data.idref.fr/sparql
    https://sophox.org/sparql
    http://sciencesource-query.wmflabs.org/proxy/wdqs/bigdata/namespace/wdq/sparql
    https://agrovoc.fao.org/sparql
    https://api.druid.datalegend.net/datasets/nlgis/gemeentegeschiedenis/services/gemeentegeschiedenis/sparql
    https://data.niod.nl/PoolParty/sparql/WO2_biografieen
    http://data.muziekschatten.nl/sparql
    https://jpsearch.go.jp/rdf/sparql/
    https://slod.fiz-karlsruhe.de/sparql
    http://dbpedia.org/sparql*
    https://wdhqs.wmflabs.org/sparql
    http://dati.isprambiente.it/sparql
    https://linkeddata.cultureelerfgoed.nl/sparql
    https://tora.entryscape.net/store/sparql
    http://de.dbpedia.org/sparql*
    https://api.data.pldn.nl/datasets/cbs/wijken-buurten/services/wijken-buurten/sparql
    https://idsm.elixir-czech.cz/sparql/endpoint/wikidata
    https://lingualibre.org/sparql
    http://publications.europa.eu/webapi/rdf/sparql
    https://linked.ec-dataplatform.eu/sparql
    https://test-linked.ec-dataplatform.eu/sparql
    https://sweopendata.wikibase.cloud/query/sparql
    https://query.linkedopendata.eu/sparql
    https://data.europa.eu/sparql
    https://opendata.aragon.es/sparql
    https://vocabularies.unesco.org/sparql
    https://digitale.bncf.firenze.sbn.it/openrdf-workbench/repositories/NS/query
    https://lila-erc.eu/sparql/lila_knowledge_base/sparql
    https://qlever.cs.uni-freiburg.de/api/imdb
    https://qlever.cs.uni-freiburg.de/api/dblp
    https://qlever.cs.uni-freiburg.de/api/wikidata
    https://qlever.cs.uni-freiburg.de/api/wikimedia-commons
    https://data.nlg.gr/sparql
    https://nfdi4culture.de/sparql
    https://query.mimotext.uni-trier.de/proxy/wdqs/bigdata/namespace/wdq/sparql
    https://iconclass.org/sparql
    https://orkg.org/triplestore
    https://imagehash-sparql.wmcloud.org/sparql
    https://kaiko.getalp.org/sparql
    https://metabase.wikibase.cloud/query/sparql
    https://linkeddata.cultureelerfgoed.nl/_api/datasets/rce/cho/services/cho/sparql
    https://database.factgrid.de/sparql
    http://api.finto.fi/sparql
    https://kg.kunsten.be/query/proxy/wdqs/bigdata/namespace/wdq/sparql
    https://rkd.triply.cc/_api/datasets/rkd/RKD-Knowledge-Graph/services/SPARQL/sparql
    https://qlever.cs.uni-freiburg.de/api/osm-planet
    https://artsdata-trifid-production.herokuapp.com/query
    https://dbpedia.org/sparql
    https://snarc-llgc.wikibase.cloud/query/sparql
    https://yago-knowledge.org/sparql/query
    https://ddbkg.fiz-karlsruhe.de/sparql
    https://rkd.triply.cc/_api/datasets/rkd/RKD-SDO-Knowledge-Graph/sparql
    https://beta.sparql.swisslipids.org/sparql
    https://sparql.swisslipids.org
    https://query.portal.mardi4nfdi.de/sparql
    https://data.food.gov.uk/codes/system/query
    https://sparql.rhea-db.org/sparql
    http://sparql.archives-ouvertes.fr/sparql
    https://data.library.sh.cn/sparql
    https://triplestore.iccu.sbn.it/sparql
    https://id.ndl.go.jp/auth/ndla/sparql
    https://data.muziekweb.nl/_api/datasets/MuziekwebOrganization/Muziekweb/services/Muziekweb/sparql
    https://www.performing-arts.ch/sparql
    https://query.wikitrek.org/sparql
    https://ld.ncl.edu.tw/fuseki/lod/query
    https://dati.cultura.gov.it/sparql
    https://beta.sparql.swisslipids.org/sparql
    https://sparql.api.identifiers.org/sparql
    https://schema.gov.it/sparql
    https://virtuoso.virtualtreasury.ie/sparql
    https://sparql.dnb.de/api/gnd
    https://gtaa.apis.beeldengeluid.nl/sparql
    https://cat.apis.beeldengeluid.nl/sparql
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: queryservice
    meta.helm.sh/release-namespace: default
  creationTimestamp: "2024-10-24T11:58:48Z"
  labels:
    app.kubernetes.io/managed-by: Helm
  name: queryservice-allowlist
  namespace: default
  resourceVersion: "1776333845742927013"
  uid: db8c5650-36c0-4ba4-8914-faf37ad156c4

I deployed the change on staging with make apply-staging saw again that only the image tag was being changed in the diff.

Viewing the ConfigMap again we can see that the allowlist.txt key has indeed been blanked.

$ kubectl --context=staging get configmap queryservice-allowlist -o yaml
apiVersion: v1
data:
  allowlist-static.txt: |
    https://query-scholarly.wikidata.org/sparql
    https://query.wikidata.org/subgraph/scholarly_articles
    https://query-main.wikidata.org/sparql
    https://query.wikidata.org/subgraph/wikidata_main
    https://query.wikidata.org/bigdata/namespace/categories/sparql
    https://query.wikidata.org/bigdata/namespace/dcatap/sparql
    https://query.wikidata.org/bigdata/namespace/wdq/sparql
    https://query.wikidata.org/sparql
    http://sparql.europeana.eu/
    http://nomisma.org/query
    http://data.cervantesvirtual.com/openrdf-sesame/repositories/data
    http://datos.bne.es/sparql
    http://edan.si.edu/saam/sparql
    http://data.bnf.fr/sparql
    http://dbpedia.org/sparql
    http://rdf.insee.fr/sparql
    https://sparql.wikipathways.org/sparql
    http://dati.camera.it/sparql
    http://dati.emilia-romagna.it/sparql
    http://data.ordnancesurvey.co.uk/datasets/os-linked-data/apis/sparql
    http://statistics.data.gov.uk/sparql
    http://opendatacommunities.org/sparql
    http://data.plan4all.eu/sparql
    http://vocab.getty.edu/sparql.json
    http://linkeddata.uriburner.com/sparql
    http://mw2sparql.toolforge.org/sparql
    http://linkedgeodata.org/sparql
    http://sparql.hegroup.org/sparql/
    http://lov.okfn.org/dataset/lov/sparql
    http://opencitations.net/sparql
    http://dati.beniculturali.it/sparql
    http://zbw.eu/beta/sparql/stw/query
    http://zbw.eu/beta/sparql/gnd/query
    http://zbw.eu/beta/sparql/econ_pers/query
    http://zbw.eu/beta/sparql/econ_corp/query
    http://id.nlm.nih.gov/mesh/sparql
    https://id.nlm.nih.gov/mesh/sparql
    http://collection.britishmuseum.org/sparql
    http://lod.opentransportdata.swiss/query
    http://data.bibliotheken.nl/sparql
    https://bag.basisregistraties.overheid.nl/sparql
    https://brt.basisregistraties.overheid.nl/sparql
    http://collection.britishart.yale.edu/openrdf-sesame/repositories/ycba
    http://bnb.data.bl.uk/sparql
    https://dati.quirinale.it/sparql
    http://sparql.uniprot.org/sparql
    https://sparql.uniprot.org/sparql
    https://ld.stadt-zuerich.ch/query
    http://data.nobelprize.org/sparql
    http://zbw.eu/beta/sparql/pm20/query
    http://libris.kb.se/sparql
    https://api.parliament.uk/sparql
    https://data.istex.fr/sparql/
    https://data.epo.org/linked-data/query
    https://data.niod.nl/PoolParty/sparql/WO2_Thesaurus
    https://www.ebi.ac.uk/rdf/services/sparql
    https://integbio.jp/rdf/ddbj/sparql
    https://integbio.jp/rdf/kero/sparql
    https://integbio.jp/rdf/sparql
    https://data.idref.fr/sparql
    https://sophox.org/sparql
    http://sciencesource-query.wmflabs.org/proxy/wdqs/bigdata/namespace/wdq/sparql
    https://agrovoc.fao.org/sparql
    https://api.druid.datalegend.net/datasets/nlgis/gemeentegeschiedenis/services/gemeentegeschiedenis/sparql
    https://data.niod.nl/PoolParty/sparql/WO2_biografieen
    http://data.muziekschatten.nl/sparql
    https://jpsearch.go.jp/rdf/sparql/
    https://slod.fiz-karlsruhe.de/sparql
    http://dbpedia.org/sparql*
    https://wdhqs.wmflabs.org/sparql
    http://dati.isprambiente.it/sparql
    https://linkeddata.cultureelerfgoed.nl/sparql
    https://tora.entryscape.net/store/sparql
    http://de.dbpedia.org/sparql*
    https://api.data.pldn.nl/datasets/cbs/wijken-buurten/services/wijken-buurten/sparql
    https://idsm.elixir-czech.cz/sparql/endpoint/wikidata
    https://lingualibre.org/sparql
    http://publications.europa.eu/webapi/rdf/sparql
    https://linked.ec-dataplatform.eu/sparql
    https://test-linked.ec-dataplatform.eu/sparql
    https://sweopendata.wikibase.cloud/query/sparql
    https://query.linkedopendata.eu/sparql
    https://data.europa.eu/sparql
    https://opendata.aragon.es/sparql
    https://vocabularies.unesco.org/sparql
    https://digitale.bncf.firenze.sbn.it/openrdf-workbench/repositories/NS/query
    https://lila-erc.eu/sparql/lila_knowledge_base/sparql
    https://qlever.cs.uni-freiburg.de/api/imdb
    https://qlever.cs.uni-freiburg.de/api/dblp
    https://qlever.cs.uni-freiburg.de/api/wikidata
    https://qlever.cs.uni-freiburg.de/api/wikimedia-commons
    https://data.nlg.gr/sparql
    https://nfdi4culture.de/sparql
    https://query.mimotext.uni-trier.de/proxy/wdqs/bigdata/namespace/wdq/sparql
    https://iconclass.org/sparql
    https://orkg.org/triplestore
    https://imagehash-sparql.wmcloud.org/sparql
    https://kaiko.getalp.org/sparql
    https://metabase.wikibase.cloud/query/sparql
    https://linkeddata.cultureelerfgoed.nl/_api/datasets/rce/cho/services/cho/sparql
    https://database.factgrid.de/sparql
    http://api.finto.fi/sparql
    https://kg.kunsten.be/query/proxy/wdqs/bigdata/namespace/wdq/sparql
    https://rkd.triply.cc/_api/datasets/rkd/RKD-Knowledge-Graph/services/SPARQL/sparql
    https://qlever.cs.uni-freiburg.de/api/osm-planet
    https://artsdata-trifid-production.herokuapp.com/query
    https://dbpedia.org/sparql
    https://snarc-llgc.wikibase.cloud/query/sparql
    https://yago-knowledge.org/sparql/query
    https://ddbkg.fiz-karlsruhe.de/sparql
    https://rkd.triply.cc/_api/datasets/rkd/RKD-SDO-Knowledge-Graph/sparql
    https://beta.sparql.swisslipids.org/sparql
    https://sparql.swisslipids.org
    https://query.portal.mardi4nfdi.de/sparql
    https://data.food.gov.uk/codes/system/query
    https://sparql.rhea-db.org/sparql
    http://sparql.archives-ouvertes.fr/sparql
    https://data.library.sh.cn/sparql
    https://triplestore.iccu.sbn.it/sparql
    https://id.ndl.go.jp/auth/ndla/sparql
    https://data.muziekweb.nl/_api/datasets/MuziekwebOrganization/Muziekweb/services/Muziekweb/sparql
    https://www.performing-arts.ch/sparql
    https://query.wikitrek.org/sparql
    https://ld.ncl.edu.tw/fuseki/lod/query
    https://dati.cultura.gov.it/sparql
    https://beta.sparql.swisslipids.org/sparql
    https://sparql.api.identifiers.org/sparql
    https://schema.gov.it/sparql
    https://virtuoso.virtualtreasury.ie/sparql
    https://sparql.dnb.de/api/gnd
    https://gtaa.apis.beeldengeluid.nl/sparql
    https://cat.apis.beeldengeluid.nl/sparql
  allowlist.txt: ""
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: queryservice
    meta.helm.sh/release-namespace: default
  creationTimestamp: "2024-10-24T11:58:48Z"
  labels:
    app.kubernetes.io/managed-by: Helm
  name: queryservice-allowlist
  namespace: default
  resourceVersion: "1776350699095807013"
  uid: db8c5650-36c0-4ba4-8914-faf37ad156c4

I also downloaded helmfile v1.4.2 to see if that showed the diff correctly or not - it had the same effect.

A possible solution from today: We could split the ConfigMap into two ConfigMaps (one each for allowlist-static.txt and allowlist.txt). The allowlist-static.txt ConfigMap would stay in helm/helmfile and the allowlist.txt ConfigMap would ONLY be created and updated by the Platform API. This would stop this overwriting of the dynamically generated allowlist.txt ConfigMap on every change to the queryservice. However, this would have the negative effect of making it more difficult to run a standalone version of this queryservice chart as the ConfigMap it would be reading won't have been created (as there is no Platform API).

Ollie.Shotton_WMDE moved this task from Doing to Done on the Wikibase Cloud (Kanban Board) board.Thu, Apr 16, 3:24 PM

Martyn.ranyard closed this task as Resolved.Mon, Apr 20, 9:39 AM

Federated queries with WDQS failing in Wikibase Cloud instancesClosed, ResolvedPublicBUG REPORTActions

Description

Related Objects

Event Timeline

Federated queries with WDQS failing in Wikibase Cloud instances
Closed, ResolvedPublicBUG REPORT
Actions