Page MenuHomePhabricator
Create Task
Maniphest T364233

add https://imagehash-sparql.wmcloud.org/sparql endpoint to wikidata federated query whitelists
Closed, ResolvedPublic
Actions

Description

Hi, I have been creating an image hash database for Wikimedia Commons images and I am currently setting up a SPARQL interface for it to wmflabs (T364063). SPARQL server used is Ontop. (Ontop is a SPARQL->SQL translator.)

I would like to make queries from the Wikimedia Commons query service, so I would like to request that the imagehash endpoint be added to the https://query.wikidata.org and https://commons-query.wikimedia.org federated query whitelists.

Currently endpoint is added

  • query.wikidata.org
  • commons-query.wikimedia.org

Endpoint

Data in backend database:

Software used as local SPARQL endpoint.

Details

Other Assignee
• Stevemunene
Related Changes in Gerrit:
SubjectRepoBranchLines +/-
wdqs: allow 3 new federated endpointsoperations/puppetproduction+3 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

Zache created this task.May 4 2024, 2:28 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 4 2024, 2:29 PM
Maintenance_bot added a project: Wikidata.May 4 2024, 2:29 PM
Zache added a parent task: T197530: [tracking] federation query issues on Wikidata Query Server.May 4 2024, 2:30 PM
Gehel triaged this task as Medium priority.May 6 2024, 3:51 PM
Gehel added a project: Data-Platform-SRE.
Gehel moved this task from Incoming to Operations/SRE on the Wikidata-Query-Service board.
Gehel moved this task from Incoming to Upcoming Essential or OKR Work on the Data-Platform-SRE board.
Aklapper moved this task from Backlog to Hacking projects on the Wikimedia-Hackathon-2024 board.May 7 2024, 10:57 AM
RKemper subscribed.Jul 30 2024, 3:19 PM
Zache updated the task description. (Show Details)Aug 9 2024, 9:59 AM
Zache updated the task description. (Show Details)Aug 9 2024, 10:02 AM
Zache added a project: WMFI-imagehash.
Zache updated the task description. (Show Details)Aug 9 2024, 11:13 AM
Zache updated the task description. (Show Details)Aug 9 2024, 11:19 AM
gerritbot added a comment.Sep 19 2024, 3:18 PM

Change #1074199 had a related patch set uploaded (by Ryan Kemper; author: Ryan Kemper):

[operations/puppet@production] wdqs: allow 3 new federated endpoints

https://gerrit.wikimedia.org/r/1074199

gerritbot added a project: Patch-For-Review.Sep 19 2024, 3:18 PM
gerritbot added a comment.Sep 23 2024, 2:58 PM

Change #1074199 merged by Ryan Kemper:

[operations/puppet@production] wdqs: allow 3 new federated endpoints

https://gerrit.wikimedia.org/r/1074199

Stashbot mentioned this in T374195: Add https://metabase.wikibase.cloud/query/sparql to the Wikidata Federated Query Whitelist.Sep 23 2024, 3:15 PM
Stashbot mentioned this in T368085: Allow federated queries with Dbnary (kaiko.getalp.org).

Mentioned in SAL (#wikimedia-operations) [2024-09-23T15:14:59Z] <stevemunene@deploy1003> Started deploy [wdqs/wdqs@316bf7f]: allow 3 new endpoints T364233 T368085 T374195

Stashbot added a comment.Sep 23 2024, 3:20 PM

Mentioned in SAL (#wikimedia-operations) [2024-09-23T15:20:50Z] <stevemunene@deploy1003> Finished deploy [wdqs/wdqs@316bf7f]: allow 3 new endpoints T364233 T368085 T374195 (duration: 05m 51s)

Maintenance_bot removed a project: Patch-For-Review.Sep 23 2024, 3:31 PM
• Stevemunene assigned this task to RKemper.Sep 23 2024, 3:55 PM
• Stevemunene updated Other Assignee, added: • Stevemunene.
Zache added a comment.Sep 24 2024, 8:38 AM

Thank you @RKemper!

Confirmed that endpoint works now from WQS (query.wikidata.org)

However, it is not whitelisted in WCQS ( commons-query.wikimedia.org ) as it fails with error: Unknown error: Service URI https://imagehash-sparql.wmcloud.org/sparql is not allowed.

Do you know how it could be added to WCQS whitelist also?

Gehel edited projects, added Data-Platform-SRE (2024.09.06 - 2024.09.27); removed Data-Platform-SRE.Sep 25 2024, 7:57 AM
Gehel moved this task from Backlog - project to Backlog - operations on the Data-Platform-SRE (2024.09.06 - 2024.09.27) board.
• Stevemunene closed this task as Resolved.Sep 25 2024, 9:46 AM
• Stevemunene moved this task from Backlog - operations to Done on the Data-Platform-SRE (2024.09.06 - 2024.09.27) board.
Zache reopened this task as Open.Sep 25 2024, 10:22 AM
Zache updated the task description. (Show Details)
Zache added a subscriber: • Stevemunene.Sep 25 2024, 10:25 AM

@Stevemunene I reopened this because the task described in the ticket was only completed for query.wikidata.org, and it does not work yet on commons-query.wikimedia.org.

Gehel moved this task from Done to In Progress on the Data-Platform-SRE (2024.09.06 - 2024.09.27) board.Sep 27 2024, 12:54 PM
Gehel edited projects, added Data-Platform-SRE (2024.09.28 - 2024.10.18); removed Data-Platform-SRE (2024.09.06 - 2024.09.27).Sep 27 2024, 1:00 PM
Gehel moved this task from Backlog - project to In Progress on the Data-Platform-SRE (2024.09.28 - 2024.10.18) board.
• Stevemunene added a comment.Sep 30 2024, 8:45 AM
In T364233#10174875, @Zache wrote:

@Stevemunene I reopened this because the task described in the ticket was only completed for query.wikidata.org, and it does not work yet on commons-query.wikimedia.org.

Thanks @Zache, working on it

BTullis edited projects, added Data-Platform-SRE (2024.10.19 - 2024.11.08); removed Data-Platform-SRE (2024.09.28 - 2024.10.18).Oct 18 2024, 3:08 PM
BTullis moved this task from Backlog - project to In Progress on the Data-Platform-SRE (2024.10.19 - 2024.11.08) board.
Gehel edited projects, added Data-Platform-SRE (2024.11.09 - 2024.11.29); removed Data-Platform-SRE (2024.10.19 - 2024.11.08).Nov 8 2024, 9:48 AM
Gehel moved this task from Backlog - project to In Progress on the Data-Platform-SRE (2024.11.09 - 2024.11.29) board.
Gehel edited projects, added Data-Platform-SRE (2024.11.30 - 2024.12.20); removed Data-Platform-SRE (2024.11.09 - 2024.11.29).Nov 29 2024, 1:22 PM
Gehel moved this task from Backlog - project to In Progress on the Data-Platform-SRE (2024.11.30 - 2024.12.20) board.
bking claimed this task.Dec 17 2024, 6:56 PM
bking closed this task as Resolved.Dec 17 2024, 7:08 PM
bking moved this task from In Progress to Done on the Data-Platform-SRE (2024.11.30 - 2024.12.20) board.

@Zache I ran your example query against commons-query and it appears to be giving a response. As such, I'm going to resolve this ticket. That being said: if it does not work, respond here and the ticket will re-open.

Gehel moved this task from Done to Reported on the Data-Platform-SRE (2024.11.30 - 2024.12.20) board.Dec 20 2024, 1:33 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits