Multiple deleted items still available in Wikidata Query Service
Closed, ResolvedPublicBUG REPORT
Actions

Description

A SPARQL query we're using for a bot (https://github.com/konstin/github-wikidata-bot) has started return a page that doesn't exist anymore.

Minimized example query:

PREFIX schema: <http://schema.org/>

SELECT ?project ?repo WHERE {
  ?project wdt:P1324 ?repo.
  FILTER(contains(STR(?repo), "https://github.com/dingocoin/dingocoin")).
}

This returns wd:Q124831300 and https://github.com/dingocoin/dingocoin. The page (https://www.wikidata.org/wiki/Q124831300) has been deleted 2025-10-03, but the page is still returned by SPARQL queries. It seems that the deletion has not been synced.

Related Objects

Mentioned In: T422179: Set up regular bulk data ingestion and indexing pipeline
T411958: database / search / user interface inconsistency in Wikidata object
T410026: Deleted Wikidata item still turns up on Sparql
Mentioned Here: T204024: Store WikibaseQualityConstraint check data in persistent storage instead of in the cache
T105427: Need a way for WDQS updater to become aware of suppressed deletes
T155954: WDQS returns deleted and redirected items
T272120: Deleted item still gets shown in WDQS query results

Event Timeline

Konstin created this task.Oct 19 2025, 8:08 AM

Restricted Application added a subscriber: • Aklapper. · View Herald TranscriptOct 19 2025, 8:08 AM

Maintenance_bot added a project: Wikidata.Oct 19 2025, 8:29 AM

karapayneWMDE added a project: Wikidata-Omega.Oct 21 2025, 5:06 PM

A_smart_kitten mentioned this in T410026: Deleted Wikidata item still turns up on Sparql.Nov 14 2025, 11:03 AM

Same issue with Q136764759 which also still is available for querying even if it's deleted.

Mbch331 renamed this task from Wikidata SPARQL query returns page (Q124831300) that has been deleted days ago to Multiple deleted items still available in Wikidata Query Service.Nov 14 2025, 11:40 AM

Issue now also occurs for Q136803226

And now Q137154988 as well

gmodena subscribed.Dec 2 2025, 12:57 PM

RKemper subscribed.Dec 3 2025, 1:27 AM

It's interesting that the deletion in the task description is still not processed. I spot-checked every wdqs host in eqiad, and they all agree that the item still exists, so there's definitely an issue here and it's not merely confined to a few hosts.

We can likely resolve the items listed in this ticket with a data-reload, but that will take some time (and also doesn't answer the question of how the delete event was missed in the first place). We'll need to do some more investigation on that front.

BTracy-WMF moved this task from Incoming to User Support on the Wikidata-Query-Service board.Dec 3 2025, 9:00 PM

Mahir256 subscribed.Dec 13 2025, 5:44 AM

Previous tickets include T272120 and T155954 - I also have a case found today, in https://w.wiki/7NMU I still see http://www.wikidata.org/entity/Q137425313 which was deleted on 17 December (6 days ago).

And T105427 seems also related to this issue.

gmodena edited projects, added: Wikidata-Query-Service (Current Sprint); removed: Wikidata-Query-Service.Jan 20 2026, 2:07 PM

Could this be resolved by this ticket: T204024: Store WikibaseQualityConstraint check data in persistent storage instead of in the cache

I don’t see how, they’re different systems.

This problem persists with lexemes as well; https://www.wikidata.org/wiki/Lexeme:L1531124 and https://www.wikidata.org/wiki/Lexeme:L9913 still appeared in query results for a long time. (I had to undelete and redelete them in order to get the deletions to register, but I don't think this workaround is particularly scalable.)

DSantamaria edited projects, added: Wikidata Platform Team (Sprint 01 (2026/01/13)); removed: Wikidata-Query-Service (Current Sprint).Feb 4 2026, 12:52 PM

BTracy-WMF assigned this task to lerickson.Feb 6 2026, 12:34 PM

gmodena added a project: Wikidata-Query-Service.Feb 10 2026, 12:06 PM

DSantamaria added a project: Essential-Work.Feb 17 2026, 2:01 PM

DSantamaria edited projects, added: Wikidata Platform Team (Sprint 02 (2026/02/09)); removed: Wikidata Platform Team (Sprint 01 (2026/01/13)).Feb 17 2026, 2:08 PM

DSantamaria moved this task from Backlog to Ready on the Wikidata Platform Team (Sprint 02 (2026/02/09)) board.Feb 17 2026, 2:29 PM

The problem persists with items also: https://www.wikidata.org/wiki/Q137425313 (del. 2025-12-17) and also https://www.wikidata.org/wiki/Q121585638 (del. 2025-10-04) still appear in query results.

DSantamaria moved this task from Sprint 02 (2026/02/09) to Sprint 03 (2026/03/03) on the Wikidata Platform Team board.Mar 3 2026, 2:15 PM

DSantamaria edited projects, added: Wikidata Platform Team (Sprint 03 (2026/03/03)); removed: Wikidata Platform Team (Sprint 02 (2026/02/09)).

DSantamaria moved this task from Backlog to Ready on the Wikidata Platform Team (Sprint 03 (2026/03/03)) board.

gmodena mentioned this in T411958: database / search / user interface inconsistency in Wikidata object.Mar 6 2026, 10:50 AM

karapayneWMDE moved this task from Incoming to Radar/Epics/Stalled on the Wikidata-Omega board.Mar 19 2026, 4:42 PM

karapayneWMDE edited projects, added: Wikidata-Omega (Radar/Epics/Stalled); removed: Wikidata-Omega.

karapayneWMDE moved this task from Incoming to Radar on the Wikidata-Omega (Radar/Epics/Stalled) board.

lerickson moved this task from Ready to In Engineering on the Wikidata Platform Team (Sprint 03 (2026/03/03)) board.Mar 31 2026, 6:24 PM

lerickson changed the task status from Open to In Progress.Mar 31 2026, 8:11 PM

Hi there! First, sorry for the slow response on this. I've just run a script to trigger RDF reconciliation for all the entities named here, and they should be deleted from WDQS now. If you see that your deleted entry is still queryable, please follow up and let me know.

Some useful background context is that the data behind WDQS is actually a secondary store. It is periodically synced with the primary source of truth, which is Wikidata and the databases backing it. To make the data queryable via SPARQL, WDQS maintains a separate copy of the data in RDF form. When updates come in to Wikidata, they are published to a Kafka topic, which a WDQS infrastructure component reads and translates into SPARQL update statements. These are then propagated to the graph database underlying WDQS, and the data is queryable there. To anticipate possible breakdowns in this streaming-update setup, we have an automatic reconciliation flow that is supposed to retry missed updates. We confirmed that the entities mentioned on this ticket were not in this situation. That is, we conclude that for some reason the updates never got posted to Kafka.

Ideally, we would have had some sort of periodic batch reloading and reindexing of all the data, which would have caught any events missed like that. However, in practice right now, we can't reload all of the data in WDQS in a reasonable amount of time. So the mitigation has to be via the streaming flow as well.

We have a script that we can run to re-read the source of truth, which is the mediawiki update that deleted an entity. For now, that's the best way to deal with this problem. I will document this on wikitech shortly.

The backend migration of WDQS will unblock better solutions in this realm. For example, that ideal bulk data reload I mentioned above will be possible, meaning that we won't have a situation of updates being missed for weeks or months. But for now we will unfortunately need somebody to notice that the deleted entities are still queryable.

I'll post the documentation here when I've written it. Leaving the phab open until then.

I have 2 items that are still in WDQS, but are deleted: http://www.wikidata.org/entity/Q137209877, http://www.wikidata.org/entity/Q138658750

lerickson mentioned this in T422179: Set up regular bulk data ingestion and indexing pipeline.Apr 2 2026, 3:25 PM

Thank you for reporting those @Mbch331 ; I ran the same script on those entities and they should be un-queryable now (or soon, due to eventual consistency).

Update: I've added some documentation to help others solve this problem in the future.

lerickson moved this task from In Engineering to Needs Sign-off on the Wikidata Platform Team (Sprint 03 (2026/03/03)) board.Apr 2 2026, 5:11 PM

BTracy-WMF closed this task as Resolved.Apr 2 2026, 6:31 PM

@lerickson Hi, I noticed the following three deleted items are being returned by WDQS:
https://www.wikidata.org/wiki/Q136696641
https://www.wikidata.org/wiki/Q136766770
https://www.wikidata.org/wiki/Q138507916

Thanks @Yirba for the report! I attempted to fix these entities in the same way, but I am unfortunately still seeing them returned in WDQS. I will investigate and get back to you.

Hi again @Yirba , I believe it's all ready now (I don't see the items in WDQS anymore). Thanks again for the report.

Multiple deleted items still available in Wikidata Query ServiceClosed, ResolvedPublicBUG REPORTActions

Description

Related Objects

Event Timeline

Multiple deleted items still available in Wikidata Query Service
Closed, ResolvedPublicBUG REPORT
Actions