Page MenuHomePhabricator

Wikidata Query Service is swapping items and properties
Open, HighPublicBUG REPORT

Description

The Wikidata Query Service seems to have swapped the triples of items and properties, somehow.

SELECT ?entity ?label WHERE {
  VALUES ?entity { wd:P31 wd:Q31 }
  ?entity rdfs:label ?label.
  FILTER(LANG(?label) = "en")
}
wd:P31Belgium
wd:Q31instance of

Only the wd: namespace appears to be affected, statements are still found under e. g. wdt:P31 and not wdt:Q31.


Original report:


Steps to Reproduce:

Go to, e.g., https://tools.wmflabs.org/scholia/author/Q18618629

Actual Results:

Some of the panels that include embedding report, e.g., "Unable to display result: Bar chart"

Expected Results:

The embedded visualization from WDQS should show.


An error is 'TypeError: "i is undefined"' from embed.wdqs.min.3f7096aef907206f2a35.js:1:17344

Furthermore, https://query.wikidata.org/css/embed.style.min.fa3ff6a142279256ede4.css gives 404

Event Timeline

Restricted Application added a project: Wikidata. · View Herald TranscriptAug 15 2019, 10:20 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Lucas_Werkmeister_WMDE triaged this task as High priority.EditedAug 15 2019, 10:35 PM

Those errors usually happen when the query returns 0 results and the visualization is not able to handle it, and that seems to be what’s happening.

@Moebeus also reported on Telegram that there are lots of broken “distinct values” constraint violations, e. g. on Donald Trump’s subreddit, that the value is also present on “P28228919 (Deleted Property)”. In fact, the value is present on Q28228919, the dedicated item for r/The_Donald.


And the “cats” example query returns 0 results:

SELECT ?item ?itemLabel WHERE {
  ?item wdt:P31 wd:Q146.
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
}

However, after changing Q146 to P146, it returns the usual 142 results (CC, Mrs. Chippy, F. D. C. Willard, etc.):

SELECT ?item ?itemLabel WHERE {
  ?item wdt:P31 wd:P146.
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
}

@Smalyshev it looks like the query service is mixing up P- and Q-IDs, somehow? Were there any recent changes to the internal vocabulary?

https://query.wikidata.org/css/embed.style.min.fa3ff6a142279256ede4.css gives me CSS content (Bootstrap v3.4.1)

but i have the same issue in https://tools.wmflabs.org/scholia/author/Q18618629 with the same error message (in french)

Lucas_Werkmeister_WMDE renamed this task from Wikidata Query Service visualization with embedding gives error to Wikidata Query Service is swapping items and properties.Aug 15 2019, 10:42 PM
Lucas_Werkmeister_WMDE updated the task description. (Show Details)

Yes, there is a Q/P mixup. DESCRIBE wd:P20980928 gives 512 results.

There has been a recent namespace change announced at https://lists.wikimedia.org/pipermail/wikidata/2019-August/013350.html but this was with dcatap.

Mentioned in SAL (#wikimedia-operations) [2019-08-15T23:25:55Z] <smalyshev@deploy1001> Started deploy [wdqs/wdqs@b4da6e4]: Rollback blazegraph due to T230588

Probably my fault, I've deployed new WDQS with some URL refactoring, but looks like something went wrong with URI scheme. I will be rolling it back.

Mentioned in SAL (#wikimedia-operations) [2019-08-15T23:35:44Z] <smalyshev@deploy1001> Finished deploy [wdqs/wdqs@b4da6e4]: Rollback blazegraph due to T230588 (duration: 09m 48s)

Looks like Belgium is back to Q31 and cats are cats again. I will investigate what happened to the URIs (most likely the order in the dictionary switched somehow because of changing of underlying storage class but I have missed it since the content of the dictionary is still the same).

Lucas_Werkmeister_WMDE lowered the priority of this task from High to Normal.Aug 15 2019, 11:50 PM

Scholia also looks fine again – lowering priority since only the investigation is missing. Thanks a lot for the quick response Stas!

Change 530463 had a related patch set uploaded (by Smalyshev; owner: Smalyshev):
[wikidata/query/rdf@master] Fix prefixes order - it was P, then Q

https://gerrit.wikimedia.org/r/530463


Maybe it's just the cache, but I still see the problem, albeit on different identifiers than those affected yesterday.

Ghuron added a subscriber: Ghuron.Aug 16 2019, 9:23 AM

Still getting a few dozens of wdt:Pxxx, query:

SELECT * { ?obj wdt:P3083 [] FILTER(STRSTARTS(STR(?obj),"http://www.wikidata.org/entity/P")) }
Mmarx added a subscriber: Mmarx.Aug 16 2019, 12:09 PM

Another example, currently has 101 results:

SELECT ?item WHERE {
  ?item wdt:P495 [] .
  {{ FILTER NOT EXISTS { ?item wdt:P161 [] }} UNION { FILTER NOT EXISTS { ?item wdt:P364 [] }}}
  FILTER(STRSTARTS(STR(?item), "http://www.wikidata.org/entity/P"))
}

Change 530463 merged by jenkins-bot:
[wikidata/query/rdf@master] Fix prefixes order - it was P, then Q

https://gerrit.wikimedia.org/r/530463

I think all the items that were updated during the breakage, and were not updated since then, would be broken now – their data was imported using the swapped Ps and Qs, and then after the fix it’s swapped the wrong way. We should be able to get a list of all potentially affected items from recentchanges (Quarry) – @Smalyshev is it possible to tell the updater to re-import those specific entities?

I definitely can re-import mine, but existing wdt:Pxxx makes operating existing bots much more difficult than it should be

I think all the items that were updated during the breakage, and were not updated since then, would be broken now – their data was imported using the swapped Ps and Qs, and then after the fix it’s swapped the wrong way. We should be able to get a list of all potentially affected items from recentchanges (Quarry) – @Smalyshev is it possible to tell the updater to re-import those specific entities?

Can confirm. By deleting and immediately re-creating value pairs I see the error disappear.

Ghuron added a comment.EditedAug 18 2019, 3:12 PM

Mine wdt:P disappear

Smalyshev closed this task as Resolved.Aug 19 2019, 3:32 AM

Should be back to normal now.

Hmm looks like I updated all Qs but forgot about some Ps... These should be fine now.

Change 531985 had a related patch set uploaded (by Smalyshev; owner: Smalyshev):
[wikidata/query/rdf@master] Fix initials order

https://gerrit.wikimedia.org/r/531985

Change 531985 merged by jenkins-bot:
[wikidata/query/rdf@master] Fix initials order

https://gerrit.wikimedia.org/r/531985

Lucas_Werkmeister_WMDE reopened this task as Open.EditedThu, Sep 5, 12:59 PM

It looks like there are still some properties hidden in wikibase:quantityUnit values (e. g. P199 instead of Q199, which is the RDF version of "1"), as reported on project chat.

Edit: see T232212: QuantityValue quantityUnit contains both Q and P value in Wikidata Query Service - P value is wrong for a more detailed description of that issue (I’ve marked it as a duplicate of this one since I think it’s the same underlying cause).

agray added a subscriber: agray.EditedSat, Sep 7, 12:55 PM

I'm still getting this with "normal" properties (ie not just wikibase:quantityUnit) on https://www.wikidata.org/wiki/Q334443 - it's showing up as a duplicate as 'P334443' in P39-based searches which return the item, and I can replicate this just with a targeted search by identifier (https://w.wiki/825).

I've tried editing the item, or deleting and recreating some claims, but it doesn't seem to affect the query results.

On the plus side, this is the only one I've found in a test run of ten thousand or so, so it seems to be a rare problem...

Lucas_Werkmeister_WMDE removed Smalyshev as the assignee of this task.Mon, Sep 9, 10:03 AM
Lucas_Werkmeister_WMDE raised the priority of this task from Normal to High.
Lucas_Werkmeister_WMDE added a subscriber: Esc3300.

Also affecting some calendar models (reported by @Esc3300).

See T232212#5472285 / Wikitech for update instructions (needs to be done by someone with WDQS production access, which I don’t have as far as I’m aware).