Page MenuHomePhabricator

mw.wikibase.entity:getBacklinks (lua API in wikibase client)
Open, MediumPublic

Description

Use cases:

  1. I want to be able to automatically produce pages like https://en.wikipedia.org/wiki/Zaikin (based on P734 sentences in wikidata)
  2. I want to be able to automatically produce lists like https://en.wikipedia.org/wiki/List_of_Aloe_species (based on P171 statements in wikidata)

I think I have everything to implement that except for ability to get all elements, that links to the specific one (similar to https://www.wikidata.org/wiki/Special:WhatLinksHere/Q18390678). This functionality exists in Mediawiki API (query/links), but does not exists in wikibase client. The output of mw.wikibase.entity:getBacklinks() can be just a table of qids or (preferrably) qids+propertyIds

Event Timeline

Ghuron created this task.Jan 19 2018, 12:37 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 19 2018, 12:37 PM
Restricted Application added a project: Wikidata. · View Herald TranscriptJan 19 2018, 1:49 PM
Uzume added a subscriber: Uzume.EditedJan 20 2018, 8:44 PM

This seems like a more generalized version of T99899 (see also T149108).

I like the idea of a Wikibase client reverse claim query interface that returns entity IDs (but I am not sure it makes sense to hang this off an Lua table serialized entity object). One of the key tenets of Wikidata was always to provide a means to create automatically generated lists of related items.

This seems like a more generalized version of T99899 (see also T149108).

Well, although I can some similarities between this one and T99899, I believe there are different use cases involved. Lookup for external identifier is mostly needed in javascript (e.g. notify user that wikidata instance with the same imdb-id already exists) and can be implemented either via Markus resolver or directly via sparql endpoint (e.g. https://ru.wikipedia.org/w/index.php?diff=84445799). I can imagine some use cases where it would be nice to have it in lua (e.g. for additional decoration of wikipdate articles), but its exotic stuff. In contrast, building lists andbased on wikidata is more straightforward case and must be pure severside (for performance reasons), so I'd see a need for backlinks in lua.

Uzume added a comment.Jan 23 2018, 4:02 PM

Well, although I can some similarities between this one and T99899, I believe there are different use cases involved. Lookup for external identifier is mostly needed in javascript (e.g. notify user that wikidata instance with the same imdb-id already exists) and can be implemented either via Markus resolver or directly via sparql endpoint (e.g. https://ru.wikipedia.org/w/index.php?diff=84445799). I can imagine some use cases where it would be nice to have it in lua (e.g. for additional decoration of wikipdate articles), but its exotic stuff. In contrast, building lists andbased on wikidata is more straightforward case and must be pure severside (for performance reasons), so I'd see a need for backlinks in lua.

That may or may not be true but this issue seems extremely similar to that requested in T149108 which was closed as a duplicate of T99899 and thus I drew the relation between the two.

hoo moved this task from Incoming to To do later on the Wikibase-Lua board.Apr 12 2018, 1:56 PM
Ghuron triaged this task as Medium priority.Apr 12 2018, 8:57 PM
Ghuron added a project: patch-welcome.

I'd like to explain another scenario, where this functionality can be useful. Ru-wiki infoboxes sometimes add categories to the corresponding articles. Sometimes we do this based on wikidata, but we are pretty much limited to a few cases, defined in P1465, P1792, P1464 or very special usage of P910 (e.g. if astronomical object belongs to some constellation). If I want to automatically place categories like Q8454640 to the films, directed by James Marsh, I probably would need a designated property. Alternatively I should be able to do it via extracting items that has P971:Q2720543 and simultaneously P971:Q29017630. This may sound like reinventing SPARQL queries, but getting results of getBacklinks(Q2720543) and getBacklinks(Q29017630) and performing list intersection in lua would work for me.

Townie added a subscriber: Townie.May 25 2018, 8:29 AM

This task seems similar to T199887, at least judging from the examples given in the task description (haswbstatement:P734=Q18390678 and haswbstatement:P171=Q127134).

lucamauri added a subscriber: lucamauri.

As this function would be very significant for a project of mine, I would like to work on it.
I am willing to put effort in implementing such a function, but I have some difficulties to get started with it.
I read above that the function already exist in Wikibase API, so I started by looking at the documentation here https://doc.wikimedia.org/Wikibase/master/php/classWikibase_1_1Repo_1_1Api_1_1QuerySearchEntities.html but it seems I am a little lost.
Can somebody point me in the right direction so then I can start the work on it?

Thanks for any help you might provide

hoo added a subscriber: hoo.Thu, Nov 28, 4:58 PM

So I just looked into this a bit and sadly this is not a trivial change. First of all, I don't think we want to expose backlinks (as in Special:WhatLinksHere) as these don't have the semantics the use cases here call for (and using them still would probably lead to strange behavior in the long run). That's because these are just all links pointing to a page, no matter where these originate from: statement (main) values, reference values, qualifier values (and probably even more).

What I could imagine would be to expose (something similar to) haswbstatement via Lua in the client, but only if we can reasonably assure the performance and uptime of the used backend (this is going to run on page render after all!). Combining that with restrictive usage limits (function can only be used n times per page), usage tracking, query result caching and potentially cache purging on changes (run the query ever once in a while to catch for changes?), might be a way to go here. But that's just a quick thought.

Catalan Wikipedians are also waiting for this feature to be available for our Wikidata-powered infoboxes. I spoke with @alaa_wmde about that during the 2019 Wikimedia Hackathon, and he sadly thought it was nearly impossible. Maybe he can comment on @hoo's proposal.

Uzume added a comment.EditedMon, Dec 9, 4:53 PM

I agree that Special:WhatLinksHere is probably not the right semantics for this request, haswbstatement might be better semantics, however, those need to be well defined so people know if in fact they would address this request.

For example, I assume this would return statements where the claim value is a specific item (possibly filtering by a specific property of item type), however, remember items can be values for properties that appear in claim qualifiers and statement references too.

Special:WhatLinksHere would capture all such statements and more.

Instead of focusing on only item values, perhaps a Lua-based statement query is more useful (and could cover more property types than just items). I am not sure how feasible it is to consider queries during page render but the general idea is a Lua-based query that is more restrictive than SPARQL (so it can run within page render).

One possibility I can envision would be to take the query out of the page render and instead making it the result of some sort of cached value that can be obtained via Scribunto Lua. Perhaps we need another service, like a bot that runs such queries and stores/caches the results. Looking at what is available to Scribunto Lua, I am thinking of a service that pulls a SPARQL queries from Tabular Data (:Commons:Data: via Extension:JsonConfig) and returns the results there. The service bot could periodically patrol Tabular Data looking for and executing such queries and updating the results when they change. Scribunto Lua modules could then obtain the results of these queries. There is probably lots of room for optimization but that is at least one vision of mine on how to generalize and solve this problem.

The only reason I mention Special:WhatLinksHere is that I thought it would be easier to implement, which doesn't seems to be the case.
If some can expose reasonable subset of haswbstatement - I would be more than happy.