Page MenuHomePhabricator

[Task] Create dump of entity redirects (JSON or n-triples)
Open, NormalPublic

Description

There should be a dump that contains all redirects, one redirect per line (similar to the entity dumps).

In JSON, such a line would probably look like {"from":"Q123","to":"Q456"}.
In n-triples, it would be wd:Q123 owl:sameAs wd:Q456.

This would allow consumers of our JSON dumps to resolve redirects.

Details

Related Gerrit Patches:
mediawiki/extensions/Wikibase : masterAllow creating redirect only dumps with dumpRdf

Related Objects

View Standalone Graph
This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.

Event Timeline

daniel created this task.May 6 2015, 12:33 PM
daniel raised the priority of this task from to High.
daniel updated the task description. (Show Details)
Jonas renamed this task from Include redirects in JSON dumps to [Task] Include redirects in JSON dumps.Aug 15 2015, 12:27 PM
Jonas set Security to None.

a separate would avoid breaking the assumption of "one line, one entitiy" for the current dumps

hoo renamed this task from [Task] Include redirects in JSON dumps to [Task] Create a redirect JSON dump.Apr 9 2017, 7:44 PM
hoo updated the task description. (Show Details)

Reworded the task in accordance to what we discussed this week.

This is actionable now.

daniel added a comment.Apr 9 2017, 9:26 PM

I'm not sure we really need a JSON dump of redirects. We could directly generate an ntriples dump instead, perhaps even using the existing RDF dump script.

daniel lowered the priority of this task from High to Normal.May 10 2017, 4:26 PM

One reason to provide such a dump is for WikiDataToolkit to be able to handle redirects.

daniel renamed this task from [Task] Create a redirect JSON dump to [Task] Create dump of entity redirects (JSON or n-triples).May 10 2017, 4:53 PM
daniel updated the task description. (Show Details)

Change 354470 had a related patch set uploaded (by Hoo man; owner: Hoo man):
[mediawiki/extensions/Wikibase@master] Allow creating redirect only dumps with dumpRdf

https://gerrit.wikimedia.org/r/354470

Change 354470 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Allow creating redirect only dumps with dumpRdf

https://gerrit.wikimedia.org/r/354470

hoo added a subscriber: ArielGlenn.Feb 21 2018, 5:23 PM
hoo added a comment.Feb 21 2018, 5:29 PM

Hm… doing these redirect dumps separate from the current dumps will mean that the complete dumps might be inconsistent (as both aspects are dumped at separate times).

Maybe we should change dumpJson.php so that it emits both the redirect and the "usual" JSON side-by-side. Or we just include it in the original JSON?