Page MenuHomePhabricator

Create RDF diff for WDQS updating
Closed, DuplicatePublic

Description

Currently, when an item is updated, we download the whole RDF export representation and load it into the database. But on wiki, there are diffs between two revisions, and Wikibase already supports diffing two Wikidata items and presenting the result. It may be possible to present the diff in RDF or SPARQL form in a way that would not require downloading the whole RDF and doing the diff on the server. This may save a lot of effort for small edits on huge items.

Of course, the diffing may be more fragile than current approach - since it would only touch pieces edited and if there's some error, editing would not longer restore whole entity. However, the increased speed may be worth it.

So I think it may be worth evaluating whether RDF/SPARQL diffing is possible and if yes, whether it's usable for WDQS purposes.

Event Timeline

Smalyshev created this task.Aug 1 2019, 5:46 AM
Restricted Application added a project: Wikidata. · View Herald TranscriptAug 1 2019, 5:46 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript