Change Details

As a consumer of the wikidata content I want to be able to have access to the same RDF data the WMF WDQS servers use to perform their live updates so that I can keep my own replica of the wikidata query service (or another RDF store) up to date more easily. A solution might be to use the [[https://wikitech.wikimedia.org/wiki/Event_Platform/EventStreams|EventStreams]] service. Note on the stream: It was decided to go fully active/active for the flink application powering the WDQS updater. Which means the complete stream of changes is available in both topics: - eqiad.rdf-streaming-updater.mutation - codfw.rdf-streaming-updater.mutation It is sligthly different to what we currently see in our topic topology where if you want to have a complete view of the data you need to consume both eqiad.topic and codfw.topic. Here you must consume only one. AC: - RDF data is exposed through EventStreams - A java client is offered for third parties to use with store comptatible with the [[https://www.w3.org/TR/2013/REC-sparql11-protocol-20130321/#update-operation|SPARQL 1.1 Update operations]]

As a consumer of the wikidata content I want to be able to have access to the same RDF data the WMF WDQS servers use to perform their live updates so that I can keep my own replica of the wikidata query service (or another RDF store) up to date more easily. A solution might be to use the [[https://wikitech.wikimedia.org/wiki/Event_Platform/EventStreams|EventStreams]] service. Note on the stream: It was decided to go fully active/active for the flink application powering the WDQS updater. Which means the complete stream of changes is available in both topics: - eqiad.rdf-streaming-updater.mutation - codfw.rdf-streaming-updater.mutation It is sligthly different to what we currently see in our topic topology where if you want to have a complete view of the data you need to consume both eqiad.topic and codfw.topic. Here you must consume only one. AC: - RDF data is exposed through EventStreams - A java client is offered for third parties to use with stores comptatible with the [[https://www.w3.org/TR/2013/REC-sparql11-protocol-20130321/#update-operation|SPARQL 1.1 Update operations]]

As a consumer of the wikidata content I want to be able to have access to the same RDF data the WMF WDQS servers use to perform their live updates so that I can keep my own replica of the wikidata query service (or another RDF store) up to date more easily. A solution might be to use the [[https://wikitech.wikimedia.org/wiki/Event_Platform/EventStreams|EventStreams]] service. Note on the stream: It was decided to go fully active/active for the flink application powering the WDQS updater. Which means the complete stream of changes is available in both topics: - eqiad.rdf-streaming-updater.mutation - codfw.rdf-streaming-updater.mutation It is sligthly different to what we currently see in our topic topology where if you want to have a complete view of the data you need to consume both eqiad.topic and codfw.topic. Here you must consume only one. AC: - RDF data is exposed through EventStreams - A java client is offered for third parties to use with stores comptatible with the [[https://www.w3.org/TR/2013/REC-sparql11-protocol-20130321/#update-operation|SPARQL 1.1 Update operations]]