Page MenuHomePhabricator

[2.3] External links/references event stream
Open, NormalPublic

Description

Output
We will create a robust, real-time event stream tracking the creation and modification of external links and references across Wikimedia projects. This data stream will provide tool developers, content partners, and other data consumers (libraries and GLAM institutions, metadata organizations, researchers, altmetrics providers) a canonical data source to track and contribute to the sourcing work of Wikimedia volunteers. This is a dependency for the link rot initiative [2.6]

Target
An event stream tracking the creation and modification of external links and references across Wikimedia projects is delivered.

Primary team: Research

Initial notes: https://etherpad.wikimedia.org/p/RefEvents

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 10 2018, 9:32 AM

Related tasks at T115119 and T102064.

DarTar triaged this task as High priority.Jul 12 2018, 3:29 PM
DarTar lowered the priority of this task from High to Normal.
DarTar updated the task description. (Show Details)Sep 20 2018, 9:29 PM
Izno added a subscriber: Izno.Nov 6 2018, 6:50 PM
Nuria added a subscriber: Ottomata.Jan 25 2019, 5:50 PM
Nuria added a subscriber: Nuria.

You are aware of eventbus/event gate and the fact that these events can be published directly from mediawiki correct? The event streams we have for changes and similar do not use eventlogging but rather events are sent directly from mediawiki's backend when they occur. Schemas for those events can be found here: https://github.com/wikimedia/mediawiki-event-schemas/tree/master/jsonschema/mediawiki

Also please see: https://www.mediawiki.org/wiki/Wikimedia_Technology/Annual_Plans/FY2019/TEC2:_Modern_Event_Platform

And: https://github.com/wikimedia/eventgate

We are happy to help you CR your changes as needed be, just let us know if you need a meeting to come up with a plan on the steps needed to accomplish what this ticket wants to do.