Page MenuHomePhabricator

[2.3] External links/references event stream
Closed, ResolvedPublic

Description

Output
We will create a robust, real-time event stream tracking the creation and modification of external links and references across Wikimedia projects. This data stream will provide tool developers, content partners, and other data consumers (libraries and GLAM institutions, metadata organizations, researchers, altmetrics providers) a canonical data source to track and contribute to the sourcing work of Wikimedia volunteers. This is a dependency for the link rot initiative [2.6]

Target
An event stream tracking the creation and modification of external links and references across Wikimedia projects is delivered.

Primary team: Research

Initial notes: https://etherpad.wikimedia.org/p/RefEvents

Event Timeline

DarTar lowered the priority of this task from High to Medium.
Nuria subscribed.

You are aware of eventbus/event gate and the fact that these events can be published directly from mediawiki correct? The event streams we have for changes and similar do not use eventlogging but rather events are sent directly from mediawiki's backend when they occur. Schemas for those events can be found here: https://github.com/wikimedia/mediawiki-event-schemas/tree/master/jsonschema/mediawiki

Also please see: https://www.mediawiki.org/wiki/Wikimedia_Technology/Annual_Plans/FY2019/TEC2:_Modern_Event_Platform

And: https://github.com/wikimedia/eventgate

We are happy to help you CR your changes as needed be, just let us know if you need a meeting to come up with a plan on the steps needed to accomplish what this ticket wants to do.