Page MenuHomePhabricator

[Shared Event Platform] Mediawiki Stream Enrichment should consume the consolidated page-change stream.
Closed, ResolvedPublic

Description

Currently (see T307959: [Event Platform] Design and Implement realtime enrichment pipeline for MW page change with content) we consume page_delete, revision_create and page_create events separately and consolidated them into a single, enriched, stream.

After T309784: [Shared Event Platform] Design and Implement POC Flink Service to consolidate a page-change stream on top of existing EventBus streams we should consume a consolidated page-change stream and enrich its events.

Wip schema at: https://gerrit.wikimedia.org/r/807565

Draft MR to the Mediawiki Enrichment Job at Draft: Consume events from page-change.

Dependencies on pending work
  1. Page change schema has been finalised https://phabricator.wikimedia.org/T308017
  2. events should be produced in the page-change stream https://phabricator.wikimedia.org/T311129
  3. wikimedia-event-utilities 1.2 should be released with https://gerrit.wikimedia.org/r/c/wikimedia-event-utilities/+/810387 merged
  4. Enriched page change schema has been finalized. https://phabricator.wikimedia.org/T311600
  5. Row operations leverage on the type system. https://phabricator.wikimedia.org/T316555
References

Related Objects

Event Timeline

gmodena updated the task description. (Show Details)
gmodena updated the task description. (Show Details)
gmodena updated the task description. (Show Details)