Deploy Mediawiki Stream enrichment on YARN
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	gmodena
	Nov 10 2022, 12:34 PM

Description

Task to document and coordinate deployments of https://phabricator.wikimedia.org/T311084 on YARN.

Mentioned In: T320812: [SPIKE] Deploy event driven stateless Flink service to DSE cluster
Mentioned Here: T311084: [Shared Event Platform] Mediawiki Stream Enrichment should consume the consolidated page-change stream.

gmodena updated the task description. (Show Details)

Work covered in this phab:

Make flink install on stat nodes reproducible: a script to automate download and config of a Flink dist is available at https://gitlab.wikimedia.org/-/snippets/41
Gitlab CI should produce deployable artifacts (needed also for DSE deployments): MR at https://gitlab.wikimedia.org/repos/data-engineering/mediawiki-stream-enrichment/-/merge_requests/17
Service restarts and failure recovery strategy should be documented: created doc at https://www.mediawiki.org/wiki/Platform_Engineering_Team/Event_Platform_Value_Stream/PoC_Mediawiki_Stream_Enrichment