Change Details

This should be a quick spike to understand how hard it would be to replicate mw stream enrichment with pyflink. The goal is to; 1. Run a read-only python implementation of Mediawiki Stream Enrichment on YARN (https://gitlab.wikimedia.org/-/snippets/42). 2. Collect resource allocation and latency metrics for a long running pyflink job. 3. Help inform integration paths with the upcoming Flink catalog. https://phabricator.wikimedia.org/T322022. 4. Help requirement collection for https://phabricator.wikimedia.org/T322125 .

This should be a quick spike to understand how hard it would be to replicate mw stream enrichment with pyflink. The goal is to; 1. [x] Run a read-only python implementation of Mediawiki Stream Enrichment on YARN (https://gitlab.wikimedia.org/-/snippets/42). 2. [x] Collect resource allocation and latency metrics for a long running pyflink job. 3. [x] Help inform integration paths with the upcoming Flink catalog. https://phabricator.wikimedia.org/T322022. 4. [x] Help requirement collection for https://phabricator.wikimedia.org/T322125 .

This should be a quick spike to understand how hard it would be to replicate mw stream enrichment with pyflink. The goal is to; 1. [x] Run a read-only python implementation of Mediawiki Stream Enrichment on YARN (https://gitlab.wikimedia.org/-/snippets/42). 2. [x] Collect resource allocation and latency metrics for a long running pyflink job. 3. [x] Help inform integration paths with the upcoming Flink catalog. https://phabricator.wikimedia.org/T322022. 4. [x] Help requirement collection for https://phabricator.wikimedia.org/T322125 .