Page MenuHomePhabricator

Implement enriched revision visibility stream
Closed, DeclinedPublic

Description

For Dumps 2.0, we would like to have a stream that resolves the user/comment/content details when a flip of visibility happens from FALSE to TRUE. We have the Flink Stream Processing mechanism, so we can help ourselves with guidance from the Event folks. Hopefully such an enriched stream is also useful to other folks that would like this problem be solved upstream of their processes. This enriched stream should probably be compatible with the limited schema discussed in (1) over at T349845#9334970, so that in the event that (1) is implemented, we don't need to reimplement the enriched stream.

(Further context at T349845#9334970)
(Note that even though it would be nice to have T351565 first, that other task does not block this one)

In this task we should:

Event Timeline

T349845#9587239

You know, perhaps we need a more generic revision_change stream, akin to mediawiki.page_change.v1. This would often be redundant with page_change (when page_change_kind == 'create' or ' edit'), but would allow us to represent more than just visibility state changes of a revision.

In today's 'Dumps 2.0 Architecture Discussions' meeting, we decided not to pursue this mechanism.

We will instead treat this issue as part of what we look for when doing T358373: [Dumps 2] Reconciliation mechanism to detect and fetch missing/mismatched revisions.