Page MenuHomePhabricator

EventLogging MEP Upgrade Phase 3 (Stream cc-ing)
Closed, DeclinedPublic

Description

Phase 3 (Stream cc-ing)
Streams which share prefixes are automatically logged to when the prefix is logged to.

As documented in https://www.mediawiki.org/wiki/Wikimedia_Product/Analytics_Infrastructure/Stream_configuration#Stream_cc-ing

On the mobile apps, it's trivial to generate the "cc map" from an array of stream names once the stream configuration is downloaded at the start of the session and is kept in memory until the app is closed (e.g. https://github.com/wikimedia/wikipedia-ios/pull/3603/files#diff-090a2618a1a359b3ab62798303960325R362-R374)

On MediaWiki, generating the map shouldn't be done on every pageload and should be taken care of by EventStreamConfig extension: T256169

Event Timeline

mpopov moved this task from Inbox to Done! on the Better Use Of Data board.

@mpopov Seeing that recent instrumentation like T250282: Build sidebar instrumentation is still not using MEP I think we should focus on adoption rather that adding more features to EL client for now.

That's fair, plus THE use case for this feature is Growth team's experiments and their issues with EditAttemptStep.

LGoto triaged this task as Medium priority.Jul 7 2020, 5:20 PM

Change 618306 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[mediawiki/extensions/EventLogging@master] [WIP] Generate Stream CC Map, CC streams in submit()

https://gerrit.wikimedia.org/r/618306

We've decided to table this feature for now until we truly need it, but I uploaded what I had so far. Namely the PHP version of the stream CC map generator, the necessary modifications to mw.eventLog.submit(), and tests.

For example, suppose you have streams "edit.growth" and "edit.mobile" in the stream config but not "edit".

What happens if an instrumentation produces directly to "edit.growth"? Is that ok? What happens if "edit.growth.experiment" is also in stream config?

I think if an instrument produces directly to "edit.growth" that would be okay too. An example I have in mind is if – in addition to "edit.growth" getting regular editing workflow events via the CC feature and, say, VisualEditor instrumentation producing events to "edit" stream – there could be a Growth-specific gadget/widget that emits Growth-specific edit events.

Generating whatever this is in the EventLogging extension should be fine. I'd be careful about using some a naming convention to figure out what gets 'CCed' to what though. It'll probably work in most cases, but I'd predict you'll run into some unexpected bugs when people start doing weird things with it.

You could make up another EventLogging config variable for this where you keep an explicit map?

If we end up actually getting this feature in as it is (implicit map generation based on naming convention) it's definitely one of those "with great power comes great responsibility" types of things where people are gonna have to be very careful with deciding how they want to name their streams. Explicit mapping in the config is also a valid approach. Or we may just scrap it if it turns out this was a bad idea from the start.

Change 618306 abandoned by Bearloga:
[mediawiki/extensions/EventLogging@master] [WIP] Generate Stream CC Map, CC streams in submit()

Reason:
Likely going to use a manual, rather than automatic solution for stream cc-ing feature when the time comes

https://gerrit.wikimedia.org/r/618306

We abandoned the idea of stream cc-ing; this idea is vaguely present in some of the conceptualization behind stream configs in Metrics Platform.