Page MenuHomePhabricator

EventBus - move event producers to a different extension
Open, LowPublic

Description

The EventBus MediaWiki extension is both a backend library and user of its own backend library.

The main EventBus interface is a PHP API for producing Event Platform events to eventgate.

The EventBus extension also contains event producer code. This code subscribes to various MediaWiki Hooks and DomainEvents and translates them into JSON events, and then an EventBus class instance to produce the events.

This has been the status quo since EventBus extension's initial creation 11 years ago. It hasn't been a practical problem until now.

As more event streams are being created and more data fields being added, EventBus is starting to add dependencies on other extensions.

  • T430967 is adding a dependency on CentralAuth
  • T428176 is adding a dependency on WikiBase.

If EventBus depends on another extension, that means that extension can't depend on EventBus. Event producer logic should move to another extension so that EventBus can be used as a standalone library.

Done is

  • Event producer logic is no longer in EventBus extension
  • EventBus extension does not depend on other extensions that might themselves want to depend on EventBus.

Event Timeline

How about the WikimediaEvents or WikimediaCustomizations extensions?

Also, if EventBus is to be a standalone library for producing Event Platform events to EventGate, should the logic for producing events from the frontend in the EventLogging extension also be relocated?

How about the WikimediaEvents or WikimediaCustomizations extensions?

Ya something like that!

should the logic for producing events from the frontend in the EventLogging extension also be relocated?

I'm not sure! I'm not opposed, especially if we can divorce it from anything EventLogging specific and make it match the EventBus PHP send API.

But, maybe that could be a different task if we want to do it? The priority for me is lower though, since it doesn't help the extension dependency problem.

Ottomata added a project: good first task.

Thank you for tagging this task with good first task for Wikimedia newcomers!

Newcomers often may not be aware of things that may seem obvious to seasoned contributors, so please take a moment to reflect on how this task might look to somebody who has never contributed to Wikimedia projects.

A good first task is a self-contained, non-controversial task with a clear approach. It should be well-described with pointers to help a completely new contributor, for example it should clearly point to the codebase URL and provide clear steps to help a contributor get set up for success. We've included some guidelines at https://phabricator.wikimedia.org/tag/good_first_task/ !

Thank you for helping us drive new contributions to our projects <3

Anushka10patel subscribed.

Hi! I'm a new contributor, and I'd like to work on this task. I'll first investigate the codebase and understand the current implementation before preparing a patch.

Hi @Ottomata! I've investigated the current implementation. I found the event producer logic in PageChangeEventIngress, CampaignChangeHooks, and UserChangeHooks, all of which are currently registered from extension.json. Before I prepare a patch, could you confirm which extension these producers should be moved to (for example, WikimediaEvents or another extension)?

Hi wow thank you!

Before I prepare a patch, could you confirm which extension these producers should be moved to (for example, WikimediaEvents or another extension)?

A great question. I think WikimediaEvents would be a good theoretical home for this, but its purpose currently is very instrumentation/analytics focused. Hm. We might need a little consultation. I'll ask internally how to proceed and get back to you...

Okay, yes! Confirmed with some folks. WikimediaEvents is probably the most appropriate place.

I found the event producer logic in PageChangeEventIngress, CampaignChangeHooks, and UserChangeHooks

These are indeed what we want to migrate! We may also consider the RCFeed Adapter, but let's decide that later.

WikimediaEvents has a lot of other stuff, so let's keep these as isolated as possible. These are 'mediawiki state change events', so perhaps in includes/MediaWikiState or some subdirectory(?) (suggestions welcome!).


Hm ... I am now seeing a potential complication. Will post in next comment...

There are two existent top level 'change event serializers', e.g. PageChangeEventSerializer and UserChangeEventSerializer. These are 'private' serializers, in that we don't ever expect anything but this specific producer code to use them. They should be migrated with the Ingress&Hooks stuff.

However, the various *EntitySerializers are 'public' and are reused by other extensions out there, e.g. CirrusSearch, CheckUser, and...WikimediaEvents!

Where should these reusable MediaWiki entity serializers live? I suppose they could also live in WikimediaEvents. Should they? Hm. I'm not sure.

Additionally, I think the very generic EventSerializer should probably stay in EventBus, as it is concerned with common Event Platform rules and conventions for all events.

Given these open questions and decisions to make, I'm not sure this is such a great good first task after all. @Anushka10patel, what do you think?

Hi @Ottomata, thanks for looking into this and for discussing it with the team.

I understand the architectural questions around where the reusable serializers should live and how to keep the responsibilities of EventBus and WikimediaEvents well separated. I'm still interested in working on this task and would be happy to help with the migration while following the direction you and the team decide.

To keep the scope manageable, I can start by moving the producer logic along with the private serializers (PageChangeEventSerializer and UserChangeEventSerializer) into WikimediaEvents in an isolated MediaWikiState namespace, and leave the reusable entity serializers and the generic EventSerializer untouched for now. If that sounds like a reasonable first step, I'd be happy to prepare a patch and iterate based on review.

I can start by moving the producer logic along with the private serializers (PageChangeEventSerializer and UserChangeEventSerializer) into WikimediaEvents

We could do this, but I think we should probably decide what we want to do first. I've realized that the potential circular dependency coupling is not (just) the top level event producer code; but in the common Serializer services themselves. E.g. the two (yet unmerged!) tasks mentioned here (T430967 and T428176) are adding a dependency on Wikibase from PageEntitySerializer and on CentralAuth from UserEntitySerializer. If Wikibase or CentralAuth decide that they'd like to emit events using these serializers (like CheckUser and CirrusSearch are doing), they won't be able to.

Perhaps...especially since those patches are not yet merged, I should see if I can accomplish them without adding dependencies to those Serializers. And, if I do that, then the motivation for this ticket is less, and its priority drops a lot.

following the direction you and the team decide.

Given all that, I will try to remove those extension dependencies. And if we keep EventSerializer in EventBus, we should probably keep the common EntitySerializers in EventBus too. So, that would mean we'd consider moving only the concrete (private) *ChangeEvent* producer related code to WikimediaEvents. Something will have to depend on these extensions, that that will probably be these things.

Let me attempt to do this first before you take action to move the *ChangeEvent* code.

Thanks for the clarification! That makes sense.

I understand the concern about introducing extension dependencies and the potential coupling around the shared entity serializers. I'll wait until you've had a chance to simplify that part of the architecture before starting the migration of the *ChangeEvent* producer code.

Once you've updated the task or confirmed the direction, I'll be happy to continue working on it. Thanks for the detailed explanation!