Status update:
Design doc is in Draft 1.0 state. and is ready for general review.

I will next try to create a shorter version of it for easier review, possibly a slide deck.

Briefly describe what was accomplished during the hypothesis work 

• WE5.2.6 and WE5.2.5 Event Modeling were worked on in parallel.  By considering how DomainEvents might be used outside of the originating process, we productively influenced the design and modeling of MediaWiki core DomainEvents.

• We wrote a comprehensive discovery and design document that explored technical details about how MediaWiki and other systems might utilize events to achieve better data integration capabilities.

• Recommendations:

Short term: Address the problem of serving off-wiki generated data with MediaWiki by one or both of:

1. Receiving events from Kafka with MediaWiki

2. Allowing non MediaWiki systems to submit jobs into MediaWiki's JobQueue

Long term: Fully integrate MediaWiki with WMF's Event Platform, making it possible to use event driven strategies to build products with MediaWiki.

• Categorized various cross-process scenarios.

• Highlights of questions considered and/or answered:

• How MediaWiki might manage event schemas.

• How MediaWiki might manage event stream datasets.

• How to approach building global WMF platform features with 1000 distinct MediaWiki sites.

• MediaWiki should not be responsible for cross-process events for 3rd parties and or data reusers.

• MediaWiki Domain Events System should not be responsible for PII redaction.

• How MediaWiki event broadcasting and serialization might be implemented.

• How events relate to the need for inter-wiki dependency tracking.

• Implementations considered:

• JobQueue based. Does not improve data reusability, but may more quickly enable some use cases where we are lacking capabilities.

• Event bus (kafka based).  Recommendations for how to achieve standardized broadcasting and receiving of events in MediaWiki.

• Reviewed the design doc with MediaWiki and product teams, data engineering teams, and P&T leadership.

• Domain Events - Cross Process Design Doc Overview

Outline the major lessons learned

• Use of off-wiki generated data in MediaWiki is difficult.  There is no standardized way to access or ingest data into MediaWiki systems for serving features to users. Building this capability would improve WMF's product velocity.

• Leadership is prioritizing more product features that need 'generated data', but teams are not given an incentive to work together with platform teams to make it easier for them to do so. Instead, teams implement their own solutions on a per-feature basis. This results in abandoned and unsustainable features due to brittle pipelines, unacceptably long update cycles, and steep learning curves with high costs for maintaining bespoke solutions.

• Quote from Marshall Miller, Sr. Director of Product: "Our data backed features are some of our best ones. In the past, they have also been our most laborious and take the longest to develop. Because of that, we don't plan to pursue as many of them as we would otherwise want to [...] The plans are inhibited by how long it takes to build them."

• See list of Relevant examples of data backed features below.

• Improving the other cross-process scenarios would also help with product velocity, but these scenarios have existent but manual ways to be achieved today (EventBus extension, JobQueue uses, etc.).

• There are many inter-wiki use cases, but the most valuable ones require fine grained dependency tracking, which is out of scope for a basic cross-process event system.

• In most cases, transclusions (images, wikidata, templates, etc.) have a many-to-few relationship.  E.g. an image is used on only a few pages.  PubSub / event driven architectures are better suited for few-to-many, e.g. a global user block should be applied on all wikis.

• MediaWiki at WMF being 1000 different sites makes data integration and global platform features complicated.

• 'Domain events' are different than 'integration events'.

• Domain events are events within a domain.  

• Integration events are for data integration between domains.

• This resulted in some confusion and tension when we tried to map MediaWiki PHP based DomainEvents models and concepts into other systems.

• It is not clear if MediaWiki to MediaWiki events are domain events or integration events. 

• Are each 1000 different sites in the same or different business domain? 

• If they are the same domain, they are domain events. 

• If they are different domains, then they are integration events.  

• This lack of clarity muddles architecture discussions and decisions.

• To avoid future confusion, we are going to rebrand the 'cross-process DomainEvents' term used in this hypothesis, and instead refer to this body of work as "Cross-Service Event Integration".

If applicable, suggest potential next steps

• Redraft design doc using 'cross-service' rather than 'cross-process Domain Events'.

• Publish design document to mediawiki.org. (Will likely happen in Q1 FY2025-2026).

• Verify feasibility of using a PHP Kafka Consumer in MediaWiki. This work will be covered in WE5.2.13:

• First as a local PoC to see how this might be possible.

• If this proves to be possible, engage with SRE ServiceOps to explore the pros and cons of this idea more deeply.

• Propose a project that could be used to justify building out off-wiki generated data serving capabilities.

• Review AddALink and Image Suggestion data pipelines to understand how events may or may not have improved those features.

• Review FY2025-2026 KRs to demonstrate where off-wiki generated data needs a pipeline back to MediaWiki (and where events may help).

• Socialize the cross-process event potential with those teams.

• Determine if there is a good fit for either initial experiments or full cross-team commitment.

• Continue discussing the need for a data usage culture change:

• Leadership is prioritizing more product features that need 'generated data', but teams are not given an incentive to work together with platform teams to make it easier for them to do so.

• If leadership does not understand the difficulties teams face when building data-backed features, it will be difficult to justify improving our platforms.

• MediaWiki Cross Service Integration Events is just one potential solution that attempts to address this problem.  It may not help all potential data-backed features. There are likely other platform improvements we could make.  

• Until we address the problem, we will:

• Avoid building novel data-backed features because they are too onerous.

• Repeatedly expend resources building hacky and brittle data pipelines that work around the problem.

Relevant examples of data backed features

• User facing features in MediaWiki that use LiftWing generated scores

• AddALink: brittle data pipeline, including sqlite dump files and single bare metal public fileservers

• ImageSuggestions: rejected suggestions take a month to be removed from being served.

• It is difficult for Enterprise (and others) to backfill generated data

• T240387 MW REST API Historical Data Endpoint Needs was not done because of difficulty.

• T341649 Provide an easy way for MediaWiki to fetch aggregate statistics from the data lake

• T309738 Move Mediawiki QueryPages computation to Hadoop

• [T388455 [Spike] Full-year editing stats for Year in Review through Impact Module API](https://phabricator.wikimedia.org/T388455)

• T325790 Special:ContentTranslationStats is slow and getting crowded

• T221397 Store page-links-change data in a database table and make available through a Special page

• Similarusers - Wikitech

• [T379119 [Spike] Fetch Topics for Articles in History on iOS app](https://phabricator.wikimedia.org/T379119)

• [T385502 [Hypothesis] WE3.1.12 Article summaries: mobile summary pilot](https://phabricator.wikimedia.org/T385502)