As asked, I diagrammed a proposal here.

My personal preference: fewer job (business logic) repos are better (for sharing and deployment reasons). Repo organization for libraries is a more of a per-case basis. For our stuff, a lot of what is in refinery and refinery-source could probably be in the same repo. But I (think?) I'm glad that wikimedia-event-utilities is a separate repo.

Thank you so much for this! I concur with all of the options and pros and cons! let's discuss!

I think it might work, we'll need to make changes to the AQS service

Aye. Alternatively...you could make this /edits/v4/per_editor and intend to deprecate the v3 one, and do so with some backwards compat routing.

And, FWIW, once patches for T436695 land and are in use, I'll update wikitech docs with instructions for how to do all that ^ :)

+1 ya that makes sense. Any alteration to the default auto created topic.

how do we want to model event platform events emitted to Kafka?

Yesterday, @AKhatun_WMF and I discussed LinkedArtifacts: rename precompute to prefetch; no-cache on reconcile (1354866) ideas (motivation in this comment).

Remove event production

Since this will be a fully new stream, I think we can do this slowly after the new prediction event stream is fully live. I don't know what the uses of mediawiki.page_revert_risk_wikidata_prediction_change.v1 (or other LiftWing emitted streams) are, so it might take us a while to actually decom them. We might want to do so as a larger migration to LAC off of change-prop effort.

Approved

Hi thanks! Just so I understand: Only topics with any non default settings will be declared, correct?

Oh! the _weekly tables are incremental?

Change-prop doesn't emit it, LiftWing does. Change-prop only consumes mediawiki.page_change.v1, filters it, and POSTs the whole page_change event to the isvc. The model server then builds the event and sends it to EventGate itself

WMCS webrequests will not be the same as the ones in production

Okay, that is a good reason not to use the CDN webrequest schema. Would the ECS one work for you? It is being used for nginx access logging now. Perhaps not because you need to add some custom enriched fields? If so, then we can just make a new schema, based on an existent one or fully custom.

Thanks Neil!

Hm, could be okay. Let's see what others think. Since there are only 2 of these (right)?, both are new, and will be very useful and I expect will have more consumers as time goes on, it might be worth a quick rename now.

Also, thank you for the decision documentation!

The existent endpoint is /edits/v3/per_editor/{user_central_id}/{page_type}/{granularity}/{start}/{end}.

I would start by increasing replicas and just seeing what happens. If it gets better, than call it a day.

Of all those options, I think I prefer: T314956 + WMCS produces to eventgate-analytics-external.

if the logs should be sent via EventGate

I think:

Probably too late, but I think (am I wrong) to associate suffixes like _daily with aggregation granularity, not import cadence. I guess we're already there with logging_daily, but before we go too far, are there any other incremental sqoop tables that we do this for that are named like this?

Sweet! Fine with me then. My frequent sqoop objections are all about snapshot sqoops.

Discussed a bit with Thomas, and we may want to implement this somehow inside of node-rdkafka-factory.

We cannot use the mediawiki_revision_history_v1, because the event.mediawiki_page_change_v1 is a direct source for it, and therefore we would compare the table to itself. Same thing for the MWCHv1 table.

Also, FWIW, T436695: Linked Artifact Cache - Automated MediaWiki JobQueue precompute patches are in progress and I expect them to be merged and put to use by the end of November, if not earlier.

Also: I'm a little confused about the circular dependency. I understand that LiftWing itself is serving both HTTP and gRPC requests, but aren't those different (and isolated?) endpoints? I would imagine it was like this:

Thanks! How does change_prop emit the mediaiki.page_revert_risk_wikidata_prediction_change.v1 event?

Let's drop it. If someone really needs to know system user status, they can find that info elsewhere.

I understand the need for logging though: MWCH doesn't have non 'new revision' events, and MWH doesn't have revision visibility change events.

Oh my I hope not!

This makeProduce function is where you'll want to implement the metric! You can see that it is creating a produced events counter metric here, and then in the actual returned produce function incrementing it here. This should look similar :)

Hi, I understand very little about the ResultTerm idea, but FWIW:

However because it's sanitized we revision.editor.user_central_id is NULL

FWIW, I don't think this is NULL because its sanitized, but because we didn't start setting this until Oct 2025.

Oh! Today I realized that we can use a distinct job type for each artifact precompute job, while still re-using the same job class. And, I can do this dynamically with the LinkedArtifactsConfig.

Yup!

Yeha, I think we may still emit RevisionLinkedArtifactComputedEvent, but with the 'trigger' set appropriately.

LAC clients would need a mechanism to wait async

Okay, my 'friend' and I did some research on the overlaps between the intention of page_change_kind and cause.

Approved, this makes sense to me. These are the folks that would have to administer dumps as part of Data Engineering Ops Week duties, and otherwise.

Related: T415283 & T418190

+1 ty!

Oh okay interesting!

Purely on LAC HTTP response. Last-Modified response header will very slightly change format, so if anyone is parsing it, it could break.

I highly doubt anyone is examining Last-Modified from LAC yet.

I don't know why HTTP prefers GMT over UTC. UTC seems more standard to me. I guess internet history?

🎉

the timeout has been lowered to 11s in the service mesh

Oh, nevermind. That was only for eventgate-main. eventgate-analytics service mesh timeout was and is 10s.

Hm, your time range is last few months, so this might just be noise, but: T439552#12374341

Just to note as an initial follow-up post outage the timeout has been lowered to 11s in the service mesh, but that was just an initial softening and isn't an informed change.

@mpopov @phuedx btw, for T436695+T432604 (and soon for T438491), we will be adding some 'event data product' producer code to WikimediaEvents. Previously we had considering namespacing this code in some MediaWikiState directory. However, that term is a bit to constraining, in this WIP patch, I am going with EventDataProducts. Whatcha think?

Hm. I was imagining just hiding LAC behind whatever endpoint you expect LiftWing HTTP callers to call.

Even KISSer why not!

We could probably decline this or merge into T436785: Rename python-webapp to something generic, intending to use that.

Just tried it! It works!

I'm working on patch 2 and patch 3 together. patch 3 will live in WikimediaEvents. It will

• subscribe to e.g. RevisionLinkedArtifactPrecomputeEvent DomainEvent
• construct a new event platform event (data model TBD)
• Emit feature counts computed event to event platform via EventBus

I think deterministic from meta.id should work, no?

schema mismatch

Actually, I don't know which is better. I guess since we already have the delta counts published we should use those? I just want them to match, and changing delta counts now would be annoying eh?

Another ask!

@Soda, just checking, are you aware of the clickstream dumps that the toolforge tool is built from? You could try ingesting it differently and building a different tool that is more efficient?

Hm, @AKhatun_WMF I think there may be some differences in feature key names between rendering_feature_counts_change schema and article_feature_counts lambda output. I haven't totally compared, but e.g. message_box vs messagebox, note vs notes.

Nice! Good idea.

Wow, sure why not! So deployment would look like helmfile apply -e eqiad --selector release=mylambda ?

Thank you! Didn't review in depth but only 1 comment:

From a slack thread

If the client can fire off a request and disconnect while leaving hoarde (and the underlying lambdas) to carry out a disproportionate amount of work, what's to stop them from effectively DoS'ing us?

Since there aren't any EventGate owners to consult, we should probably notify some of the key EventGate users of this change.

Beautiful thank you!

if "must be able to carry existing Hive-style tables as-is" is a hard requirement

I hope it is not!

Thanks. As discussed, let's set an expiration date on weekly sqooping of change_tag and change_tag_def.

so let's plug in the real values (even if we need to SWAG them)

How about just WAG? :)

Oh, not sure why gerritbot didn't link my EventBus PoC, but here is it: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/EventBus/+/1345212

Here are the consumer usages of page_change_kind as of 2026-09-28 from reading codesearch results. I did not include any results from tests or fixtures; only actual consumer pipeline usages.

Responded at T433607#12370541.

Additionally, if it is unreliable, then surely so are stream-processing produced artifacts (meaning: what is the alternative?).

In T436709#12370166 @Eevans wrote:

Correct. Before deciding this though, I'd like to do a quick review of uses of WMF uses of page_change_kind. It would be nice if we had a schedule for actually removing it from the event to avoid future confusion, but I'm not sure.

No, 'null edits' are 'reconciliation requests' and do not create new revisions.

FYI T428176: mediawiki.page_change.v1 events - add wikidata id for pages is deloyed.

Actually, more generally, what we are really hoping for is a transactional (persisted) stable ID for DomainEvents. Logging table is just an implementation detail. So maybe

Right, and some metrics are just so standard across many services, especially http stuff: request latency, status codes, etc. It would be nice if all http services used standardized metrics and labels.

Or, please just wait for T434143: Capture relevant mediawiki tags changes in event stream(s) asked in T437961#12329516.

Changing the semantics of page_change_kind would be backwards incompatible. We'd have to carefully migrate all consumers of mediawiki.page_change.v1 (and any derived streams).

We haven't received any alerts or complaints. Unless we do I'd assume that all is fine.