It turns out that in CI (for extensions) we need to add the CI server to the spec in order to get the chai-openapi-response-validator to recognize and validate the spec.

The patch above delivers information for pages with:

Update: We're removing the latency piece of the per-module monitoring since it has a much too high cardinality and is unusable. The latency can be examined as a general property of the ActionAPI through older/other metrics, and the new unified metrics for the Action API will only measure hits (with success and failure modes).

The fix patch was merged: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1215203

I've submitted a fix for the above problems: https://gerrit.wikimedia.org/r/1215203 Please help review; I hope this catches the things that were missed in the previous review cycles. Thanks for the help on this!

Yes, you're right that the labels should be set -- this was an oversight!
I have this ticket now to fix it more systemically T411793: Fortify new API metrics method

This is probably related to my patches https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1197699 and https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1197700 since it calls recordUnifiedMetrics which I moved in that patch.

Thank you!!

In T405373#11212543, @BPirkle wrote:

Per discussion in standup meeting, here are my chief complaints with Handler as it currently exists:

First, spec generation and endpoint execution seem like related but separate concerns. Right now, all that is crammed into Handler. That makes it difficult to know which code will be invoked in which context, complicating maintenance. It also makes it difficult for developers creating their own endpoints to know which functions to override to achieve their goals, because everything is mashed together.

In T402074#11148458, @daniel wrote:

In T402074#11148420, @Mooeypoo wrote:

the ActionAPI seems to have more obscure names, but those aren't yet coming from the application layer

The action API does define these metrics in the application layer, e.g. api_executeTiming_seconds comes from https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/+/master/includes/api/ApiMain.php#963 and api_query_executeTiming_seconds from https://gerrit.wikimedia.org/g/mediawiki/core/+/e099aa914cce56fd863cee40a230a3edbaee0619/includes/api/ApiQuery.php

In T402074#11119179, @Ottomata wrote:

It creates two new metrics:

• api_modules_hits_total (counter)
• api_modules_latency_seconds (observable)

IIUC, these are metrics specifically about MediaWiki HTTP APIs, yes? If so, suggestion:

Consider prefixing these with something like mediawiki_http_, so as not to be confused with other APIs when discovering existent metrics.

Notes after some chats (thanks @Catrope for the insights!):

We just opened #api-alerts on slack, so we can use that channel.

In T401730#11088178, @andrea.denisse wrote:

@Mooeypoo , do you know if these alerts are already present in Prometheus?

After some research and a chat with the Observability team, there are some answers, insights and suggestions for the future.

This ticket is resolved as part of the yearly work. We now have a notification system in MediaWiki core that can send notifications through a unified pathway for notifications that use Echo or Watchlist, and both of those are set as Handlers within this pathway.

This ticket is resolved as part of the yearly work. We now have both a notifications system inside core and a Domain Event asynchronous pattern. The Domain Events replace (and/or add) to some hooks in the system, and can be used to trigger notifications directly.

This ticket is resolved as part of the yearly work. We now have a notification system in MediaWiki core that can send the notification information as part of the notify(...) call (including the RecipientSet) in a much more consolidated manner.

This ticket is resolved as part of the yearly work, and the details were worked on within implementation tickets around the Handlers (alternative word to "Provider") and the migration of Enotif to the new system. We now have a system in core that allows for a sinlge pathway through the Middleware to different Handlers, where Echo is a "catchall" handler; we also made sure Echo can handle an unknown notification.

This ticket is resolved as part of the yearly work. We now have a notification system in MediaWiki core that can send the notification information as part of the notify(...) call (including the RecipientSet) in a much more consolidated manner.

This ticket is resolved as part of the closing of the yearly work. We've ended up with a new Notifications system in MediaWiki core that makes things a lot simpler for development, future enhancements, and maintenance.

This is done. It was created as a tracker for the plans we wanted to explore, and was deprioritized when we discussed scope and benefits. The registration is a bit easier now with the changes we've made, but we deprioritized going farther.

In T396690#10920159, @Catrope wrote:

I think it's important to control this at the invocation level, rather than at the chart description page, because chart descriptions are intended to be usable in multiple different languages (which is why they live on Commons and have localizable labels). In theory I guess we could have a directionality-by-language map in the chart definition like "direction": { "rtl": "rtl", "he": "ltr" }, maybe that would be helpful, or maybe that would be more complexity than we need.

Hi! jumping in here after chatting about this with some folks to say that this is unfortunately a much more elaborate question than people think. There is no clear consistency when it comes to graphs in many of the RTL languages.

In T387996#10641444, @pmiazga wrote:

@Catrope that's exactly the reason why Middleware works on batch/list -> not a single Notification - even at this moment this list usually has just one notitication. The idea behind this is to:

• allow easy Notification removal - we just "empty" the batch and then pass the empty array around - this saves us from null checks in code
• allow easy replacement/injection - in case Extensions want to replace Notification, remove it and/or inject their own -> they can do it with a list easily. When MiddleWare worked on the
• allow future deduplication - as an idea - in the future we can trigger NotificationService::process() only once on POST_SEND and it will go through all Notifications that were triggered within the request and remove duplicates, do batching, or, as an example - guarantee rule of one notification per one title - just an IDEA./

Sounds good to me too, this hasn't been used for a long time either, and given what it does, it should not be used anymore in today's technologies anyways.

In T385810#10556830, @pmiazga wrote:

Thank you @Mooeypoo for the update.

I'm wondering - did you consider an option where we can let Echo Handler fallback to MediaWiki - instead of implementing fallback handling inside MediaWiki? What I mean, for simplicity - if we want to make Echo handle everything anyway, plus the MediaWiki Legacy Handler will most likely be a service anyway. Then we would have only one "active" handler (in this case Echo), but then code in Echo Handler would have hardcoded fallback logic like if I don't know this Event, let me call the Legacy Handler.

Yes, notifications will not work on a clean installation of core without a provider -- either Echo, or another provider. This is current behavior, and is acceptable. Echo is bundled with MediaWiki, so 3rd parties will retain behavior, and anyone that installs "core" alone will need to add Echo for functionality, just like they need to add a skin (and just like current behavior).

We've had a conversation about this in the tech meeting, and here's our thoughts and summary:

I want to outline my concerns around providing (or not providing) a mechanism for "fallback" to use MW Core's straightforward 'email only' provider even if Echo is the available provider. I think that having a Provider take over the entire behavior is a valid course of action, and works with our intended architectural concerns, but there is one aspect that is user-facing that we should be aware of, and make a decision on.

In T385810#10532604, @matmarex wrote:

There are pros and cons for either of those options, and conversations so far have leaned heavily towards #2 (no Fallback)

I got the opposite impression, and I implemented option #1 in https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1108511. I'd be okay with doing #2 too, but for me that increases the scope of the project, since now we have to make sure that every provider/handler supports every kind of notification, or the whole system will drop some notifications on the floor.

(I don't care if it's called "provider" or "handler", they are both meaningless words, so I chose the shorter one.)

In T385810#10529266, @pmiazga wrote:

There is also nit-pick when it comes to naming. A class that extension provides to handle/deliver notifications here is called "Provider". We had similar conversation in the gerrit PR: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1108511/comment/99b9bcb6_b81574d3/

@matmarex implemented Handlers as Echo would provide a mechanism to handle notification. Where Providers sounds more like a extension that would provide notifications, but it may not handle it.