I would lean towards the service polling alertmanager, to avoid having to "teach" alermanager that there's a toolforge service using it, and keep it separated (alertmanager does not need the toolforge service, but the toolforge service needs alertmanager)

In T432865#12230378, @dcaro wrote:

I would lean towards the service polling alertmanager, to avoid having to "teach" alermanager that there's a toolforge service using it, and keep it separated (alertmanager does not need the toolforge service, but the toolforge service needs alertmanager)

or just not having the toolforge service side for now xd

I'm honestly still undecided between the 3 possible implementations (webhook, polling, no service). I tried summarizing the pros/cons below, @dcaro @taavi @fgiunchedi please add your comments/concerns.

Option A: webhook, Alertmanager pushes to a new service running in Toolforge

Pros:

• Alertmanager already handles repeat interval, resolved notifications
• The service is quite simple, it only has to render the email and send it whenever it receives a webhook

Cons:

• alertmanager config must authenticate to the service (auth header?)
• the service URL has to be stored in alertmanager

Option B: polling, a new service in Toolforge reads the Alertmanager API

Pros:

• cleaner dependency direction (alertmanager doesn't know that the service exists)
• zero alertmanager-side changes

Cons:

• the service must reimplement some alertmanager features (repeat interval, resolved notifications)
• we need to store a persistent state in the service from day 1 (in the future we will need persistent state anyway for the alert-history feature discussed in T432860#12217066)

Option C: no service, Alertmanager sends the emails itself

Pros:

• email_configs.to is a template string, so we could keep a single receiver and use something like to: '{{ .GroupLabels.tool | reReplaceAll "^tool-" "tools." }}@toolforge.org'
• group_by and repeat_interval can be used to provide grouping (one email per tool if there are multiple alerts) and re-notification (e.g. one email every 24h while the alert stays firing)

Cons:

• storing a template in the EMAIL field of the Metricsinfra db is a bit of a hack
• I think we need an additional group_by: in the custom alert_routes, otherwise alertmanager generates notifications spanning multiple tools, and cannot render the email address using .GroupLabels.tool
• individual opt-out (per-maintainer instead of per-tool) is impossible, if we wanted to add it in the future

My vote goes to option A. Option B to me seems to re-implement features that Alertmanager already does well with little benefits, and option C gives you the least amount of controls so you'd be stuck with the same email templates as everyone else, limitations like what you mention with opt-outs, etc.

Another advantage of option B:

• Easy decoupling from alertmanager, allowing us to easily use other systems instead or besides alertmanager in the future.

Option A seems best to me: very little implementation time/effort compared to the alternatives and the external "surface" for alertmanager/toolforge integration is a single POST with json payload. In other words mocking alertmanager in this case is quite easy (e.g. for end-to-end testing in lima-kilo)

Thanks for the comments. I agree that A looks like a good option and is easier to implement than option B. @dcaro regarding decoupling, I think the interface is similar, only the direction changes: the new service will receive a bunch of alerts as the input, and send emails to maintainers as the output. In both directions it's quite easy to mock with a JSON payload as @fgiunchedi suggested.

For this task, using the webhook is significantly easier because we can reuse some Alertmanager features, and we don't have to implement persistent storage right now. Nothing stops us from switching to polling in the future if we see compelling reasons: I imagine for example that polling the Alertmanager API might be useful to implement the future "alert-listing" feature.

Option C would require the smallest amount of work, but I agree it feels like the one with more limitations in the long run.

Unless there are objections, I will start implementing option A. @taavi do you agree with using an auth header for authenticating the webhook? I can send a patch to metricsinfra to add it.

Thanks for the comments. I agree that A looks like a good option and is easier to implement than option B. @dcaro regarding decoupling, I think the interface is similar, only the direction changes: the new service will receive a bunch of alerts as the input, and send emails to maintainers as the output. In both directions it's quite easy to mock with a JSON payload as @fgiunchedi suggested.

Note that this also means that we can't easily deploy different things in toolsbeta and tools, as alertmanager will have to know about the tools deployment (unless we deploy a metricsinfra-toolsbeta sort of thing).

In summary, if we are doing it to save time right now by making sure that we will spend the time to move out of it as soon as possible, that's ok. But I would like to have a really really clear idea that there's a need to get it right now, instead of 1 moth later.

But if we are donig it with the hopes that we will not have to do it in the future I strongly believe that we will be just pushing a ball rolling down a snow hill, and will have to spend way more time later to decouple with the added pressure.

@dcaro I can see two separate concerns in your comment, one is storing configuration in the Metricsinfra db, the other is discovering later that using webhooks has some limitation that requires us to switch to polling. My proposal is to address those concerns in 3 steps:

1. Implement the new service using webhooks, configured in Metricsinfra. The service can be developed and tested using mocked webhook payloads (as @fgiunchedi proposed). On the Metricsinfra side, we need one small patch (to support a webhook with auth header) and adding two webhook URLs in the metricsinfra db (one for tools, one for toolsbeta), so that we can deploy a test version of the service in toolsbeta.
2. Add a dedicated instance of Alertmanager, deployed in Toolforge k8s (new task T435636). This instance would take over the alerts for tool-maintainers, while leaving the admin alerts flowing to Metricsinfra. This means the webhook config would move into toolforge-deploy config values. I filed this as a sub-task of the ST5.4.2 hypothesis, but I would do this only after completing step 1; if needed it can be pushed to the next quarter (Q2), because we can still deliver the hypothesis without it, but I would not postpone it forever. Note that there is not much to "undo" in this step, we simply have to remove the two webhooks from the Metricsinfra db.
3. Later, consider switching to polling if and only if the webhook mechanism proves insufficient. In my view, using webhooks is not technical debt, but a deliberate design decision, consistent with Alertmanager's own design (alerts are pushed from Prometheus into Alertmanager, not polled). My main concern with polling is that it requires a persistent state machine inside the Toolforge Alerting Service, which I think should only be added with a strong justification. The features I can think of don't require it: listing alerts (that I mentioned in my previous comment) could be implemented in other ways (e.g. querying the Prometheus alerts API); per-user opt-in (instead of per-tool) can also be achieved with the same webhook, doing user lookups inside the service.

one is storing configuration in the Metricsinfra db, the other is discovering later that using webhooks has some limitation that requires us to switch to polling

Not really no, my concern is related to coupling our design to other systems, those are just ways to couple them, essentially:

• Coupling to the metricsinfra service
• Coupling a user-focused service to infrastructure-focused service
• Coupling to prometheus/alertmanager

Each of those couplings forces different restrictions in the short, mid and long term, and has different benefits also.

My point is, that given the resources we have, the time restrictions that we have, and the ones we foresight we'll have, we should try to decouple as much as we can from the start, creating clear boundaries that are as thin as possible between them, so we can evolve our system separately from any others without having to spend too much time keeping up/waiting for the external systems to evolve.

For example:

• Those external systems should know as little as possible about our system, with the best being not knowing anything at all (so we don't need to change them when changing our code).
• Those external systems should be deployed independently from ours, so we don't need to coordinate upgrades, test upgrade paths, and we can deploy different versions in different environments, etc.
• Our system should depend as little as possible on the ways those systems work, for example, instead of depending on their configuration/setup, or inner workings (like webhook payloads), we control how to consume the data they provide (so we don't have to modify how alertmanager sends the webhook data, instead we change the way we consume it from it's api, allowing us to switch between alertmanager instances/versions with a config value). This is the "other direction" view of the first bullet.
• Our system should avoid depending on us (or others) having to prioritize a third system (ex. depending on alertmanager development, or metricsinfra being an okr for the infra team). Note that this does not mean that we don't help upstream, just that we avoid creating systems where we are forced to get blocked by it, it applies also to coupling our own systems between themselves.

2. Add a dedicated instance of Alertmanager, deployed in Toolforge k8s

+1, this takes care of the first two couplings, that is already a huge improvement.

consistent with Alertmanager's own design (alerts are pushed from Prometheus into Alertmanager, not polled)

That phrase is the key point there, this means that from there on you will have to follow alertmanager own design, instead of following your own (or until you change the paradigm, that will take extra effort to achieve).

This is similar to using k8s controllers as part of the apis, it works, yes, it's k8s own design, yes, but you get bound to it and now have to follow it everywhere, becomeing quite hard later to decouple from it later, so it's a long-term decision.

Note that using polling or webhooks by itself is not the issue, the issue is having to depend on alertmanager configuration and version (in case it changes the way it sends payloads, how the debounce is configured, it being able to discern users from tools, etc.).

Btw. karma (the UI we use) itself does not use webhooks, but uses polling, it also does deduplication/aggregation itself (on top of that polling), and manages silences (through the alertmanager API). So maybe we should follow that design instead of alertmanager itself xd

My main concern with polling is that it requires a persistent state machine inside the Toolforge Alerting Service, which I think should only be added with a strong justification. The features I can think of don't require it: listing alerts (that I mentioned in my previous comment) could be implemented in other ways (e.g. querying the Prometheus alerts API); per-user opt-in (instead of per-tool) can also be achieved with the same webhook, doing user lookups inside the service.

It already requires the state machine, but instead of "owning" it the service uses alertmanager as the state machine and implementations around it. You are not making the system less complicated, you are pushing the complexity to a third party that you'll have to maintain and own too (that is also ok, as long as it's understood that that's what's going on).

All those features don't need any state besides service configuration and the config endpoint T433690: [components-api] Create endpoint to get tools based on a set config for user/tool specific settings. I think that what will need some state is the debouncing of alerts (bundling them in one single email, and/or avoid sending too many emails at once), probably a shared cache (redis/valkey/memcache...) would be enough for that too, that we can use in-memory for starters until we need to scale up (alertmanager also needs something like that to scale up, currently we only have one instance alerting at a time, see https://oneuptime.com/blog/post/2026-01-28-alertmanager-clustering for a clustered setup).

we can use in-memory for starters until we need to scale up

@dcaro your proposal of using polling without a persistent storage is interesting, as it removes my main argument against polling (that it would require a stateful service). I created a naive PoC using Claude (P96252) that covers the features needed by our use case, so my proposal is to flip my previous reasoning: instead of starting from the webhook, and only migrating to polling with a strong justification, we can start from an in-memory polling implementation (akin to the PoC above), but retain the possibility of migrating to webhooks in the future.

Please note the PoC is polling the Prometheus API directly instead of the Alertmanager API: the information we need is available in both, both are easy to test with JSON fixtures, but calling Prometheus removes any coupling between the alerting feature and Metricsinfra, without the need of a dedicated Toolforge Alertmanager instance.

I think this gives us the cheapest and quickest way to deploy the Toolforge Alerting Service, and start testing our hypothesis with real users.

The parts of Alertmanager we are reimplementing in this scenario are:

• merging alerts from multiple prometheus instances
• repeat interval
• resolve notifications

The parts of the PoC above that we would still need even if we used webhooks:

• keep track of emails sent with an in-memory structure, because if alertmanager is restarted, it would send a bunch of duplicate webhooks

The PoC does not need any persistent state because if the pod is restarted, it checks the activeAt values returned from the Prometheus API to decide which alerts were already firing and skips sending emails for those. Note that this assumes the pod was only briefly down during the restart.

Important caveat: this implementation fits in about 100 lines of Python, but I would be very careful to avoid the risk of reimplementing Alertmanager one piece at a time. The moment we need any more complex feature, that we cannot easily reconstruct with a call to the Prometheus API, I would abandon our implementation and switch to using Alertmanager instead. I would also abandon this implementation if during live testing we find other issues (e.g. we're sending duplicate emails where we shouldn't).

Let me know if there are objections to this plan. In the meantime, I will start creating the scaffolding of the Toolforge Alerting Service in sub-task T435971: Add scaffolding for the Toolforge Alerting Service.

Let me know if there are objections to this plan. In the meantime, I will start creating the scaffolding of the Toolforge Alerting Service in sub-task T435971: Add scaffolding for the Toolforge Alerting Service.

Sounds good to me, let's give it a try

In T432865#12252684, @fnegri wrote:

Please note the PoC is polling the Prometheus API directly instead of the Alertmanager API: the information we need is available in both, both are easy to test with JSON fixtures, but calling Prometheus removes any coupling between the alerting feature and Metricsinfra, without the need of a dedicated Toolforge Alertmanager instance.

In Toolforge, the Prometheus API is not highly available nor clustered, unlike the metricsinfra Alertmanager service.

The parts of the PoC above that we would still need even if we used webhooks:

• keep track of emails sent with an in-memory structure, because if alertmanager is restarted, it would send a bunch of duplicate webhooks

The metricsinfra Alertmanager instance is a stateful HA pair which means alert state is not lost when the service restarts.

In Toolforge, the Prometheus API is not highly available nor clustered, unlike the metricsinfra Alertmanager service.

@taavi fair point, I didn't remember that prometheus.svc.toolforge.org is just a DNS pointer to one of the two instances. This is handled in the PoC by polling two instances and reconciling them (see the fetch_firing() method). In case the hostname changes (tools-prometheus-9 becomes tools-prometheus-10), we would need to update the service config, or we could use two DNS pointers instead of one, and update those (tools-prometheus-a and tools-prometheus-b).

What we gain if we poll from the tools prometheus instances is that we don't have any kind of coupling between the maintainer-alerts feature and metricsinfra (one of the concerns raised by @dcaro). If reconciling the two prometheus instances in the service proves to be fragile, we can fall back to using metricsinfra's alertmanager API ("exit hatch 1" below).

The metricsinfra Alertmanager instance is a stateful HA pair which means alert state is not lost when the service restarts.

Right, I was thinking of the potential scenario after T435636: Add a dedicated Toolforge alertmanager instance, where we would have a single alertmanager in k8s. We should keep it in mind if we decide to switch from polling to webhooks ("exit hatch 2" below).

Updated plan:

• Write a first iteration of the service polling both prometheus instances, based on the PoC above, so that we test this design in production.
• Exit hatch 1: if we see evidence this is fragile in production, we move to polling metricsinfra's alertmanager instead. No changes are needed to metricsinfra, and we can remove the two-instance-reconciliation code from the service. We do add a coupling but it's a weak coupling because the maintainer alerts would not rely on any custom config stored in metricsinfra.
• Exit hatch 2: if we see evidence that polling causes double-emails or other edge conditions, we switch to webhooks, preferably with a dedicated alertmanager in k8s (T435636: Add a dedicated Toolforge alertmanager instance) to avoid strong coupling with metricsinfra's alertmanager.

fnegri opened https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/3

Fetch alerts from Prometheus

fnegri opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1419

alerts-api: configure prometheus hostnames

fnegri opened https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/4

Keep polling when a prometheus host is unreachable

fnegri merged https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/3

Fetch alerts from Prometheus

group_203_bot_3c0afd0d9fd9529f3b7bc7e69a4a3bce updated https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1422

alerts-api: bump to 0.0.5-20260918130948-21768f68

fnegri merged https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/4

Keep polling when a prometheus host is unreachable

Mentioned in SAL (#wikimedia-cloud-feed) [2026-09-18T13:18:45Z] <fnegri@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component alerts-api (T432865)

Mentioned in SAL (#wikimedia-cloud-feed) [2026-09-18T13:37:26Z] <fnegri@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component alerts-api (T432865)

Mentioned in SAL (#wikimedia-cloud-feed) [2026-09-18T13:39:10Z] <fnegri@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component alerts-api (T432865)

Mentioned in SAL (#wikimedia-cloud-feed) [2026-09-18T13:59:00Z] <fnegri@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component alerts-api (T432865)

fnegri merged https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1422

alerts-api: bump to 0.0.6-20260918131347-8faad192

Mentioned in SAL (#wikimedia-cloud-feed) [2026-09-18T14:19:19Z] <fnegri@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component alerts-api (T432865)

Mentioned in SAL (#wikimedia-cloud-feed) [2026-09-18T14:19:53Z] <fnegri@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component alerts-api (T432865)

Mentioned in SAL (#wikimedia-cloud-feed) [2026-09-18T14:20:38Z] <fnegri@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component alerts-api (T432865)

Mentioned in SAL (#wikimedia-cloud-feed) [2026-09-18T14:21:07Z] <fnegri@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component alerts-api (T432865)

Mentioned in SAL (#wikimedia-cloud-feed) [2026-09-18T14:31:05Z] <fnegri@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component alerts-api (T432865)

Mentioned in SAL (#wikimedia-cloud-feed) [2026-09-18T14:31:46Z] <fnegri@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component alerts-api (T432865)

fnegri merged https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1419

alerts-api: configure prometheus hostnames

The MRs above are now merged and deployed to both toolsbeta and tools. They are working as expected, but no email is being sent out yet, that will need a few more patches that I'm still refining are not ready for review yet.

fnegri@tools-k8s-control-7:~$ sudo kubectl logs -n alerts-api --timestamps alerts-api-poller-c7856d66f-rfbvf |head -n 10
2026-09-18T14:31:33.676377309Z /alerts/alerts/poller.py:29:main INFO - Starting alerts-api poller with config: Config(debug=False, loop_interval_seconds=60, prometheus_urls=['http://tools-prometheus-8.tools.eqiad1.wikimedia.cloud/tools/api/v1/alerts', 'http://tools-prometheus-9.tools.eqiad1.wikimedia.cloud/tools/api/v1/alerts'])
2026-09-18T14:31:33.799592766Z /alerts/alerts/poller.py:22:poll_cycle INFO - firing alerts for tool maintainers: 76
2026-09-18T14:32:33.723489818Z /alerts/alerts/poller.py:22:poll_cycle INFO - firing alerts for tool maintainers: 76
2026-09-18T14:33:33.720087043Z /alerts/alerts/poller.py:22:poll_cycle INFO - firing alerts for tool maintainers: 76
2026-09-18T14:34:33.733782716Z /alerts/alerts/poller.py:22:poll_cycle INFO - firing alerts for tool maintainers: 77
2026-09-18T14:35:33.729145398Z /alerts/alerts/poller.py:22:poll_cycle INFO - firing alerts for tool maintainers: 77
2026-09-18T14:36:33.732953541Z /alerts/alerts/poller.py:22:poll_cycle INFO - firing alerts for tool maintainers: 77
2026-09-18T14:37:33.732788330Z /alerts/alerts/poller.py:22:poll_cycle INFO - firing alerts for tool maintainers: 77
2026-09-18T14:38:33.730167083Z /alerts/alerts/poller.py:22:poll_cycle INFO - firing alerts for tool maintainers: 77
2026-09-18T14:39:33.738274284Z /alerts/alerts/poller.py:22:poll_cycle INFO - firing alerts for tool maintainers: 77

fnegri opened https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/5

Notify when an alert fires or is resolved

fnegri opened https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/6

Do not re-notify old alerts after a restart

All the review comments in https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/5 and https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/6 were addressed. Those two MRs are pending a re-review before they can be deployed.

After that, a few things are still missing to complete this task:

• compose the emails and send them over SMTP
• add the HTML version as defined in T437081: Create template for maintainer alerts emails
• read from components-api which tools have opted in (currently the list is hardcoded) (EDIT: I created its own task for this, T439554: Read the opted-in tools from components-api)

I have draft patches for these on my laptop, I will push those for review next week.

fnegri opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1434

alerts-api: Use the root URL of each prometheus host

fnegri opened https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/8

Send the notification emails

fnegri merged https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/5

Notify when an alert fires or is resolved

fnegri merged https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/6

Do not re-notify old alerts after a restart

group_203_bot_3c0afd0d9fd9529f3b7bc7e69a4a3bce updated https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1436

alerts-api: bump to 0.0.8-20260929162949-8bda6ff7

fnegri opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1438

alerts-api: Use the root URL for prometheus hosts

fnegri closed https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1434

alerts-api: Use the root URL of each prometheus host

Mentioned in SAL (#wikimedia-cloud-feed) [2026-09-30T09:11:00Z] <fnegri@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component alerts-api (T432865)

Mentioned in SAL (#wikimedia-cloud-feed) [2026-09-30T09:12:05Z] <fnegri@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component alerts-api (T432865)

Mentioned in SAL (#wikimedia-cloud-feed) [2026-09-30T09:31:14Z] <fnegri@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component alerts-api (T432865)

Mentioned in SAL (#wikimedia-cloud-feed) [2026-09-30T09:32:12Z] <fnegri@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component alerts-api (T432865)

fnegri merged https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1438

alerts-api: Use the root URL for prometheus hosts

fnegri merged https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1436

alerts-api: bump to 0.0.9-20260929163209-a3bb1242

fnegri merged https://gitlab.wikimedia.org/repos/cloud/toolforge/alerts-api/-/merge_requests/8

Send the notification emails

group_203_bot_3c0afd0d9fd9529f3b7bc7e69a4a3bce updated https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1442

alerts-api: bump to 0.0.11-20261002103146-d9ef1155

fnegri opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1443

alerts-api: Turn on the emails in tools and toolsbeta

Mentioned in SAL (#wikimedia-cloud-feed) [2026-10-02T11:05:59Z] <fnegri@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component alerts-api (T432865)

Mentioned in SAL (#wikimedia-cloud-feed) [2026-10-02T11:07:06Z] <fnegri@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component alerts-api (T432865)

Mentioned in SAL (#wikimedia-cloud-feed) [2026-10-02T13:45:39Z] <fnegri@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component alerts-api (T432865)

Mentioned in SAL (#wikimedia-cloud-feed) [2026-10-02T13:46:47Z] <fnegri@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component alerts-api (T432865)

fnegri merged https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1442

alerts-api: bump to 0.0.11-20261002103146-d9ef1155

fnegri merged https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1443

alerts-api: Turn on the emails in tools and toolsbeta