Page MenuHomePhabricator

EventBus extension must not send batches that are too large
Open, HighPublic


EventBus is capable of sending events in batches. Currently there's no limit in the size of an outgoing batch, which could be problematic, for example if a MassMessage job with a large message is submitted to a large number of users, the overall EventBus extension -> eventgate request size could exceed the limits set up in eventgate (or exceed any common sense size of a POST body).

We need to introduce a config variable with the max size of a batch (probably in bytes, not messages) and attempt to chop up large batches into smaller ones. Ideally the configuration value should be synchronized with eventgate settings.

Event Timeline

Pchelolo created this task.Sep 9 2019, 9:01 PM
Restricted Application added a project: Analytics. · View Herald TranscriptSep 9 2019, 9:01 PM
Krinkle moved this task from Untriaged to EventBus infra on the WMF-JobQueue board.Sep 9 2019, 9:02 PM
Johan added a subscriber: Johan.Sep 24 2019, 1:30 PM

I suspect this is the reason why MassMessage isn't delivering Tech News to most subscribers. We're getting a good number of complaints, as the editors don't know what's happening – we can't warn them of changes that will happen, they're missing calls for action to tell us about problems some changes will cause, they're unaware of new tools and so on.

Another week when Tech News wouldn't deliver if the MassMessage target list was too big.

daniel triaged this task as Medium priority.Nov 11 2019, 8:07 PM
Ottomata moved this task from Backlog to Radar on the Event-Platform board.Apr 14 2020, 1:23 PM
Pchelolo raised the priority of this task from Medium to High.Apr 29 2020, 8:45 PM
Pchelolo added subscribers: Krinkle, Joe, Mholloway.

Bumping to high priority since this is happening in production and it's not a theoretical issue.

Base added a subscriber: Base.Thu, Jun 11, 6:15 PM