Page MenuHomePhabricator

EventBus extension must not send batches that are too large
Closed, ResolvedPublic

Description

EventBus is capable of sending events in batches. Currently there's no limit in the size of an outgoing batch, which could be problematic, for example if a MassMessage job with a large message is submitted to a large number of users, the overall EventBus extension -> eventgate request size could exceed the limits set up in eventgate (or exceed any common sense size of a POST body).

We need to introduce a config variable with the max size of a batch (probably in bytes, not messages) and attempt to chop up large batches into smaller ones. Ideally the configuration value should be synchronized with eventgate settings.

Event Timeline

I suspect this is the reason why MassMessage isn't delivering Tech News to most subscribers. We're getting a good number of complaints, as the editors don't know what's happening – we can't warn them of changes that will happen, they're missing calls for action to tell us about problems some changes will cause, they're unaware of new tools and so on.

Another week when Tech News wouldn't deliver if the MassMessage target list was too big.

daniel triaged this task as Medium priority.Nov 11 2019, 8:07 PM
Pchelolo raised the priority of this task from Medium to High.Apr 29 2020, 8:45 PM
Pchelolo added subscribers: Krinkle, Joe, Mholloway.

Bumping to high priority since this is happening in production and it's not a theoretical issue.

Change 618353 had a related patch set uploaded (by Clarakosi; owner: Clarakosi):
[mediawiki/extensions/EventBus@master] EventBus: Partition batch messages that are too large

https://gerrit.wikimedia.org/r/618353

Change 618353 merged by jenkins-bot:
[mediawiki/extensions/EventBus@master] EventBus: Partition batch messages that are too large

https://gerrit.wikimedia.org/r/618353