Page MenuHomePhabricator

EventBus extension must not send batches that are too large
Open, Needs TriagePublic

Description

EventBus is capable of sending events in batches. Currently there's no limit in the size of an outgoing batch, which could be problematic, for example if a MassMessage job with a large message is submitted to a large number of users, the overall EventBus extension -> eventgate request size could exceed the limits set up in eventgate (or exceed any common sense size of a POST body).

We need to introduce a config variable with the max size of a batch (probably in bytes, not messages) and attempt to chop up large batches into smaller ones. Ideally the configuration value should be synchronized with eventgate settings.

Event Timeline

Pchelolo created this task.Sep 9 2019, 9:01 PM
Restricted Application added a project: Analytics. · View Herald TranscriptSep 9 2019, 9:01 PM
Krinkle moved this task from Untriaged to EventBus infra on the WMF-JobQueue board.Sep 9 2019, 9:02 PM
Johan added a subscriber: Johan.Tue, Sep 24, 1:30 PM

I suspect this is the reason why MassMessage isn't delivering Tech News to most subscribers. We're getting a good number of complaints, as the editors don't know what's happening – we can't warn them of changes that will happen, they're missing calls for action to tell us about problems some changes will cause, they're unaware of new tools and so on.

Another week when Tech News wouldn't deliver if the MassMessage target list was too big.