Page MenuHomePhabricator

WDQS - Throttle updates to WDQS
Closed, DeclinedPublic

Description

On July 6 around 7 UTC a user started to mass edit Wikidata, generating a flood of updates to WDQS, geenrating a slowdown of WDQS. Leave asside that this is bad etiquette. We should ensure that WDQS is robust in front of something like this happening again.

My guess is that it should not be too hard to add throttling into wdqs-updater. Graphs show that writes peaked around 50 updates / second (which seems low to me). So the exact throttling will need to be tuned based on experience. Ideally, being able to hot modify this throttling value would be great (JMX, hot reload of config file, whatever), but not really necessary (restart of wdqs-updater does not cost much).

Event Timeline

Restricted Application added subscribers: Zppix, Aklapper. · View Herald Transcript

I think we don't need throttling for now, WDQS should recover by itself even if backlog is backed up due to a spike.