Migrated from: https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/623
Right now thumbnail requests are bucketed by original image name by PoolCounter, as a stampede protection. To prevent GWToolset-style outages, we should define an expensive image group, and use a wider bucket for these, so that there cannot be more than X expensive images thumbnailed at the same time. This is a poor solution (would create more waiting jobs and thus open connections than really necessary, due to hash collisions) but would probably still suck way less than the current state.
Acceptance criteria
- have the media handler decide whether scaling the file is expensive, and pass that flag on
- for expensive files, the pool counter should bucket by some hash of the file name instead of the file name itself, so that there are only <range of hash function> slots available for expensive thumbnailing operations
- the hash has to be deterministic to avoid stampeding, and the range has to be configurable