Page MenuHomePhabricator

Normalize thumbnail request URLs in Varnish to avoid cachebusting
Closed, ResolvedPublic

Description

Currently Varnish attempts to prevent cachebusting on upload.* URLs by stripping query parameters and double slashes. However, that's not enough for thumbnails, where multiple URLs can point to the same thumbnail, by virtue of having a no-op part of the thumbnail URI (the repeated filename).

This could be easily prevented by normalizing thumbnail request URIs further in Varnish.

Event Timeline

Gilles triaged this task as Normal priority.Feb 16 2019, 11:38 PM
Gilles created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 16 2019, 11:38 PM

Change 491215 had a related patch set uploaded (by Gilles; owner: Gilles):
[mediawiki/vagrant@master] Normalize thumbnail request URLs in Varnish

https://gerrit.wikimedia.org/r/491215

kchapman moved this task from Inbox to Doing on the Performance-Team board.Feb 19 2019, 9:09 PM
ema moved this task from Triage to Caching on the Traffic board.Mar 6 2019, 9:55 AM

Change 491215 merged by jenkins-bot:
[mediawiki/vagrant@master] Normalize thumbnail request URLs in Varnish

https://gerrit.wikimedia.org/r/491215

Change 495243 had a related patch set uploaded (by Gilles; owner: Gilles):
[mediawiki/vagrant@master] Normalize abbreviated case + remove redundant ext

https://gerrit.wikimedia.org/r/495243

Change 495643 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/puppet@production] Normalize thumbnail URLs to avoid cachebusting

https://gerrit.wikimedia.org/r/495643

Change 495243 merged by jenkins-bot:
[mediawiki/vagrant@master] Normalize abbreviated case + remove redundant ext

https://gerrit.wikimedia.org/r/495243

Gilles changed the task status from Open to Stalled.Mar 21 2019, 9:50 AM
jijiki added a subscriber: jijiki.Apr 27 2019, 8:25 AM

Change 495643 had a related patch set uploaded (by Ema; owner: Gilles):
[operations/puppet@production] Normalize thumbnail URLs to avoid cachebusting

https://gerrit.wikimedia.org/r/495643

Gilles changed the task status from Stalled to Open.Jun 19 2019, 2:10 PM

Change 495643 merged by Ema:
[operations/puppet@production] Normalize thumbnail URLs to avoid cachebusting

https://gerrit.wikimedia.org/r/495643

Gilles closed this task as Resolved.Jun 21 2019, 8:22 AM
Gilles reopened this task as Open.Jun 21 2019, 9:45 AM
Gilles closed this task as Resolved.Jun 21 2019, 1:58 PM

This has caused a spike of thumbor thumbnailing requests, by virtue of making some objects hotter through deduplication, making them pass the webp hotness threshold.

Things seem to have stabilised for now, but the level of thumbnails generated is still higher than usual and will remain at a higher level for some time. Maybe even permanently, as a result of the hotness criteria being met more often.