Page MenuHomePhabricator

Thumbnail cache should be automatically discarded after 6 months
Closed, DeclinedPublic

Description

Thumbnails which are older than 6 months should automatically be deleted to force creation of new ones. Reasons:

  • Updated image scalers: Many thumbnails were created with image scalers which are deprecated by now. There are plenty of examples where thumbnails older than 6 years are not sharpened (jpeg sharpening was introduced later, I think) or broken (Quite a few svg rendering issues were fixed over the years)
  • Many bug reports about cache issues: There are many(!) bug reports open right now complaining about caching issues with thumnails.
  • Manual purge does not scale.
  • Clean up of unused data: I am not sure about that but possibly the deletion of old thumbnails makes free space available again on the hard drives.

See also: T19577: Thumbnail urls should be versioned and sent with Expires headers

Details

Reference
bz44310

Event Timeline

bzimport raised the priority of this task from to Low.Nov 22 2014, 1:37 AM
bzimport set Reference to bz44310.
bzimport added a subscriber: Unknown Object (MLST).

Also see http://lists.wikimedia.org/pipermail/wikitech-l/2012-October/064011.html

(In reply to comment #0)

  • Many bug reports about cache issues: There are many(!) bug reports open

right now complaining about caching issues with thumnails.

This proposal will not fix that problem.

Is this bug a matter of setting [[mw:Manual:$wgThumbnailEpoch]] appropriately?

(In reply to comment #3)

Assigning to Reedy.

If you try to assign a bug without leaving a comment, Bugzilla throws an error and asks you to go back in your Web browser. However, when you go back in your Web browser, the assigned to field is not preserved. Grr.

(In reply to comment #0)

Thumbnails which are older than 6 months should automatically be deleted to
force creation of new ones.

Why six months? I think thirty days would be more likely to be expected, given other cache expiries.

Image Scalers do not get updated that much. 30 days would be too much of an effort for the servers. (Keep in mind the servers have to go through ALL files in 30 days to keep up...)

(In reply to comment #5)

(In reply to comment #0)

Thumbnails which are older than 6 months should automatically be deleted to
force creation of new ones.

Why six months? I think thirty days would be more likely to be expected,
given
other cache expiries.

If we did that, we probably might as well just get rid of the thumbnail cache and rely on squid/varnish only. [Saying that without looking at the varnish cache hit ratio or average time things get stuck in varnish cache, etc take with salt]

(In reply to comment #2)

Is this bug a matter of setting [[mw:Manual:$wgThumbnailEpoch]]
appropriately?

No, $wgThumbnailEpoch has no effect when the 404 handler is used. We could delete files from the backend based on modification time, and prevent long-term caching in varnish with a Cache-Control header if that's not done already.

(In reply to comment #7)

If we did that, we probably might as well just get rid of the thumbnail cache
and rely on squid/varnish only. [Saying that without looking at the varnish
cache hit ratio or average time things get stuck in varnish cache, etc take
with salt]

We would like to get rid of the thumbnail backend storage and use varnish only, for various reasons. But we wouldn't try to do it with a 30 day expiry time. 6 months is probably reasonable.

This won't solve the caching issues or significantly reduce the need for manual purging, so those bugs should be addressed separately.

Krinkle updated the task description. (Show Details)
Krinkle removed a subscriber: wikibugs-l-list.
Krinkle subscribed.

Per Ops feedback on T356, it seems this is no longer needed and/or being handled routinely by other means now.