Page MenuHomePhabricator

Enable Facial and Feature Detectors
Closed, DeclinedPublic

Description

Currently the apps crop images to provide a better reading experience. To do this, they perform face detection to perform a "smarter" crop.

IMG_0484.PNG (2×1 px, 3 MB)

This comes with a few problems:

  1. All detection is done on the device and every device must run this
  2. Devices generally can only do detection of faces, not important features - leading to some bad crops.
  3. Devices that can't do face detection are unable to use this data to show a better crop

It would be much better if this was done on the server and the results were cached so all clients could get this benefit, not just the WMF apps.

Thumbor looks like a great way to do this.

Are there any blockers to enabling this feature? Since this would be pretty useful to Reading clients, we are interested in collaborating to make this happen.

Event Timeline

Fjalapeno renamed this task from Enable Facial an Feature Detectors to Enable Facial and Feature Detectors.Oct 31 2017, 4:35 PM

This kind of processing is better done async because it's quite time-consuming. Ideally you would issue a non-smart thumb on first request and serve the smart-cropped one once it's ready. But this something our thumbnail caching infrastructure doesn't currently support. All our thumbnails are rendered on the fly "as fast as possible". In that limited paradigm, something like smart cropping would slow down initial thumbnail requests considerably, meaning that clients might end up waiting seconds for a smart cropped thumbnail where the non-smart one would have taken milliseconds. I don't think this is a smart tradeoff.

A "quick thumb on first request" followed by a refined thumb is something desirable in general, which would allow us to perform time-consuming agressive visually lossless compression on our most popular thumbnails, for example.

I don't think there's an existing task for the work I'm describing, but it's a significant undertaking involving our entire thumbnail infrastructure from URL scheme, our various layers of caching, generation and storage.

So while this is very simple at the Thumbor level, it's everything else around it that makes it a big undertaking.

LGoto closed this task as Declined.Oct 9 2020, 4:50 PM