Thumbnails more "blurry"
Closed, InvalidPublicBUG REPORT
Actions

Assigned To

None

Authored By

	dom_walden
	Sep 26 2022, 1:59 PM

Description

What is the problem?

For many images, the thumbnail produced by my local thumbor is blurrier when compared to beta.

beta	local

Reproduction

My configuration (thumbor.conf) is: P34931

I generate the thumbnail on my local system by going to the url: http://localhost:8800/thumbor/unsafe/<width>x/<url>.

I have reproduced it with thumbor running in docker on Buster:

$ blubber .pipeline/blubber.yaml dev > Dockerfile
$ make build

I have also reproduced this when I installed locally via python on Bullseye:

python3 -m pip install -r requirements.txt

Images to reproduce problem

Here is just a small sample:

Environment

Operating system: Debian Bullseye and Buster
thumbor-plugins: commit 92eafe2477fdd122244b572e1a0f57652bdbc8f8

Related Objects

Mentioned Here: P37284 thumbor config from beta
T318548: Incorrect orientation of some thumbnails
P34931 Thumbor local configuration
rTHMBREXT92eafe2477fd: Remove division operation hack related to Python2

Event Timeline

dom_walden created this task.Sep 26 2022, 1:59 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 26 2022, 1:59 PM

@dom_walden: Please provide clear steps to reproduce. There's many parameters that could be passed or not when somehow creating a thumbnail. Thanks.

dom_walden updated the task description. (Show Details)Sep 27 2022, 7:20 AM

dom_walden updated the task description. (Show Details)Sep 27 2022, 8:23 AM

Looks like you don't have the conditional sharpening configuration. DEFAULT_FILTERS_JPEG = 'conditional_sharpen(0.0,0.8,1.0,0.0,0.85)' Here's the config I used for test and development (it may need to be updated for the recent work):

import thumbor
QUALITY = 87
RESPECT_ORIENTATION = True
LOADER = 'wikimedia_thumbor.loader.proxy'
#LOADER = 'wikimedia_thumbor.loader.video'
FILE_LOADER_ROOT_PATH = '/srv/thumbor-plugins/tests/integration/originals/'
STORAGE = 'thumbor.storages.no_storage'
ENGINE = 'wikimedia_thumbor.engine.proxy'
HTTP_LOADER_CA_CERTS = '/etc/ssl/certs/ca-certificates.crt'
HTTP_LOADER_REQUEST_TIMEOUT = 300
FFMPEG_PATH = '/usr/bin/ffmpeg'
FILTERS = [
    'wikimedia_thumbor.filter.conditional_sharpen',
    'wikimedia_thumbor.filter.format',
    'wikimedia_thumbor.filter.lang',
    'wikimedia_thumbor.filter.page',
    'wikimedia_thumbor.filter.crop',
    'wikimedia_thumbor.filter.flip',
    'thumbor.filters.quality',
    'thumbor.filters.rotate'
]
EXIF_FIELDS_TO_KEEP = [ 'Artist', 'Copyright', 'ImageDescription' ]
EXIF_TINYRGB_PATH = '/srv/thumbor-plugins/tests/integration/tinyrgb.icc'
EXIF_TINYRGB_ICC_REPLACE = 'sRGB IEC61966-2.1'
PROXY_ENGINE_ENGINES = [
    ('wikimedia_thumbor.engine.xcf', ['xcf']),
    ('wikimedia_thumbor.engine.djvu', ['djvu']),
    ('wikimedia_thumbor.engine.vips', ['tiff', 'png']),
    ('wikimedia_thumbor.engine.tiff', ['tiff']),
    ('wikimedia_thumbor.engine.ghostscript', ['pdf']),
    ('wikimedia_thumbor.engine.gif', ['gif']),
    ('wikimedia_thumbor.engine.stl', ['stl']),
    ('wikimedia_thumbor.engine.svg', ['svg']),
    ('wikimedia_thumbor.engine.imagemagick', ['jpg', 'png', 'webp']),
]
HTTP_LOADER_MAX_BODY_SIZE = 4*1024*1024*1024  # 4GB
PROXY_LOADER_LOADERS = [
    #'wikimedia_thumbor.loader.file',
    'wikimedia_thumbor.loader.https',
]
COMMUNITY_EXTENSIONS = [
    'wikimedia_thumbor.handler.images',
    'wikimedia_thumbor.handler.core',
]
SUBPROCESS_USE_TIMEOUT = True
SUBPROCESS_TIMEOUT = 59
VIPS_ENGINE_MIN_PIXELS = 10000000
CHROMA_SUBSAMPLING = '4:2:0'
QUALITY_LOW = 40
DEFAULT_FILTERS_JPEG = 'conditional_sharpen(0.0,0.8,1.0,0.0,0.85)'
LOADER_EXCERPT_LENGTH = 4096
MANHOLE_DEBUGGING = True
APP_CLASS = 'wikimedia_thumbor.app.App'
HTTP_LOADER_TEMP_FILE_TIMEOUT = 20
MAX_ANIMATED_GIF_AREA = 100000000

In T318552#8315729, @AntiCompositeNumber wrote:

Looks like you don't have the conditional sharpening configuration. DEFAULT_FILTERS_JPEG = 'conditional_sharpen(0.0,0.8,1.0,0.0,0.85)' Here's the config I used for test and development (it may need to be updated for the recent work):

Adding that to my thumbor config did not improve the "blurriness" by any perceptible amount.

I could not get your config working locally.

I changed the FILE_LOADER_ROOT_PATH and EXIF_TINYRGB_PATH but got the error: AttributeError: EXIFTOOL_PATH

So I added EXIFTOOL_PATH = 'exiftool' but then go the error: AttributeError: SUBPROCESS_TIMEOUT_PATH

I commented out SUBPROCESS_USE_TIMEOUT = True but then got the error: AttributeError: CONVERT_PATH

At which point I did not continue.

However, I did get the config that thumbor is using on beta (I am not sure I am allowed to share it publicly though) and ran it locally, but it is still generating "blurrier" thumbnails compared to beta.

Also, thanks, you have pointed to the fix for T318548 (including RESPECT_ORIENTATION = True)!

Looks like there's also nothing in your configuration loading the conditional sharpening filter. I don't know if anything's changed in that plumbing, but I would expect that it might have. The relevant line from my old config is

FILTERS = [
    'wikimedia_thumbor.filter.conditional_sharpen',
    'wikimedia_thumbor.filter.format',
    'wikimedia_thumbor.filter.lang',
    'wikimedia_thumbor.filter.page',
    'wikimedia_thumbor.filter.crop',
    'wikimedia_thumbor.filter.flip',
    'thumbor.filters.quality',
    'thumbor.filters.rotate'
]

You can try adding it and see what happens.

Conditional sharpening is applied to JPEG files, but not to PNG files. Does the level of blurriness change between test and production PNG thumbnails? If they don't look different, it's probably the conditional sharpening.

VirginiaPoundstone triaged this task as Medium priority.Oct 25 2022, 1:50 PM

In T318552#8330519, @AntiCompositeNumber wrote:

Conditional sharpening is applied to JPEG files, but not to PNG files. Does the level of blurriness change between test and production PNG thumbnails? If they don't look different, it's probably the conditional sharpening.

I am not seeing blurriness for PNGs.

Here is the config I am currently using: P37284

It does have:

FILTERS = [
    'wikimedia_thumbor.filter.conditional_sharpen',
    'wikimedia_thumbor.filter.lang',
    'wikimedia_thumbor.filter.page',
    'thumbor.filters.format',
    'thumbor.filters.quality'
]

So conditional sharpening should be loaded.

EDIT: So, I find the sharpening does occur locally if I generate the thumbnail with a url like: http://localhost:8800/thumbor/unsafe/640x/filters:conditional_sharpen(0.0,0.8,1.0,0.0,0.85)/https://upload.wikimedia.beta.wmflabs.org/wikipedia/commons/a/a4/Lens_lens.jpg

I think the difference in fuzziness between my local and beta is due to filters:conditional_sharpen(0.0,0.8,1.0,0.0,0.85) in the url. I am therefore marking this as invalid.

When I pass that parameter to my local and do the comparisons again, I see no difference compared to beta I can detect with my naked eye.

Testing en masse (with this script) the differences between beta and local with the parameter according to imagemagick's compare are very low (and much lower than when I was comparing beta to local without the parameter).

I have also compared local with production Thumbor (both en masse and by eye) and not seen any differences related to fuzziness.

It appears that that parameter gets automatically passed to thumbor on beta (I am not sure how, perhaps url rewriting).

For example:

dwalden@deployment-imagescaler03:~$ curl -I 'localhost:8800/wikipedia/en/thumb/1/1b/CuteGoat.jpg/640px-CuteGoat.jpg'
HTTP/1.1 200 OK
Thumbor-Memcache-Get-Time: 252
Thumbor-Wikimedia-Original-Path: 1/1b/CuteGoat.jpg
Thumbor-Request-Date: Tue, 22 Nov 2022 17:17:08 GMT
Thumbor-Swift-Thumbnail-Success-Time: 55
Content-Disposition: inline;filename*=UTF-8''CuteGoat.jpg
Content-Length: 62298
Thumbor-Wikimedia-Thumbnail-Path: 1/1b/CuteGoat.jpg/640px-CuteGoat.jpg
Thumbor-Wikimedia-Original-Container: wikipedia-en-local-public.1b
Server: Thumbor/6.3.2
Thumbor-Wikimedia-Thumbnail-Container: wikipedia-en-local-thumb.1b
Etag: "f5bc884200dca50560927a6293a2247b728bace7"
Thumbor-Parameters: {"width": "640", "image": "http://deployment-ms-fe04.deployment-prep.eqiad1.wikimedia.cloud/v1/AUTH_mw/wikipedia-en-local-public.1b/1/1b/CuteGoat.jpg", "filters": "conditional_sharpen(0.0,0.8,1.0,0.0,0.85):format(jpg)"}
Thumbor-Poolcounter-Time: 32
Date: Tue, 22 Nov 2022 17:17:08 GMT
Xkey: File:CuteGoat.jpg
Proxy-Request-Date: 22/Nov/2022:17:17:08 +0000
Content-Type: image/jpeg
Proxy-Response-Date: 22/Nov/2022:17:17:08 +0000
X-Upstream: 127.0.0.1:8801
Thumbor-Request-Id: 7F000001:D9E6_7F000001:2260_637D0414_3D53:355C

	F35535964: local_480px-Cute_baby_goat.jpg
	Sep 27 2022, 7:20 AM

Thumbnails more "blurry"Closed, InvalidPublicBUG REPORTActions