Thumbor is supposed to be WMF's image scaling system. However, the pre-Thumbor image scaling system in MediaWiki still thinks it is responsible for image scaling, is still maintained, and is still doing hundreds of image rendering and scaling operations per day. T260330 is a proposal to split out MediaWiki shell execution to a service. It would simplify that project if we used the pre-existing Thumbor service when MediaWiki requests immediate image scaling, or if we avoided those calls in the first place, rather than building a new Thumbor-like thing to replace those shell executions.
Code search indicates that File::transform() is called with the RENDER_NOW flag from the following places:
- ThumbnailRenderJob (should be prevented by $wgUploadThumbnailRenderMethod)
- SpecialUploadStash (should be prevented by $wgUploadStashScalerBaseUrl)
- thumb.php (should be prevented by thumbProxyUrl)
- PagedTiffHandler
- TimedMediaThumbnail
Analysis of exec.log shows convert being called due to UploadStash, mostly via a job, e.g.
2020-08-06 12:39:28 [fa727d75-2be8-4ab4-831f-ac72a3aa1150] mw1387 idwiki 1.36.0-wmf.2 exec DEBUG: MediaWiki\Shell\Command::execute: /bin/bash '/srv/mediawiki/php-1.36.0-wmf.2/includes/shell/limit.sh' 'OMP_NUM_THREADS='\''1'\'' MAGICK_TMPDIR='\''/tmp/magick-tmp'\'' /usr/bin/firejail --quiet --profile=/srv/mediawiki/php-1.36.0-wmf.2/includes/shell/firejail.profile --blacklist=/srv/mediawiki/php-1.36.0-wmf.2/LocalSettings.php --noroot --seccomp --private-dev -- '\''/usr/local/bin/mediawiki-firejail-convert'\'' '\''-quality'\'' '\''80'\'' '\''-background'\'' '\''white'\'' '\''-define'\'' '\''jpeg:size=120x151'\'' '\''/tmp/localcopy_2031ebe257d3.jpg'\'' '\''-thumbnail'\'' '\''120x151!'\'' '\''-set'\'' '\''comment'\'' '\''File source: https://id.wikipedia.org/wiki/Istimewa:UploadStash/file/17o6wwd3amws.xc7gzv.1023578.jpg'\'' '\''+set'\'' '\''Thumb::URI'\'' '\''-depth'\'' '\''8'\'' '\''-sharpen'\'' '\''0x0.8'\'' '\''-rotate'\'' '\''-0'\'' '\''-sampling-factor'\'' '\''2x2,1x1,1x1'\'' '\''/tmp/transform_c48664777a79.jpg'\''' 'MW_INCLUDE_STDERR=1;MW_CPU_LIMIT=50; MW_CGROUP='\''/sys/fs/cgroup/memory/mediawiki/job'\''; MW_MEM_LIMIT=1048576; MW_FILE_SIZE_LIMIT=524288; MW_WALL_CLOCK_LIMIT=180; MW_USE_LOG_PIPE=yes'
Trying to filter out UploadStash mostly just gives me other translations of UploadStash. Other conversion types (vips, rsvg, etc.) are also present but don't have such useful debugging information in exec.log. Maybe once UploadStash is fixed, the log entries will go away and we can make transform(RENDER_NOW) throw an exception. Then we can uninstall the file transform utilities on the appservers.