Page MenuHomePhabricator

Undeploy VipsScaler from Wikimedia wikis
Open, Needs TriagePublic

Description

VipsScaler was used for thumbnailing especially large files, but is no longer used, we moved to Thumbor years ago and it uses vips too: T119184: Thumbor VIPS engine.

Special:VipsTest will disappear from wikis, but it's always explicitly been for testing.

Event Timeline

Change 720355 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/mediawiki-config@master] Undeploy VipsScaler: I \u2013 Disable on all wikis

https://gerrit.wikimedia.org/r/720355

Change 720356 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/mediawiki-config@master] Undeploy VipsScaler: II \u2013 Don't load regardless of config

https://gerrit.wikimedia.org/r/720356

Change 720357 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/mediawiki-config@master] Undeploy VipsScaler: III \u2013 Don't set wmgUseVips, now ignored

https://gerrit.wikimedia.org/r/720357

Change 720358 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/mediawiki-config@master] Undeploy VipsScaler: IV \u2013 Don't load the i18n

https://gerrit.wikimedia.org/r/720358

Change 720359 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/tools/release@master] Stop branching VipsScalar for Wikimedia production

https://gerrit.wikimedia.org/r/720359

Legoktm renamed this task from Undeploy VipsScaler to Undeploy VipsScaler from Wikimedia wikis.Sep 10 2021, 6:23 PM
Legoktm updated the task description. (Show Details)

Even with Thumbor, it is/was still being used for some thumbnails (in the current config) - T199938: VipsScaler broken for MediaWiki production (/usr/bin/vips: No such file)

I'm presuming Thumbor will handle them fine...

In T199938, @Krinkle wrote:

With the deployment of Thumbor, MediaWiki in-process scaling is much rarer, but it is still used for scaling temporary files (UploadStash) before publishing to Swift.

This should be disproven / confirmed that we have somehow integrated Thumbor in a lambda-like way into MediaWiki where it can process even media files that aren't yet in Swift, as shown to the user during the upload process. I thought this didn't exist, but maybe it does.

Change 720355 merged by jenkins-bot:

[operations/mediawiki-config@master] Undeploy VipsScaler: I \u2013 Disable on all wikis

https://gerrit.wikimedia.org/r/720355

Mentioned in SAL (#wikimedia-operations) [2021-09-13T19:52:22Z] <jforrester@deploy1002> Synchronized wmf-config/InitialiseSettings.php: T290759 Undeploy VipsScaler: I – Disable on all wikis (duration: 00m 57s)

In T199938, @Krinkle wrote:

With the deployment of Thumbor, MediaWiki in-process scaling is much rarer, but it is still used for scaling temporary files (UploadStash) before publishing to Swift.

This should be disproven / confirmed that we have somehow integrated Thumbor in a lambda-like way into MediaWiki where it can process even media files that aren't yet in Swift, as shown to the user during the upload process. I thought this didn't exist, but maybe it does.

Tim recently changed it so MediaWiki no longer thumbnails at all: T260504: Get rid of remaining non-Thumbor MediaWiki image scaling in WMF production. For vips specifically, I looked through exec.log before bringing it up that the only invocations of vips in the past 30 days was from Special:VipsTest.

One this is done should we revert https://gerrit.wikimedia.org/r/c/operations/puppet/+/446640/ if all calls to Vips are going to be done on the Thumbor hosts?

One this is done should we revert https://gerrit.wikimedia.org/r/c/operations/puppet/+/446640/ if all calls to Vips are going to be done on the Thumbor hosts?

Yes, Reedy filed T290802: Remove libvips-tools from mediawiki appservers (parent task) for it.

One this is done should we revert https://gerrit.wikimedia.org/r/c/operations/puppet/+/446640/ if all calls to Vips are going to be done on the Thumbor hosts?

Yes, Reedy filed T290802: Remove libvips-tools from mediawiki appservers (parent task) for it.

Ha, apparently I'm blind. Thanks. :-)

In T199938, @Krinkle wrote:

With the deployment of Thumbor, MediaWiki in-process scaling is much rarer, but it is still used for scaling temporary files (UploadStash) before publishing to Swift.

This should be disproven / confirmed that we have somehow integrated Thumbor in a lambda-like way into MediaWiki where it can process even media files that aren't yet in Swift, […].

Tim recently changed it so MediaWiki no longer thumbnails at all: T260504 […]

Ack, thanks. I recall that task from last year but didn't know it had already been solved (as of Feb 2021). Neat. Consider my concern addressed.

Change 720356 merged by jenkins-bot:

[operations/mediawiki-config@master] Undeploy VipsScaler: II \u2013 Don't load regardless of config

https://gerrit.wikimedia.org/r/720356

Mentioned in SAL (#wikimedia-operations) [2021-09-13T23:41:14Z] <jforrester@deploy1002> Synchronized wmf-config/CommonSettings.php: T290759: Undeploy VipsScaler: II – Don't load regardless of config (duration: 00m 58s)

Change 720357 merged by jenkins-bot:

[operations/mediawiki-config@master] Undeploy VipsScaler: III \u2013 Don't set wmgUseVips, now ignored

https://gerrit.wikimedia.org/r/720357

Mentioned in SAL (#wikimedia-operations) [2021-09-13T23:45:31Z] <jforrester@deploy1002> Synchronized wmf-config/InitialiseSettings.php: T290759: Undeploy VipsScaler: III – Don't set wmgUseVips, now ignored (duration: 00m 58s)

Change 720358 merged by jenkins-bot:

[operations/mediawiki-config@master] Undeploy VipsScaler: IV \u2013 Don't load the i18n

https://gerrit.wikimedia.org/r/720358

Change 720359 merged by jenkins-bot:

[mediawiki/tools/release@master] Stop branching VipsScaler for Wikimedia production

https://gerrit.wikimedia.org/r/720359

Change 720974 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/puppet@production] mediawiki::packages: remove libvips

https://gerrit.wikimedia.org/r/720974

hashar added a subscriber: hashar.

After VipsScaler has been disabled on all wikis, we started seeing errors fetching thumbnails mimetype which is T290973 . It might be unrelated though.

Because of T291010: MediaWiki refuses to generate thumbnail URLs for large PNGs and TIFFs with "Error creating thumbnail: File with dimensions greater than 100 MP" and T290973: Error: Call to a member function getMimeType() on null we're going to put VipsScaler back for now, even though it wasn't actually shelling out to vips, its hooks were still doing other things that apparently we depend on. Those things need to be fixed but not by just seeing what breaks :/

I'll work on reploying it now.

Change 721092 had a related patch set uploaded (by Legoktm; author: Legoktm):

[mediawiki/core@wmf/1.37.0-wmf.23] Re-add VipsScaler

https://gerrit.wikimedia.org/r/721092

Change 721092 merged by Legoktm:

[mediawiki/core@wmf/1.37.0-wmf.23] Re-add VipsScaler

https://gerrit.wikimedia.org/r/721092

Mentioned in SAL (#wikimedia-operations) [2021-09-14T22:11:00Z] <legoktm@deploy1002> Started scap: Rebuild i18n for redeployment of VipsScaler (T290759)

Mentioned in SAL (#wikimedia-operations) [2021-09-14T22:34:49Z] <legoktm@deploy1002> Finished scap: Rebuild i18n for redeployment of VipsScaler (T290759) (duration: 23m 49s)

Change 721098 had a related patch set uploaded (by Legoktm; author: Legoktm):

[operations/mediawiki-config@master] Revert \"Undeploy VipsScaler: I, II & III\"

https://gerrit.wikimedia.org/r/721098

Change 721098 merged by jenkins-bot:

[operations/mediawiki-config@master] Revert \"Undeploy VipsScaler: I, II & III\"

https://gerrit.wikimedia.org/r/721098

Change 720974 merged by Giuseppe Lavagetto:

[operations/puppet@production] mediawiki::packages: remove libvips

https://gerrit.wikimedia.org/r/720974