Page MenuHomePhabricator

Undeploy VipsScaler from Wikimedia wikis
Closed, ResolvedPublic

Description

VipsScaler was used for thumbnailing especially large files, but is no longer used, we moved to Thumbor years ago and it uses vips too: T119184: Thumbor VIPS engine.

Special:VipsTest will disappear from wikis, but it's always explicitly been for testing.

  • group0
  • group1
  • group2

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 720356 merged by jenkins-bot:

[operations/mediawiki-config@master] Undeploy VipsScaler: II \u2013 Don't load regardless of config

https://gerrit.wikimedia.org/r/720356

Mentioned in SAL (#wikimedia-operations) [2021-09-13T23:41:14Z] <jforrester@deploy1002> Synchronized wmf-config/CommonSettings.php: T290759: Undeploy VipsScaler: II – Don't load regardless of config (duration: 00m 58s)

Change 720357 merged by jenkins-bot:

[operations/mediawiki-config@master] Undeploy VipsScaler: III \u2013 Don't set wmgUseVips, now ignored

https://gerrit.wikimedia.org/r/720357

Mentioned in SAL (#wikimedia-operations) [2021-09-13T23:45:31Z] <jforrester@deploy1002> Synchronized wmf-config/InitialiseSettings.php: T290759: Undeploy VipsScaler: III – Don't set wmgUseVips, now ignored (duration: 00m 58s)

Change 720358 merged by jenkins-bot:

[operations/mediawiki-config@master] Undeploy VipsScaler: IV \u2013 Don't load the i18n

https://gerrit.wikimedia.org/r/720358

Change 720359 merged by jenkins-bot:

[mediawiki/tools/release@master] Stop branching VipsScaler for Wikimedia production

https://gerrit.wikimedia.org/r/720359

Change 720974 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/puppet@production] mediawiki::packages: remove libvips

https://gerrit.wikimedia.org/r/720974

hashar subscribed.

After VipsScaler has been disabled on all wikis, we started seeing errors fetching thumbnails mimetype which is T290973 . It might be unrelated though.

Because of T291010: MediaWiki refuses to generate thumbnail URLs for large PNGs and TIFFs with "Error creating thumbnail: File with dimensions greater than 100 MP" and T290973: Error: Call to a member function getMimeType() on null we're going to put VipsScaler back for now, even though it wasn't actually shelling out to vips, its hooks were still doing other things that apparently we depend on. Those things need to be fixed but not by just seeing what breaks :/

I'll work on reploying it now.

Change 721092 had a related patch set uploaded (by Legoktm; author: Legoktm):

[mediawiki/core@wmf/1.37.0-wmf.23] Re-add VipsScaler

https://gerrit.wikimedia.org/r/721092

Change 721092 merged by Legoktm:

[mediawiki/core@wmf/1.37.0-wmf.23] Re-add VipsScaler

https://gerrit.wikimedia.org/r/721092

Mentioned in SAL (#wikimedia-operations) [2021-09-14T22:11:00Z] <legoktm@deploy1002> Started scap: Rebuild i18n for redeployment of VipsScaler (T290759)

Mentioned in SAL (#wikimedia-operations) [2021-09-14T22:34:49Z] <legoktm@deploy1002> Finished scap: Rebuild i18n for redeployment of VipsScaler (T290759) (duration: 23m 49s)

Change 721098 had a related patch set uploaded (by Legoktm; author: Legoktm):

[operations/mediawiki-config@master] Revert \"Undeploy VipsScaler: I, II & III\"

https://gerrit.wikimedia.org/r/721098

Change 721098 merged by jenkins-bot:

[operations/mediawiki-config@master] Revert \"Undeploy VipsScaler: I, II & III\"

https://gerrit.wikimedia.org/r/721098

Change 720974 merged by Giuseppe Lavagetto:

[operations/puppet@production] mediawiki::packages: remove libvips

https://gerrit.wikimedia.org/r/720974

@Jdforrester-WMF: Per emails from Sep18 and Oct20 and https://www.mediawiki.org/wiki/Bug_management/Assignee_cleanup , I am resetting the assignee of this task because there has not been progress lately (please correct me if I am wrong!). Resetting the assignee avoids the impression that somebody is already working on this task. It also allows others to potentially work towards fixing this task. Please claim this task again when you plan to work on it (via Add Action...Assign / Claim in the dropdown menu) - it would be welcome. Thanks for your understanding!

Change 977250 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/mediawiki-config@master] Disable VipsScaler in group0

https://gerrit.wikimedia.org/r/977250

Change 977250 merged by jenkins-bot:

[operations/mediawiki-config@master] Disable VipsScaler in group0

https://gerrit.wikimedia.org/r/977250

Mentioned in SAL (#wikimedia-operations) [2023-11-28T20:29:04Z] <ladsgroup@deploy2002> Started scap: Backport for [[gerrit:977250|Disable VipsScaler in group0 (T290759)]]

Mentioned in SAL (#wikimedia-operations) [2023-11-28T20:30:28Z] <ladsgroup@deploy2002> ladsgroup: Backport for [[gerrit:977250|Disable VipsScaler in group0 (T290759)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2023-11-28T20:39:13Z] <ladsgroup@deploy2002> Finished scap: Backport for [[gerrit:977250|Disable VipsScaler in group0 (T290759)]] (duration: 10m 08s)

This doesn't fail anymore https://test.wikipedia.org/w/index.php?title=User:Legoktm/sandbox&diff=prev&oldid=583704 but maybe because the image is in commons? Anyway, I make sure T291010#7353546 doesn't happen again before moving forward.

This doesn't fail anymore https://test.wikipedia.org/w/index.php?title=User:Legoktm/sandbox&diff=prev&oldid=583704 but maybe because the image is in commons? Anyway, I make sure T291010#7353546 doesn't happen again before moving forward.

Are we happy to proceed?

I played with it and it's waaay more complicated than it looks. So I much that I gave up. If someone is willing to work on it, I'd be happy to help.

We should definitely do T291014 and then this, but I'm not doing it right now.

We should definitely do T291014 and then this, but I'm not doing it right now.

I had a look and I don't think there's anything to do there now, T291014#10889064

In T260504, https://gerrit.wikimedia.org/r/c/mediawiki/core/+/664086 added a disableLocalTransform which was turned on of k8s in January in T352515 / https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/987432. If calling transform was still heading us down that path, presumably we'd have seen it.

I played with it and it's waaay more complicated than it looks. So I much that I gave up. If someone is willing to work on it, I'd be happy to help.

Is there anything you remember seeing that was giving you pause? Otherwise, I think it might be worth trying to proceed to group1.

Change #1154128 had a related patch set uploaded (by Arlolra; author: Arlolra):

[operations/mediawiki-config@master] Disable VipsScaler in group1

https://gerrit.wikimedia.org/r/1154128

I played with it and it's waaay more complicated than it looks. So I much that I gave up. If someone is willing to work on it, I'd be happy to help.

Is there anything you remember seeing that was giving you pause? Otherwise, I think it might be worth trying to proceed to group1.

My main problem was the $wgMaxImageArea issue which when the extension was disabled a lot of png uploads started to error out which is coming from this hook handler: https://github.com/wikimedia/mediawiki-extensions-VipsScaler/blob/master/includes/VipsScaler.php#L393C18-L393C47 I tried porting that logic to core https://github.com/wikimedia/mediawiki-extensions-VipsScaler/blob/master/includes/VipsScaler.php#L299 (::getHandlerOptions()) but it was so messy I gave up. Maybe it's fixed via different route. If you think so (T291014#7578524) then let's go ahead.

Maybe it's fixed via different route. If you think so (T291014#7578524) then let's go ahead.

Yeah, without the VipsScaler hook, isImageAreaOkForThumbnaling should return early, based on the value in production.

But from T290759#10889091, we shouldn't really be in TransformationalImageHandler::doTransform to begin with. If we are, the production value is set based on Thumbor and we're about to do local scaling, which doesn't seem right.

Maybe it's fixed via different route. If you think so (T291014#7578524) then let's go ahead.

Yeah, without the VipsScaler hook, isImageAreaOkForThumbnaling should return early, based on the value in production.

But from T290759#10889091, we shouldn't really be in TransformationalImageHandler::doTransform to begin with. If we are, the production value is set based on Thumbor and we're about to do local scaling, which doesn't seem right.

To my knowledge, I don't think we do transform locally, the containers shouldn't even have the packages to do it. Regardless, sure it'd be definitely quite rare otherwise we would notice it.

I'm not sure how meaningful this is but I downloaded sampling trace logs from https://performance.wikimedia.org/arclamp/logs/daily/ and got

λ grep onBitmapHandlerCheckImageArea 2025-06-09.excimer.index.log | wc -l
     223
λ grep onBitmapHandlerTransform 2025-06-09.excimer.index.log | wc -l
       0

Change #1154128 merged by jenkins-bot:

[operations/mediawiki-config@master] Disable VipsScaler in group1

https://gerrit.wikimedia.org/r/1154128

Mentioned in SAL (#wikimedia-operations) [2025-06-09T20:03:57Z] <arlolra@deploy1003> Started scap sync-world: Backport for [[gerrit:1154128|Disable VipsScaler in group1 (T290759)]]

Mentioned in SAL (#wikimedia-operations) [2025-06-09T20:05:56Z] <arlolra@deploy1003> arlolra: Backport for [[gerrit:1154128|Disable VipsScaler in group1 (T290759)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-06-09T20:14:20Z] <arlolra@deploy1003> Finished scap sync-world: Backport for [[gerrit:1154128|Disable VipsScaler in group1 (T290759)]] (duration: 10m 23s)

Change #1156515 had a related patch set uploaded (by Arlolra; author: Arlolra):

[operations/mediawiki-config@master] Disable VipsScaler in group2

https://gerrit.wikimedia.org/r/1156515

Change #1156515 merged by jenkins-bot:

[operations/mediawiki-config@master] Disable VipsScaler in group2

https://gerrit.wikimedia.org/r/1156515

Mentioned in SAL (#wikimedia-operations) [2025-06-16T20:10:01Z] <arlolra@deploy1003> Started scap sync-world: Backport for [[gerrit:1156515|Disable VipsScaler in group2 (T290759)]]

Mentioned in SAL (#wikimedia-operations) [2025-06-16T20:11:59Z] <arlolra@deploy1003> arlolra: Backport for [[gerrit:1156515|Disable VipsScaler in group2 (T290759)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-06-16T20:19:54Z] <arlolra@deploy1003> Finished scap sync-world: Backport for [[gerrit:1156515|Disable VipsScaler in group2 (T290759)]] (duration: 09m 53s)

Change #1159599 had a related patch set uploaded (by Arlolra; author: Arlolra):

[operations/mediawiki-config@master] Undeploy VipsScaler

https://gerrit.wikimedia.org/r/1159599

Change #1159599 merged by jenkins-bot:

[operations/mediawiki-config@master] Undeploy VipsScaler

https://gerrit.wikimedia.org/r/1159599

Mentioned in SAL (#wikimedia-operations) [2025-06-25T20:03:47Z] <arlolra@deploy1003> Started scap sync-world: Backport for [[gerrit:1159599|Undeploy VipsScaler (T290759)]]

Mentioned in SAL (#wikimedia-operations) [2025-06-25T20:34:41Z] <arlolra@deploy1003> arlolra: Backport for [[gerrit:1159599|Undeploy VipsScaler (T290759)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-06-25T20:51:25Z] <arlolra@deploy1003> Finished scap sync-world: Backport for [[gerrit:1159599|Undeploy VipsScaler (T290759)]] (duration: 47m 37s)

Change #1164233 had a related patch set uploaded (by Arlolra; author: Arlolra):

[integration/config@master] Move VipsScaler to other

https://gerrit.wikimedia.org/r/1164233

Change #1164233 merged by jenkins-bot:

[integration/config@master] Move VipsScaler to other

https://gerrit.wikimedia.org/r/1164233