I checked software updates deployed on the Thumbor hosts, but none of the multimedia lib updates add it. Timing-wise the most likely source of error is the python-thumbor-wikimedia update which was deployed on the 26th of June. I'll investigate this further tomorrow.

I looked into the changes between python-thumbor-wikimedia 1.16 and 2.0 and the most likely cause for this bug are the changes made for T27611:
https://github.com/wikimedia/operations-debs-python-thumbor-wikimedia/commit/23b589a5b56257e37c82840a3e091cceae72a959

Another case where this happens is this image:
https://de.wikipedia.org/wiki/Datei:Logo_PeerTube.png

here the transparent background gets black:

This image successfully retains the transparent background layer when previewed on its File: page, but when used in the article it was designated for, the transparent layer still does not appear, and it is still a solid white.

(unsure if this is related)

This image is in .png and has a transparent background when the original is opened directly, but a white background in the media viewer and infobox. It also has a white background in all versions other than the largest, original file.
https://upload.wikimedia.org/wikipedia/commons/6/60/SpaceX_CRS-15_Patch.png

Sorry, can anybody explain why images from https://commons.wikimedia.org/wiki/Category:Transparent_squares are not affected? Look e. g. at https://upload.wikimedia.org/wikipedia/commons/thumb/0/02/Transparent_square.svg/512px-Transparent_square.svg.png – the SVG⇒PNG conversion occurred today, the PNG wasn’t cached from þe olde goode times, and it is fully transparent. Some essential conditions appear to be missing from description of the bug.

Oops, from https://en.wikipedia.org/wiki/Wikipedia:Village_pump_%28technical%29/Archive_167#Transparent_png_files and other discussions this is irrelevant to SVG at all, but something with Web typesetting by MediaWiki. Missed the point.

@PhilipTerryGraham: As you increased the priority of this task, do you plan to work on fixing this bug?

It appears that "High" has been redefined to "currently working on." So then I must ask, how is one supposed to indicate that a project should be higher priority if you've redefined "High" to mean "currently working on"? I don't see a way to vote on a bug.

It is quite surprising to me that this has taken so long. This is a regression, and the update that caused it has been identified. Yet there has been no talk of simply pulling out the change until it can be reapplied without breaking anything. I'd get it if that's not possible because of other changes or security issues, but nothing like that has been proposed.

Is there any reason you don't just roll back to Python 1.16 until you can figure out how to get this working on Python 2.0?

Once this is resolved, is there any way to programmatically regenerate all PNG thumbnails created since the change to python-thumbor-wikimedia 2.0?

Oops, I don't know how those projects got removed...

@Aklapper I wouldn't know how to fix this, I'm just a Wikipedia editor that was pointed to this task by fellow editors about a month ago. I'm simply trying to express the most amount of frustration with the least amount of repercussions towards the fact that even after two months, 30 subscribers to the task, and potentially hundreds broken PNG files strewn about Wikimedia, work towards fixing this relatively major flaw seems non-existent. I'm sorry if I crossed a line or two, but I feel like two months of sitting on something this major is too long. If there's a better way to express frustration and prompt more immediate action in situations like this, be sure to let me know.

In T198370#4537063, @PhilipTerryGraham wrote:

I'm sorry if I crossed a line or two, but I feel like two months of sitting on something this major is too long. If there's a better way to express frustration and prompt more immediate action in situations like this, be sure to let me know.

Please appreciate that there are a lot of bugs, few people to work on these, fewer people with knowledge of this particular area and even fewer people during a vacation period.
This is a non-destructive problem, so it's priority is relatively low compared to the many other day-to-day problems.

I've found the source, it's a change in the resizing/cropping logic (old logic was incorrect in specific cases) that triggers what is likely to be an IM bug on Debian Jessie:

convert -define tiff:exif-properties=no -resize 400x300 -quality 95 -sampling-factor 4:2:0 ~/thumbor-plugins/tests/integration/originals/PNG_transparency_demonstration_1.png png:- > old.png

convert -define tiff:exif-properties=no -resize 400x300^ -gravity center -extent 400x300 -quality 95 -sampling-factor 4:2:0 ~/thumbor-plugins/tests/integration/originals/PNG_transparency_demonstration_1.png png:- > new.png

The newer IM version I have locally isn't affected (same command as above running on OS X/IM 7.0.8-2):

This was actually caught by existing tests during the big refactor, but I overrode the value when the test failed on Jessie (it passed locally), assuming it was something to do with the compression of the alpha layer on the IM version we have running in production, when in fact the alpha layer was just not transparent at all.

Either I find a workaround, or PNGs will have to live with the slightly incorrect resizing/cropping in edge cases we used to have until we upgrade Debian/IM on the Thumbor hosts.

Correct transparency is more critical than off-by-one-pixel sizes (with the simpler IM old IM resizing command that retains transparency on Jessie, images can sometimes be 1px wider or narrower than requested).

Change 456145 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 2.01

https://gerrit.wikimedia.org/r/456145

Change 456145 merged by Filippo Giunchedi:
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 2.1

https://gerrit.wikimedia.org/r/456145

Since potentially tens of thousands differrnt sized thumbnails were affected by this bug, will the server eventually regenerate all of these, or will they need to be purged manually?

@Ahecht it seems to be fixing itself... rather slowly. Most of the files I've uploaded when the transprency layer bug was around now have their transparency layers in thumbnails, but some still have the opaque background. For example, the lead images in the Microsoft Office 2019 and Microsoft Word articles on the English Wikipedia now appear as intended, but the image on the Microsoft Excel article still has an opaque background; my browser cache has been cleared and both the article and file page have been purged. Though, I suspect this will be fixed naturally as the server catches up with all these files.

They won't necessarily all fix themselves, no, this is why I mentioned that they need purging to ensure that they're fixed. I can purge all PNGs, but that's going to be quite time-consuming.

@Gilles can the everyday Wikipedia user such as myself help in the process of purging these PNGs? The usual "?action=purge" URL technique doesn't seem to work.

That's action=purge yes. After you've done that, you need to make sure that you clear your browser cache or to shift-refresh the page where the thumbnail is, otherwise you'll keep seeing the old incorrect thumbnail coming from your browser cache.

@Gilles I should also clarify, just as I had already said earlier, the opaque backgrounds still appear even after my browser's cache was cleared.

In T198370#4547673, @PhilipTerryGraham wrote:

@Gilles I should also clarify, just as I had already said earlier, the opaque backgrounds still appear even after my browser's cache was cleared.

Please always provide specific examples so others can reproduce (or not). Which browser? Also see https://en.wikipedia.org/wiki/Wikipedia:Bypass_your_cache

@Aklapper Firefox Nightly 63.0, Windows 10 version 1803, Ctrl+F5 doesn't work either; I've been trying on the Microsoft Excel English Wikipedia article to get the lead image to show up with a transparent layer with all three aforementioned methods.

Ah, you need to purge the file itself, not the article it appears in. I.e. https://en.wikipedia.org/wiki/File:Microsoft_Excel.png?action=purge

I'll let you try it yourself, I didn't purge it.

@Gilles This works! Thanks for the tip; I'll go ahead and purge as many PNGs as I can!

In T198370#4547738, @PhilipTerryGraham wrote:

@Gilles This works! Thanks for the tip; I'll go ahead and purge as many PNGs as I can!

You can try T170039#3473755.

Doesn't seem to be completely fixed: see

• https://en.wikipedia.org/wiki/File:Tummy_Touch_logo.png
• https://en.wikipedia.org/wiki/File:Cincinnati_Bell_logo.png

Both are PNG8 (aka Indexed PNG) if that's relevant.

Change 461793 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 2.2

https://gerrit.wikimedia.org/r/461793

Change 461793 merged by Filippo Giunchedi:
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 2.2

https://gerrit.wikimedia.org/r/461793

Mentioned in SAL (#wikimedia-operations) [2018-09-24T09:30:06Z] <godog> upgrade / roll restart thumbor in eqiad / codfw - T20871 T198370

Yay!

How about "Greyscale with Alpha" images? https://commons.wikimedia.org/wiki/File:Quillette.png seems to still be getting the white background as described in this task.

Also it seems to do the white background with RGB and greyscale images using the tRNS chunk rather than a full alpha channel:

• https://test.wikipedia.org/wiki/File:RGB_with_tRNS.png
• https://test.wikipedia.org/wiki/File:Greyscale_with_tRNS.png

Thanks for the test files, the fix looks straightforward at a glance.

Change 487785 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/software/thumbor-plugins@master] Fix PNG transparency for more cases

https://gerrit.wikimedia.org/r/487785

Change 487785 merged by Gilles:
[operations/software/thumbor-plugins@master] Fix PNG transparency for more cases

https://gerrit.wikimedia.org/r/487785

Change 488037 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/software/thumbor-plugins@master] Version bump

https://gerrit.wikimedia.org/r/488037

Change 488037 merged by Gilles:
[operations/software/thumbor-plugins@master] Version bump

https://gerrit.wikimedia.org/r/488037

Change 488060 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 2.3

https://gerrit.wikimedia.org/r/488060

Change 488060 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 2.3

https://gerrit.wikimedia.org/r/488060

Change 488060 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 2.3

https://gerrit.wikimedia.org/r/488060

Change 488060 merged by Gilles:
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 2.3

https://gerrit.wikimedia.org/r/488060