It's 823px and below which gives working thumbs:

https://upload.wikimedia.org/wikipedia/commons/thumb/c/c2/014c04d520f6cb4286db835ee934a2d7.png/823px-014c04d520f6cb4286db835ee934a2d7.png

Is there anything particularly special about the test case being used? (You have it labelled as bug test case). Does this happen to a lot of files?

The error messages will more likely to be there once I manage to get gerrit change 83974 merged.

(In reply to comment #2)

Is there anything particularly special about the test case being used?

No, this is the test case because it does not work. I hope to get an error msg to find out why.

Testing locally with VIPS, things work fine. Not sure what's up with this file.

(In reply to comment #3)

(In reply to comment #2)

Is there anything particularly special about the test case being used?

No, this is the test case because it does not work. I hope to get an error
msg
to find out why.

In this sort of case, most likely issue is vips used either too much memory (more than 400 mb) or made a temporary file that was too big (> 400mb).

Although I can't imagine that happening for this file.

(In reply to comment #4)

Testing locally with VIPS, things work fine. Not sure what's up with this
file.

Which doesn't mean much, as this image is < 35 MP, so scaled with image magick.

[17:56] <Reedy> 2013-09-28 20:29:31 mw1156 commonswiki: thumbnail failed on mw1156: error 137 "" from "'/usr/bin/convert' -quality 95 -background white '/tmp/localcopy_d25f99b3976c-1.png' -thumbnail '1280x640!' -set comment 'File source: http://commons.wikimedia.org/wiki/File:014c04d520f6cb4286db835ee934a2d7.png' -depth 8 -rotate -0 '/tmp/transform_35daeff19465-1.png' 2>&1"

The error 137 (137 = 128+9) means that the process was killed. Typically this means it exceeded the memory limit (which is currently set to 400 mb)

I think the best solution would be to just start using VIPS for all png images.

Other files that appear to have memory limit problems:

*City_Park_Panorama_2.png at 2800px
*File:Deutsche_PrivatBriefe_Mittelalter_V2_014.png at 1280px

Additionally [[it:File:Stemma-Sofia.png]] fails a lot. This appears to be because the file itself is broken (invalid colour map index)

Bryan.TongMinh wrote:

(In reply to comment #7)

I think the best solution would be to just start using VIPS for all png
images.

Sounds like the best solution.
Alternatively, we could introduce a new condition so that we can trigger VipsScaler on all indexed PNGs.

(In reply to comment #9)

(In reply to comment #7)

I think the best solution would be to just start using VIPS for all png
images.

Sounds like the best solution.

Bug 51400 shoud be fixed first, of course.

(In reply to comment #9)

(In reply to comment #7)

I think the best solution would be to just start using VIPS for all png
images.

Sounds like the best solution.
Alternatively, we could introduce a new condition so that we can trigger
VipsScaler on all indexed PNGs.

I think it is not limited to indexed PNGs. Look @ https://commons.wikimedia.org/wiki/File:World_population_density_1994.png where Imagemagick fails to generate larger PNG thumbs.

(In reply to comment #11)

(In reply to comment #9)

(In reply to comment #7)

I think the best solution would be to just start using VIPS for all png
images.

Sounds like the best solution.
Alternatively, we could introduce a new condition so that we can trigger
VipsScaler on all indexed PNGs.

I think it is not limited to indexed PNGs. Look @
https://commons.wikimedia.org/wiki/File:World_population_density_1994.png
where
Imagemagick fails to generate larger PNG thumbs.

Hmm? That file looks fine to me.

It's only for the "larger" thumbs. E.g. https://upload.wikimedia.org/wikipedia/commons/thumb/6/67/World_population_density_1994.png/1007px-World_population_density_1994.png fails.

ranonenmacher wrote:

I've been seeing this problem with indexed PNG files that I've uploaded to wikimedia commons and have created the category: https://commons.wikimedia.org/wiki/Category:Images_that_will_not_render for the problem images.

I created a table on the category page that shows that the maximum renderable thumbnail width is inversely related to the image size, but also depends on the aspect ratio. The problem starts with very large thumbnails (~1200px wide) for images around 10 Mpx in size, and stops somewhere between 33 and 36 Mpx for thumbnails around 600px wide. Images larger than that don't seem to have a problem.

Anyway the table and images are there if anyone needs them to debug the problem.

(In reply to comment #14)

I've been seeing this problem with indexed PNG files that I've uploaded to
wikimedia commons and have created the category:
https://commons.wikimedia.org/wiki/Category:Images_that_will_not_render for
the
problem images.

I created a table on the category page that shows that the maximum renderable
thumbnail width is inversely related to the image size, but also depends on
the
aspect ratio. The problem starts with very large thumbnails (~1200px wide)
for
images around 10 Mpx in size, and stops somewhere between 33 and 36 Mpx for
thumbnails around 600px wide. Images larger than that don't seem to have a
problem.

Anyway the table and images are there if anyone needs them to debug the
problem.

It should stop precisely at 35 mp. That's when we switch rendering programs

ranonenmacher wrote:

(In reply to comment #15)

(In reply to comment #14)

I've been seeing this problem with indexed PNG files that I've uploaded to
wikimedia commons and have created the category:
https://commons.wikimedia.org/wiki/Category:Images_that_will_not_render for
the
problem images.

I created a table on the category page that shows that the maximum renderable
thumbnail width is inversely related to the image size, but also depends on
the
aspect ratio. The problem starts with very large thumbnails (~1200px wide)
for
images around 10 Mpx in size, and stops somewhere between 33 and 36 Mpx for
thumbnails around 600px wide. Images larger than that don't seem to have a
problem.

Anyway the table and images are there if anyone needs them to debug the
problem.

It should stop precisely at 35 mp. That's when we switch rendering programs

It does. I cropped an image to 5916 × 5916, which made it 34.999 mp. and it would not render at widths greater than 477px. I re-cropped it to 5917 × 5917 and it rendered fine at 1,024 × 1,024 pixels.

ranonenmacher wrote:

Would it be possible to switch to using Vips based on both the original image size and the desired thumbnail size? With the PNG-8 files that I've uploaded, I've found that the problem occurs consistently when:

original_image_size + 29.85 * thumbnail_size > 41.8 (+/- 0.1) (Megapixels)

Bryan.TongMinh wrote:

That's a rather complicated one, but we can trigger Vips on a certain minArea AND minShrinkFactor.

ranonenmacher wrote:

(In reply to comment #11)

(In reply to comment #9)

(In reply to comment #7)

I think the best solution would be to just start using VIPS for all png
images.

Sounds like the best solution.
Alternatively, we could introduce a new condition so that we can trigger
VipsScaler on all indexed PNGs.

I think it is not limited to indexed PNGs. Look @
https://commons.wikimedia.org/wiki/File:World_population_density_1994.png
where
Imagemagick fails to generate larger PNG thumbs.

I agree that it's not limited to indexed PNGs. I see the same problem with https://commons.wikimedia.org/wiki/File:First_colored_senator_and_reps.png and https://commons.wikimedia.org/wiki/File:Regla21.2.png which are 24-bit images (not indexed).

https://gerrit.wikimedia.org/r/117809 makes Vips be used for over 20MP images...

(In reply to Sam Reed (reedy) from comment #20)

https://gerrit.wikimedia.org/r/117809 makes Vips be used for over 20MP
images...

Deployed

ranonenmacher wrote:

This (https://gerrit.wikimedia.org/r/117809) still doesn't fix the problem for the files between ~12MP and 20MP that have this problem:

https://commons.wikimedia.org/wiki/File:Abies_concolor_range_map_2.png
https://commons.wikimedia.org/wiki/File:Acer_floridanum_range_map_1.png
https://commons.wikimedia.org/wiki/File:Acer_grandidentatum_range_map_1.png
https://commons.wikimedia.org/wiki/File:Aire_urbaine_Chamb%C3%A9ry_-_Savoie.png
https://commons.wikimedia.org/wiki/File:Artemisia_tridentata_range_map_1.png
https://commons.wikimedia.org/wiki/File:Austrian_Repressions_Against_Animal_Rights_Activists.png
https://commons.wikimedia.org/wiki/File:Byrsonima_lucida_range_map_2.png
https://commons.wikimedia.org/wiki/File:Cornus_rugosa_range_map_1.png
https://commons.wikimedia.org/wiki/File:India_roadway_map.png
:
etc.

Right now there are two files left in https://commons.wikimedia.org/wiki/Category:Images_that_will_not_render

• https://commons.wikimedia.org/wiki/File:Fraxinus_nigra_range_map_4.png with failing thumb e.g. https://upload.wikimedia.org/wikipedia/commons/thumb/f/f3/Fraxinus_nigra_range_map_4.png/1261px-Fraxinus_nigra_range_map_4.png
• https://commons.wikimedia.org/wiki/File:United_States_Statutes_at_Large_Volume_46_Part_2_scan_image_1407.png with failing thumb e.g. https://upload.wikimedia.org/wikipedia/commons/thumb/5/5c/United_States_Statutes_at_Large_Volume_46_Part_2_scan_image_1407.png/1321px-United_States_Statutes_at_Large_Volume_46_Part_2_scan_image_1407.png

In T56744#1290495, @McZusatz wrote:

Right now there are two files left [...]

I can no longer reproduce. This bug "fixed itself", imo.

Some things happened in the meanwhile, such as OS upgrades and changes in memory allocated to the various processes, so it's possible the memory issue was fixed by one of those. Closing as we don't have test cases; thanks McZusatz for the in-depth inspection of our files!