At some point, things just become insane, but we should probably see which resource is the limiting factor for "huge" (>140MP) files.
Example files:
[[:file:Agriculture_characterisation_map,_Flanders.png]], [[:file:Basemap_for_Afil.png]], [[:file:Drapeau_hillion.png]]
Version: unspecified
Severity: normal
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T43371 Thumbnail/imagescaler (tracking) | |||
| Open | None | T108517 PNG thumbnails issues (tracking) | |||
| Resolved | None | T54050 VIPS doesn't scale PNGs larger than roughly 140 megapixels | |||
| Resolved | • Dcoetzee | T11497 Thumbnails of large PNGs are not generated |
Testing locally (Important, locally I'm not using cgroups, along with probably a different version of vips (vips-7.34.2), so these local tests might not mean much)
[[:file:Agriculture_characterisation_map,_Flanders.png]],
This one worked fine locally (It did require more than 100mb of memory, but still less than 400mb).
[[:file:Basemap_for_Afil.png]]
Seems to need a bigger temporary file (575 mb seemed to do it)
, [[:file:Drapeau_hillion.png]]
Seems to need a 738 mb temp file.
So it would appear increasing $wgMaxShellFileSize to say 1 GB would decrease the number of images we fail to resize by quite a bit. Not sure if that's something viable to do. At the very least, probably not until bug 52203 is fixed.
(In reply to comment #1)
At the very least, probably not until bug 52203 is
fixed.
Well, if your bug 52203 comment 4 is correct, raising the limit to fix this bug would be a workaround for that bug too, because if uninterrupted the extension cleans up its garbage by itself.
The existing workaround now in place seems to make disk storage not a problem, but I'm sure that's not the only consideration.
(In reply to comment #3)
(In reply to comment #1)
At the very least, probably not until bug 52203 is
fixed.Well, if your bug 52203 comment 4 is correct, raising the limit to fix this
bug
would be a workaround for that bug too, because if uninterrupted the
extension
cleans up its garbage by itself.
The existing workaround now in place seems to make disk storage not a
problem,
but I'm sure that's not the only consideration.
J had a tweak to the vips command line that should significantly reduce size of temp files (which should be deployed next week). That may stop many of the failures.
Proper solution to the temp file issue would maybe be to create temporary working directory for vips and delete everything in directory when done.
In regards to j's tweak - http://test.wikipedia.org/wiki/File:Huge.png is 160 mp and scaling fine now. True test will come when changes hit commons next week.
Ok, these are the biggest pngs (all pngs bigger than 170 MP) on commons:
MariaDB [commonswiki_p]> select img_name, img_width*img_height from image where img_media_type = 'BITMAP' and img_minor_mime = 'png' and img_major_mime = 'image' and img_width *img_height > 170000000 order by img_width*img_height desc;
+-----------------------------------------------------------+----------------------+
| img_name | img_width*img_height |
+-----------------------------------------------------------+----------------------+
| Iowa_County,_Wisconsin,_Courthouse_-_Icon_-_3000-dpi.png | 4161055734 |
| Mandelbrot_fractal_large.png | 715776516 |
| Mandelbrot_High_Resolution.png | 600000000 |
| Iowa_County,_Wisconsin,_Courthouse_-_Icon_-1000-dpi.png | 462349287 |
| Gonders_flats.png | 375955914 |
| U-undStraßenbahnGleisplanBerlinundUmgebung2013.png | 293840000 |
| Union_Stock_Yards,_Omaha,_1899.png | 260564824 |
| Union_Stock_Yards,_Omaha,_1925.png | 259465592 |
| Union_Stock_Yards,_Omaha,_1958.png | 258749920 |
| Union_Stock_Yards,_Omaha,_1916.png | 258264072 |
| Union_Stock_Yards,_Omaha,_1999.png | 257947776 |
| Union_Stock_Yards,_Omaha,_1887.png | 257896800 |
| Redstone_Rocket_Test_Stand_HAER_AL-129-A_sheet_1_of_7.png | 241231872 |
| World4-isometric_north.png | 200950784 |
| Rensselaerswyck_Original_Map_Large.png | 195843032 |
| Drapeau_hillion.png | 192853712 |
| Income_inequality_in_the_United_States,_1979-2007.png | 174420000 |
+-----------------------------------------------------------+----------------------+
enwiki has no pngs bigger than 170 MP
I tested the top 5, they all seem to render. I will note, that [[commons:file:Iowa_County,_Wisconsin,_Courthouse_-_Icon_-_3000-dpi.png]] doesn't seem to render the courthouse part of the image (compare with [[commons:file:Iowa_County,_Wisconsin,_Courthouse_-_Icon_-1000-dpi.png]]). I'm not sure if that's a problem with our rendering, or if the original image is just broken. I'm going to go with the original image is just broken unless someone comes up with evidence to the contrary.
In any case, marking this bug fixed