Page MenuHomePhabricator

Thumbnail for 50MB TIFF file not shown: HTTP 429 error (due to profile info in TIFF file?)
Closed, DuplicatePublicBUG REPORT

Event Timeline

Aklapper renamed this task from File doesn't show, network tab of browser shows 429 error to Thumbnail for 50MB TIFF file not shown: HTTP 429 error.Sep 2 2019, 7:48 AM
Aklapper edited projects, added Thumbor; removed Wikimedia-production-error.

The file itself is shown; its thumbnail is not shown. :)
@Mbch331: Adding Thumbor as this is about thumbnail generation; removing Wikimedia-production-error (see its description).

'Too many requests' just means that throttling is being applied, because the attempt to generate the thumbnail failed too often in a row. Probably the thumbnail renderer is running into memory limits.

This is a huge Tiff image btw. Is that supposed to be the entire book ? Because in that case, the tiff is missing important markup information that lists the various pages. That would explain why the software doesn't recognise it as paged and has trouble rendering it.

I think this is an upload by @Nemo_bis, maybe he can help figuring out this file.

The Tiff contains a single uncompressed bitmap, 3668 × 4660 × 3 = 51MB.

@Bdijkstra i tried compressing it in Preview.app and then it turned into 64MB, Thats what made me suspicious if this really was a single page. ;)

Here's an update on this particular case, and then my run-ins with the same problem and my workaround in the next post.

A new upload seems to have fixed this, though I'm not sure exactly what the modification was. I've left a message on the page of User:Marco Chemello (BEIC)

https://commons.wikimedia.org/w/index.php?title=File:Moivre_-_Doctrine_of_chances,_1761_-_722666.tif&diff=378007661&oldid=338988816

So I ran into this same problem with uploads from the Smithsonian with 40-60 Mbyte TIFF files. TIFF thumbnails don't render, and JPG versions give the same error above.

Example of problem: https://commons.wikimedia.org/wiki/File:S-NPG_82_77.tif

I found that if I download the TIFF file, then strip out the profile info using ImageMagick, then the re-uploaded TIFF then does fine.

Example of fixed version: https://commons.wikimedia.org/wiki/File:S-NPG_82_76.tif

I ran "convert -quiet infile.tiff -strip outfile.tiff"

Does this give any insights on how a fix might be done on the server end? Is there a way to fail gracefully or at least with a meaningful error message? In this case, the ICC profile was for "ProPhoto RGB" which I believe is Adobe Lightroom. Would downloading the ICC profile to the server side help alleviate this problem? Would doing some stripping on the server side be appropriate?

I'd rather not have to preprocess all the files on my end to strip profile info before uploading.

Aklapper renamed this task from Thumbnail for 50MB TIFF file not shown: HTTP 429 error to Thumbnail for 50MB TIFF file not shown: HTTP 429 error (due to profile info in TIFF file?).Apr 29 2020, 9:58 AM

Sorry for the delay... The bug was solved but I'll add some quick context.

This is a huge Tiff image btw. [...]
I think this is an upload by @Nemo_bis, maybe he can help figuring out this file.

This is a rather typical size for a one-page master. Some of these books are many GiB heavy. In general it might be advisable to make some lossless conversion before the upload, for instance a simple convert -compress LZW reduces this file to 24 MiB.

For BEIC, however, we just decided to upload the original masters as they were, in the maximum quality available. Sadly the suppliers of the scans often use bizarre formats. This seems to be a scanner which was marketed around 2005 or so:

$ identify -verbose Moivre_-_Doctrine_of_chances,_1761_-_722666.tif | grep -A 12 Properties
  Properties:
    date:create: 2020-09-08T10:27:04+00:00
    date:modify: 2020-05-12T16:04:12+00:00
    icc:copyright: Copyright by LOGO GmbH, Steinfurt
    icc:description: OS12000_mG
    signature: 0c70e553dfcd5993a29c3222a5493a55d155621cc4a163f68927ffc733963c56
    tiff:alpha: unspecified
    tiff:artist: Zeutschel Omniscan 11
    tiff:endian: lsb
    tiff:photometric: RGB
    tiff:rows-per-strip: 1
    tiff:software: Omniscan 11.10 SR1 Build8
    tiff:timestamp: 2008:02:28 14:10:25

https://web.archive.org/web/20200908103454/http://www.archidigita.de/uploads/media/os11.pdf
https://web.archive.org/web/20200908103623/https://www.zeutschel.de/en/produkte/scanner/farbscanner/os-12000-din-a1.html