Some half-broken PNGs cannot be rendered with the Jessie IM version
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Darius2807
	Oct 27 2017, 6:37 PM

Description

Podrobnosti o chybě jsou připojeny k hlášení, které bude veřejně dostupné. Pokud se vám to nelíbí, můžete níže hlášení upravit a odstranit všechny údaje, které nechcete sdílet.

Error details:

error: could not load image from https://upload.wikimedia.org/wikipedia/commons/thumb/b/b1/Nokia_3310_2017_DS.png/320px-Nokia_3310_2017_DS.png
URL: https://sk.wikipedia.org/wiki/Nokia_3310_2017#/media/File:Nokia_3310_2017_DS.png
user agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.62 Safari/537.36
screen size: 1366x768
canvas size: 1349x502
image size: 1265x2849
thumbnail size: CSS: 223x502, screen width: 223, real width: 320

This image can render fine on Stretch

Details

	Subject	Repo	Branch	Lines +/-
	Upgrade to 1.16	operations/debs/python-thumbor-wikimedia	master	+137 -22

Customize query in gerrit

Revisions and Commits

rTHMBREXT Thumbor Plugins
		rTHMBREXTcd54f47a777c Fix location of skip exception
		rTHMBREXTf4d1726c3859 Skip broken PNG test for now
	Restricted Differential Revision	rTHMBREXTf321adbfa13d Return any thumbnail ImageMagick managed to render

Related Objects
Search...

Status	Assigned	Task
Resolved	jijiki	T170817 Upgrade Thumbor servers to Stretch
Resolved	• Gilles	T179200 Some half-broken PNGs cannot be rendered with the Jessie IM version
Resolved	• Gilles	T179954 Thumbor errors should contain a trackable request id
Resolved	• Gilles	T186492 Improve Thumbor error logging
Resolved	• Gilles	T189647 Thumbor missing log extras when hitting ImageMagicException

Event Timeline

Darius2807 created this task.Oct 27 2017, 6:37 PM

Restricted Application added a project: Multimedia. · View Herald TranscriptOct 27 2017, 6:37 PM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

There is an internal error when trying to render the thumbnail. @Gilles this is presumably thumbor related. Not sure how to debug issues like that; in the past there used to be an X-Request-Id that could be looked up in the logs, but it's not being set now, and while there is an X-MW-Thumbnail-Renderer field, trying to filter for that in logstash yields no results.

Oh I see. Thanks so much for information regarding this annoying error.

Aklapper edited projects, added Thumbor; removed MediaViewer.Oct 28 2017, 3:49 PM

It's true that there's no convenient way to tie an error returned by thumbor to its corresponding entry in logstash. I'll file a task about that.

• Gilles created subtask T179954: Thumbor errors should contain a trackable request id.Nov 7 2017, 4:39 PM

• Ramsey-WMF moved this task from Untriaged to Tracking on the Multimedia board.Nov 16 2017, 5:49 PM

• Gilles closed subtask T179954: Thumbor errors should contain a trackable request id as Resolved.Feb 14 2018, 4:10 PM

https://logstash.wikimedia.org/app/kibana#/doc/logstash-*/logstash-2018.03.14/logback?id=AWIje8ltGqt6ZYdwCWHh&_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:now-15m,mode:quick,to:now))

I need to look into why that particular entry doesn't have the url field, which would have made it easy to find.

It looks like ImageMagick can't convert that particular file due to data corruption.

Locally even a fairly recent version of ImageMagick (Version: ImageMagick 7.0.5-10 Q16 x86_64 2017-06-04) fails to convert this file, with the same error message. OS X Preview.app does render it, albeit the bottom of the image is black. Combined with the ImageMagick error message, this suggests that the file is somewhat cut off before its end.

Hard to say if that corruption happened during the upload, you should check if the original you have exhibits the same symptoms. But there's not much we can do with files ImageMagick refuses to process.

Actually, I've just noticed that despite the error it spews and the non-zero exit code, ImageMagick (at least locally) does render a valid output PNG. We currently throw it away due to the exit code, but we could keep it.

• Gilles renamed this task from Not showing my own image of nokia 3310 to ImageMagick may return errors when it actually converted what it could.Mar 14 2018, 8:20 AM

• Gilles claimed this task.

• Gilles triaged this task as Low priority.

• Gilles added a project: Performance-Team.

• Gilles added a revision: Restricted Differential Revision.Mar 14 2018, 8:59 AM

• Gilles added a project: Patch-For-Review.Mar 14 2018, 8:59 AM

• Gilles moved this task from Backlog to Doing on the Thumbor board.Mar 14 2018, 10:23 AM

• Gilles moved this task from Inbox, needs triage to Doing (old) on the Performance-Team board.Mar 16 2018, 9:47 AM

• Gilles added a commit: rTHMBREXTf321adbfa13d: Return any thumbnail ImageMagick managed to render.Mar 26 2018, 1:57 PM

Turns out that the production version of IM we use (6.8.9-9 Q16 x86_64 2017-11-16) cannot render that particular PNG. More recent versions of IM (7.0.5-10 Q16 x86_64 2017-06-04) can. The change made is worthwhile, but this file won't be fixed until an ImageMagick upgrade happens.

• Gilles removed a project: Patch-For-Review.Mar 26 2018, 2:13 PM

• Gilles added a commit: rTHMBREXTf4d1726c3859: Skip broken PNG test for now.Mar 26 2018, 2:13 PM

Change 419172 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 1.16

https://gerrit.wikimedia.org/r/419172

gerritbot added a project: Patch-For-Review.Mar 26 2018, 4:08 PM

• Gilles added a commit: rTHMBREXTcd54f47a777c: Fix location of skip exception.Apr 3 2018, 9:12 AM

Change 419172 merged by Filippo Giunchedi:
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 1.16

https://gerrit.wikimedia.org/r/419172

Mentioned in SAL (#wikimedia-operations) [2018-04-03T09:51:53Z] <godog> deploy thumbor 1.16 in codfw and eqiad - T186528 T179200 T189647 T191028

Stashbot mentioned this in T189647: Thumbor missing log extras when hitting ImageMagicException.Apr 3 2018, 9:51 AM

Stashbot mentioned this in T191028: Thumbor incorrectly normalizes .jpe and .jpeg into .jpg for Swift thumbnail storage.

• Gilles closed subtask T189647: Thumbor missing log extras when hitting ImageMagicException as Resolved.Apr 3 2018, 10:15 AM

This should fix some images, but not the one in the task description because that's blocked on Stretch. I'll update the task info accordingly.

• Gilles renamed this task from ImageMagick may return errors when it actually converted what it could to Some half-broken PNGs cannot be rendered with the Jessie IM version.Apr 3 2018, 11:24 AM

• Gilles removed • Gilles as the assignee of this task.

• Gilles removed a project: Patch-For-Review.

• Gilles updated the task description. (Show Details)

• Gilles added a parent task: T170817: Upgrade Thumbor servers to Stretch.

• Gilles moved this task from Doing (old) to Blocked (old) on the Performance-Team board.May 21 2018, 8:27 PM

• Imarlier edited projects, added Performance-Team (Radar); removed Performance-Team.Jun 21 2018, 10:38 AM

Krinkle moved this task from Limbo to Watching on the Performance-Team (Radar) board.Jun 21 2018, 10:43 AM

Maowtm moved this task from Doing to Pending review on the Thumbor board.Dec 17 2018, 11:29 AM

Maowtm moved this task from Pending review to Doing on the Thumbor board.