Page MenuHomePhabricator

IA Upload unable to convert a JPEG2000 to JPEG
Open, MediumPublic

Description

IA uploader appears to be hanging "in progress"
here is a link of the log
https://tools.wmflabs.org/ia-upload/log/Httpsdl.wdl.org11960service11960.pdf

Event Timeline

Slowking4 created this task.Apr 8 2017, 2:00 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 8 2017, 2:00 PM
Zppix added a subscriber: Zppix.Apr 8 2017, 2:07 PM

Log for convenience:

[2017-04-08 03:45:11] LOG.INFO: Creating DjVu for Httpsdl.wdl.org11960service11960.pdf from Jp2 [] []
[2017-04-08 03:45:11] LOG.INFO: Saving IA metadata to /mnt/nfs/labstore-secondary-tools-project/ia-upload/ia-upload/jobqueue/Httpsdl.wdl.org11960service11960.pdf/metadata.json [] []
[2017-04-08 03:45:11] LOG.INFO: Downloading Httpsdl.wdl.org11960service11960.pdf/11960_djvu.xml [] []
[2017-04-08 03:45:13] LOG.INFO: Downloading Httpsdl.wdl.org11960service11960.pdf/11960_jp2.zip [] []
[2017-04-08 03:45:26] LOG.INFO: Unzipping /mnt/nfs/labstore-secondary-tools-project/ia-upload/ia-upload/jobqueue/Httpsdl.wdl.org11960service11960.pdf/11960_jp2.zip [] []
[2017-04-08 03:45:31] LOG.DEBUG: Zip file extracted to /mnt/nfs/labstore-secondary-tools-project/ia-upload/ia-upload/jobqueue/Httpsdl.wdl.org11960service11960.pdf/11960_jp2 [] []
[2017-04-08 03:45:31] LOG.INFO: Processing JP2 files [] []
[2017-04-08 03:45:31] LOG.INFO: Converting 50 individual JP2s to DjVus [] []
[2017-04-08 03:45:31] LOG.DEBUG: Converting /mnt/nfs/labstore-secondary-tools-project/ia-upload/ia-upload/jobqueue/Httpsdl.wdl.org11960service11960.pdf/11960_jp2/11960_0000.jp2... [] []
[2017-04-08 03:45:31] LOG.DEBUG: ...to /mnt/nfs/labstore-secondary-tools-project/ia-upload/ia-upload/jobqueue/Httpsdl.wdl.org11960service11960.pdf/build/Httpsdl.wdl.org11960service11960.pdf_p0.jpg [] []
[2017-04-08 03:45:36] LOG.CRITICAL: Command "convert -resize 1500x1500 "/mnt/nfs/labstore-secondary-tools-project/ia-upload/ia-upload/jobqueue/Httpsdl.wdl.org11960service11960.pdf/11960_jp2/11960_0000.jp2" "/mnt/nfs/labstore-secondary-tools-project/ia-upload/ia-upload/jobqueue/Httpsdl.wdl.org11960service11960.pdf/build/Httpsdl.wdl.org11960service11960.pdf_p0.jpg" 2>&1" exited with code 1: jpc_dec_decodecblks failed error: cannot decode code stream convert.im6: unable to decode image file `/mnt/nfs/labstore-secondary-tools-project/ia-upload/ia-upload/jobqueue/Httpsdl.wdl.org11960service11960.pdf/11960_jp2/11960_0000.jp2' @ error/jp2.c/ReadJP2Image/402. convert.im6: no images defined `/mnt/nfs/labstore-secondary-tools-project/ia-upload/ia-upload/jobqueue/Httpsdl.wdl.org11960service11960.pdf/build/Httpsdl.wdl.org11960service11960.pdf_p0.jpg' @ error/convert.c/ConvertImageCommand/3044. [] []
Zppix removed a subscriber: Zppix.Apr 8 2017, 2:07 PM
Restricted Application assigned this task to Cyberpower678. · View Herald TranscriptApr 8 2017, 2:53 PM
Cyberpower678 removed Cyberpower678 as the assignee of this task.Apr 8 2017, 3:58 PM
Cyberpower678 removed a project: InternetArchiveBot.
Cyberpower678 added a subscriber: Cyberpower678.

What is this. This doesn't apply to me.

DatGuy added subscribers: Tpt, DatGuy.Apr 8 2017, 4:07 PM

This is @Tpt's tool

Tpt added a subscriber: Samwilson.

Thank you for the report. I subscribes Sam Wilson who has worked on this part of the tool.

Tpt moved this task from Backlog to IA Upload on the Wikisource board.Apr 8 2017, 4:13 PM

There appears to be something weird going on with the JP2 files in https://archive.org/download/Httpsdl.wdl.org11960service11960.pdf/11960_jp2.zip

Will try to replicate locally.

Samwilson renamed this task from IA uploader hanging to IA Upload unable to convert a JPEG2000 to JPEG.May 11 2017, 7:24 AM
Samwilson triaged this task as Medium priority.

This appears to no longer be failing for the same cause (since we switched to Graphicsmagick). It looks like it's still hanging though! :-(

Samwilson edited projects, added IA Upload; removed Wikisource.May 25 2017, 11:29 AM