Page MenuHomePhabricator

job-queue too long
Closed, DuplicatePublic

Description

Hi,
using the IA-upload tool to create wikicommons djvu, I found a too long job-queue: i tried to upload two files on august 8th and now the process isn't finished yet.
The final line of the log is: "[2017-08-08 09:30:10] LOG.CRITICAL: File not found: [] []"
The IA names of the file are: '025CommedieDelCinquecento1Si078' and '024GiobertiDelRinnovamento3Si132'

Thanks.
Valeria

Event Timeline

Sorry for the delay in responding to this! (I only just saw it, I'm not sure why it didn't notify me.)

The file not found issue is due to the DjVu-merge process not finishing, most likely to due it running out of memory:

[2017-08-09 09:04:30] LOG.INFO: Merging all DjVu files to /mnt/nfs/labstore-secondary-tools-project/ia-upload/ia-upload/jobqueue/025CommedieDelCinquecento1Si078/025CommedieDelCinquecento1Si078.djvu [] []
[2017-08-09 09:04:38] LOG.CRITICAL: File not found:  [] []

(The file that it can't find is the one it was supposed to have created in the previous step.)

We do not yet have a conclusive solution to this problem. It's being tracked in: T161396: Memory issues for IA-upload when converting large files to djvu