Page MenuHomePhabricator

Chunked upload file fails to reconstitute...
Open, LowPublic

Description

I'v so far attempted (4 times) to get - https://ia802705.us.archive.org/8/items/mrsbeetonshouse00beetuoft/mrsbeetonshouse00beetuoft.djvu

to upload here -
https://commons.wikimedia.org/wiki/File:Mrs_Beeton%27s_Book_of_Household_Management.djvu
using the "chunked approach" as the file is bigger than the 100MB typicla for single uploads.

It uploads, but the file fails to reconstitute

Filed as a bug because I can't see a logical reason why it fails and have a concern junk is being left on servers..

Event Timeline

ShakespeareFan00 raised the priority of this task from to Needs Triage.
ShakespeareFan00 updated the task description. (Show Details)
ShakespeareFan00 added a subscriber: ShakespeareFan00.
Restricted Application added a project: Multimedia. · View Herald TranscriptJul 14 2015, 2:01 PM
Restricted Application added subscribers: Steinsplitter, Aklapper. · View Herald Transcript
ShakespeareFan00 set Security to None.
Restricted Application added a subscriber: Matanya. · View Herald TranscriptJul 14 2015, 2:03 PM
Yann added a subscriber: Yann.Jul 14 2015, 2:17 PM

I tried too, and I had the same issue.

Tgr added a subscriber: Tgr.Jul 15 2015, 10:35 PM

Reproduced on test2wiki, there are a bunch of errors like this:

2015-07-15 22:00:51 mw1004 test2wiki error-json INFO: {"id":"43b73c70","type":"ErrorException","file":"/srv/mediawiki/php-1.26wmf14/includes/GlobalFunctions.php","line":2858,"message":"PHP Notice: ","suppressed":true,"url":null,"backtrace":[{"function":"handleError","class":"MWExceptionHandler","type":"::","args":["integer","string","string","integer"]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/GlobalFunctions.php","line":2858,"function":"trigger_error","args":["string"]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/media/DjVuImage.php","line":295,"function":"wfShellExec","args":["string","string","array","array"]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/media/DjVu.php","line":361,"function":"retrieveMetaData","class":"DjVuImage","type":"->","args":[]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/filerepo/file/UnregisteredLocalFile.php","line":188,"function":"getMetadata","class":"DjVuHandler","type":"->","args":["UploadStashFile","string"]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/media/DjVu.php","line":262,"function":"getMetadata","class":"UnregisteredLocalFile","type":"->","args":[]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/media/DjVu.php","line":300,"function":"getUnserializedMetadata","class":"DjVuHandler","type":"->","args":["UploadStashFile"]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/media/DjVu.php","line":403,"function":"getMetaTree","class":"DjVuHandler","type":"->","args":["UploadStashFile"]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/filerepo/file/UnregisteredLocalFile.php","line":118,"function":"getPageDimensions","class":"DjVuHandler","type":"->","args":["UploadStashFile","integer"]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/filerepo/file/UnregisteredLocalFile.php","line":129,"function":"cachePageDimensions","class":"UnregisteredLocalFile","type":"->","args":["integer"]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/api/ApiQueryImageInfo.php","line":422,"function":"getWidth","class":"UnregisteredLocalFile","type":"->","args":[]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/upload/UploadBase.php","line":1888,"function":"getInfo","class":"ApiQueryImageInfo","type":"::","args":["UploadStashFile","array","ApiResult"]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/jobqueue/jobs/AssembleUploadChunksJob.php","line":80,"function":"getImageInfo","class":"UploadBase","type":"->","args":["ApiResult"]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/jobqueue/JobRunner.php","line":183,"function":"run","class":"AssembleUploadChunksJob","type":"->","args":[]},{"file":"/srv/mediawiki/rpc/RunJobs.php","line":42,"function":"run","class":"JobRunner","type":"->","args":["array"]}]} {"private":true}

in more readable form:

/srv/mediawiki/php-1.26wmf14/includes/GlobalFunctions.php#2858: trigger_error(string)
/srv/mediawiki/php-1.26wmf14/includes/media/DjVuImage.php#295: wfShellExec(string, string, array, array)
/srv/mediawiki/php-1.26wmf14/includes/media/DjVu.php#361: DjVuImage->retrieveMetaData()
/srv/mediawiki/php-1.26wmf14/includes/filerepo/file/UnregisteredLocalFile.php#188: DjVuHandler->getMetadata(UploadStashFile, string)
/srv/mediawiki/php-1.26wmf14/includes/media/DjVu.php#262: UnregisteredLocalFile->getMetadata()
/srv/mediawiki/php-1.26wmf14/includes/media/DjVu.php#300: DjVuHandler->getUnserializedMetadata(UploadStashFile)
/srv/mediawiki/php-1.26wmf14/includes/media/DjVu.php#403: DjVuHandler->getMetaTree(UploadStashFile)
/srv/mediawiki/php-1.26wmf14/includes/filerepo/file/UnregisteredLocalFile.php#118: DjVuHandler->getPageDimensions(UploadStashFile, integer)
/srv/mediawiki/php-1.26wmf14/includes/filerepo/file/UnregisteredLocalFile.php#129: UnregisteredLocalFile->cachePageDimensions(integer)
/srv/mediawiki/php-1.26wmf14/includes/api/ApiQueryImageInfo.php#422: UnregisteredLocalFile->getWidth()
/srv/mediawiki/php-1.26wmf14/includes/upload/UploadBase.php#1888: ApiQueryImageInfo::getInfo(UploadStashFile, array, ApiResult)
/srv/mediawiki/php-1.26wmf14/includes/jobqueue/jobs/AssembleUploadChunksJob.php#80: UploadBase->getImageInfo(ApiResult)
/srv/mediawiki/php-1.26wmf14/includes/jobqueue/JobRunner.php#183: AssembleUploadChunksJob->run()
/srv/mediawiki/rpc/RunJobs.php#42: JobRunner->run(array)
Tgr added a comment.Jul 15 2015, 10:36 PM

...which is apparently just some crappy hack in wfShellExec() which clogs up the error logs. Duh.

Tgr added a comment.Jul 15 2015, 10:52 PM

A bit hardto read the error logs since enormous chunks of the DJVU file are included, but these show up often:

2015-07-15 22:00:11 mw1203 test2wiki wfDebug DEBUG: ZipDirectoryReader: Fatal error: zip file lacks EOCDR signature. It probably isn't a zip file. 
ErrorException from line 144 of /srv/mediawiki/php-1.26wmf14/includes/upload/UploadStash.php: PHP Notice: Unable to unserialize <huge thing with the full text of the book in it>
zhuyifei1999 moved this task from Incoming to Uploading on the Commons board.Jul 16 2015, 6:06 AM
zhuyifei1999 added a subscriber: zhuyifei1999.
Tgr added a comment.Jul 17 2015, 9:34 AM
ErrorException from line 144 of /srv/mediawiki/php-1.26wmf14/includes/upload/UploadStash.php: PHP Notice: Unable to unserialize <huge thing with the full text of the book in it>

If it's a metadata size problem, something like T32906 could help.

Jdforrester-WMF triaged this task as Low priority.Sep 4 2015, 6:55 PM
Jdforrester-WMF moved this task from Untriaged to Backlog on the Multimedia board.Sep 4 2015, 6:59 PM
Restricted Application added a project: Internet-Archive. · View Herald TranscriptNov 15 2015, 8:12 AM