Chunked upload file fails to reconstitute...
Open, LowPublic
Actions

Assigned To

None

Authored By

	ShakespeareFan00
	Jul 14 2015, 2:01 PM

Description

I'v so far attempted (4 times) to get - https://ia802705.us.archive.org/8/items/mrsbeetonshouse00beetuoft/mrsbeetonshouse00beetuoft.djvu

to upload here -
https://commons.wikimedia.org/wiki/File:Mrs_Beeton%27s_Book_of_Household_Management.djvu
using the "chunked approach" as the file is bigger than the 100MB typicla for single uploads.

It uploads, but the file fails to reconstitute

Filed as a bug because I can't see a logical reason why it fails and have a concern junk is being left on servers..

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		None	T61914 Chunked upload (tracking)
		Open		None	T105791 Chunked upload file fails to reconstitute...

Event Timeline

ShakespeareFan00 created this task.Jul 14 2015, 2:01 PM

ShakespeareFan00 raised the priority of this task from to Needs Triage.

ShakespeareFan00 updated the task description. (Show Details)

ShakespeareFan00 added projects: Commons, MediaWiki-Uploading.

ShakespeareFan00 subscribed.

Restricted Application added a project: Multimedia. · View Herald TranscriptJul 14 2015, 2:01 PM

Restricted Application added subscribers: Steinsplitter, Aklapper. · View Herald Transcript

ShakespeareFan00 updated the task description. (Show Details)Jul 14 2015, 2:03 PM

ShakespeareFan00 set Security to None.

Restricted Application added a subscriber: Matanya. · View Herald TranscriptJul 14 2015, 2:03 PM

I tried too, and I had the same issue.

JeanFred added a subtask: T61914: Chunked upload (tracking).Jul 15 2015, 5:01 PM

JeanFred removed a subtask: T61914: Chunked upload (tracking).

JeanFred added a parent task: T61914: Chunked upload (tracking).

Reproduced on test2wiki, there are a bunch of errors like this:

2015-07-15 22:00:51 mw1004 test2wiki error-json INFO: {"id":"43b73c70","type":"ErrorException","file":"/srv/mediawiki/php-1.26wmf14/includes/GlobalFunctions.php","line":2858,"message":"PHP Notice: ","suppressed":true,"url":null,"backtrace":[{"function":"handleError","class":"MWExceptionHandler","type":"::","args":["integer","string","string","integer"]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/GlobalFunctions.php","line":2858,"function":"trigger_error","args":["string"]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/media/DjVuImage.php","line":295,"function":"wfShellExec","args":["string","string","array","array"]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/media/DjVu.php","line":361,"function":"retrieveMetaData","class":"DjVuImage","type":"->","args":[]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/filerepo/file/UnregisteredLocalFile.php","line":188,"function":"getMetadata","class":"DjVuHandler","type":"->","args":["UploadStashFile","string"]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/media/DjVu.php","line":262,"function":"getMetadata","class":"UnregisteredLocalFile","type":"->","args":[]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/media/DjVu.php","line":300,"function":"getUnserializedMetadata","class":"DjVuHandler","type":"->","args":["UploadStashFile"]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/media/DjVu.php","line":403,"function":"getMetaTree","class":"DjVuHandler","type":"->","args":["UploadStashFile"]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/filerepo/file/UnregisteredLocalFile.php","line":118,"function":"getPageDimensions","class":"DjVuHandler","type":"->","args":["UploadStashFile","integer"]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/filerepo/file/UnregisteredLocalFile.php","line":129,"function":"cachePageDimensions","class":"UnregisteredLocalFile","type":"->","args":["integer"]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/api/ApiQueryImageInfo.php","line":422,"function":"getWidth","class":"UnregisteredLocalFile","type":"->","args":[]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/upload/UploadBase.php","line":1888,"function":"getInfo","class":"ApiQueryImageInfo","type":"::","args":["UploadStashFile","array","ApiResult"]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/jobqueue/jobs/AssembleUploadChunksJob.php","line":80,"function":"getImageInfo","class":"UploadBase","type":"->","args":["ApiResult"]},{"file":"/srv/mediawiki/php-1.26wmf14/includes/jobqueue/JobRunner.php","line":183,"function":"run","class":"AssembleUploadChunksJob","type":"->","args":[]},{"file":"/srv/mediawiki/rpc/RunJobs.php","line":42,"function":"run","class":"JobRunner","type":"->","args":["array"]}]} {"private":true}

in more readable form:

/srv/mediawiki/php-1.26wmf14/includes/GlobalFunctions.php#2858: trigger_error(string)
/srv/mediawiki/php-1.26wmf14/includes/media/DjVuImage.php#295: wfShellExec(string, string, array, array)
/srv/mediawiki/php-1.26wmf14/includes/media/DjVu.php#361: DjVuImage->retrieveMetaData()
/srv/mediawiki/php-1.26wmf14/includes/filerepo/file/UnregisteredLocalFile.php#188: DjVuHandler->getMetadata(UploadStashFile, string)
/srv/mediawiki/php-1.26wmf14/includes/media/DjVu.php#262: UnregisteredLocalFile->getMetadata()
/srv/mediawiki/php-1.26wmf14/includes/media/DjVu.php#300: DjVuHandler->getUnserializedMetadata(UploadStashFile)
/srv/mediawiki/php-1.26wmf14/includes/media/DjVu.php#403: DjVuHandler->getMetaTree(UploadStashFile)
/srv/mediawiki/php-1.26wmf14/includes/filerepo/file/UnregisteredLocalFile.php#118: DjVuHandler->getPageDimensions(UploadStashFile, integer)
/srv/mediawiki/php-1.26wmf14/includes/filerepo/file/UnregisteredLocalFile.php#129: UnregisteredLocalFile->cachePageDimensions(integer)
/srv/mediawiki/php-1.26wmf14/includes/api/ApiQueryImageInfo.php#422: UnregisteredLocalFile->getWidth()
/srv/mediawiki/php-1.26wmf14/includes/upload/UploadBase.php#1888: ApiQueryImageInfo::getInfo(UploadStashFile, array, ApiResult)
/srv/mediawiki/php-1.26wmf14/includes/jobqueue/jobs/AssembleUploadChunksJob.php#80: UploadBase->getImageInfo(ApiResult)
/srv/mediawiki/php-1.26wmf14/includes/jobqueue/JobRunner.php#183: AssembleUploadChunksJob->run()
/srv/mediawiki/rpc/RunJobs.php#42: JobRunner->run(array)

...which is apparently just some crappy hack in wfShellExec() which clogs up the error logs. Duh.

A bit hardto read the error logs since enormous chunks of the DJVU file are included, but these show up often:

2015-07-15 22:00:11 mw1203 test2wiki wfDebug DEBUG: ZipDirectoryReader: Fatal error: zip file lacks EOCDR signature. It probably isn't a zip file. 
ErrorException from line 144 of /srv/mediawiki/php-1.26wmf14/includes/upload/UploadStash.php: PHP Notice: Unable to unserialize <huge thing with the full text of the book in it>

zhuyifei1999 moved this task from Incoming to Uploading on the Commons board.Jul 16 2015, 6:06 AM

zhuyifei1999 subscribed.

ErrorException from line 144 of /srv/mediawiki/php-1.26wmf14/includes/upload/UploadStash.php: PHP Notice: Unable to unserialize <huge thing with the full text of the book in it>

If it's a metadata size problem, something like T32906 could help.

Jdforrester-WMF triaged this task as Low priority.Sep 4 2015, 6:55 PM

Jdforrester-WMF moved this task from Untriaged to Backlog on the Multimedia board.Sep 4 2015, 6:59 PM

zhuyifei1999 moved this task from Uploading to Uploading (chunked uploading and API) on the Commons board.Nov 15 2015, 8:12 AM

Restricted Application added a project: Internet-Archive. · View Herald TranscriptNov 15 2015, 8:12 AM

zhuyifei1999 moved this task from Uploading (chunked uploading and API) to Uploading (UploadWizard, cross-wiki upload, chunked uploading and API) on the Commons board.Nov 15 2015, 8:17 AM

Harej moved this task from Backlog to Archive.org Availability on the Internet-Archive board.Nov 2 2021, 7:36 PM