Page MenuHomePhabricator
Create Task
Maniphest T129071

IA-upload bot failed
Closed, ResolvedPublic
Actions

Description

The IA-upload bot failed with https://archive.org/details/s3journalasiatiq07sociuoft

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 68986943 bytes) in /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Message/Request.php on line 423 Call Stack: 0.0149 683712 1. {main}() /data/project/ia-upload/ia-upload/web/index.php:0 0.7144 1734656 2. Silex\Application->run() /data/project/ia-upload/ia-upload/web/index.php:26 1.0315 2733832 3. Silex\Application->handle() /data/project/ia-upload/ia-upload/vendor/silex/silex/src/Silex/Application.php:515 1.8051 3525680 4. Symfony\Component\HttpKernel\HttpKernel->handle() /data/project/ia-upload/ia-upload/vendor/silex/silex/src/Silex/Application.php:538 1.8051 3525680 5. Symfony\Component\HttpKernel\HttpKernel->handleRaw() /data/project/ia-upload/ia-upload/vendor/symfony/http-kernel/Symfony/Component/HttpKernel/HttpKernel.php:66 3.1043 4542200 6. call_user_func_array() /data/project/ia-upload/ia-upload/vendor/symfony/http-kernel/Symfony/Component/HttpKernel/HttpKernel.php:145 3.1043 4542816 7. IaUpload\EntryPoint->commonsSave() /data/project/ia-upload/ia-upload/vendor/symfony/http-kernel/Symfony/Component/HttpKernel/HttpKernel.php:0 3.8592 5959152 8. IaUpload\CommonsController->save() /data/project/ia-upload/ia-upload/src/IaUpload/EntryPoint.php:53 5.1153 8457896 9. IaUpload\IaClient->downloadFile() /data/project/ia-upload/ia-upload/src/IaUpload/CommonsController.php:156 5.1322 8480280 10. Guzzle\Http\Message\Request->send() /data/project/ia-upload/ia-upload/src/IaUpload/IaClient.php:45 5.1322 8480280 11. Guzzle\Http\Client->send() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Message/Request.php:198 5.1323 8482080 12. Guzzle\Http\Curl\CurlMultiProxy->send() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Client.php:284 5.1344 8497976 13. Guzzle\Http\Curl\CurlMulti->send() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Curl/CurlMultiProxy.php:94 5.1344 8497976 14. Guzzle\Http\Curl\CurlMulti->perform() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Curl/CurlMulti.php:109 5.1345 8499936 15. Guzzle\Http\Curl\CurlMulti->executeHandles() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Curl/CurlMulti.php:215 5.5102 8523760 16. Guzzle\Http\Curl\CurlMulti->processMessages() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Curl/CurlMulti.php:231 5.5102 8524776 17. Guzzle\Http\Curl\CurlMulti->processResponse() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Curl/CurlMulti.php:248 5.5103 8529240 18. Guzzle\Http\Message\Request->setState() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Curl/CurlMulti.php:290 5.5103 8529240 19. Guzzle\Http\Message\Request->processResponse() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Message/Request.php:378 5.5104 8530224 20. Guzzle\Http\Message\Request->dispatch() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Message/Request.php:577 5.5104 8531984 21. Symfony\Component\EventDispatcher\EventDispatcher->dispatch() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Message/Request.php:536 5.5104 8532984 22. Symfony\Component\EventDispatcher\EventDispatcher->doDispatch() /data/project/ia-upload/ia-upload/vendor/symfony/event-dispatcher/Symfony/Component/EventDispatcher/EventDispatcher.php:53 5.5104 8533496 23. call_user_func() /data/project/ia-upload/ia-upload/vendor/symfony/event-dispatcher/Symfony/Component/EventDispatcher/EventDispatcher.php:164 5.5104 8533568 24. Guzzle\Http\RedirectPlugin->onRequestSent() /data/project/ia-upload/ia-upload/vendor/symfony/event-dispatcher/Symfony/Component/EventDispatcher/EventDispatcher.php:0 5.5105 8533728 25. Guzzle\Http\RedirectPlugin->sendRedirectRequest() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/RedirectPlugin.php:82 5.5109 8550496 26. Guzzle\Http\Message\Request->send() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/RedirectPlugin.php:221 5.5109 8550496 27. Guzzle\Http\Client->send() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Message/Request.php:198 5.5109 8550616 28. Guzzle\Http\Curl\CurlMultiProxy->send() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Client.php:284 5.5114 8564096 29. Guzzle\Http\Curl\CurlMulti->send() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Curl/CurlMultiProxy.php:94 5.5114 8564096 30. Guzzle\Http\Curl\CurlMulti->perform() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Curl/CurlMulti.php:109 5.5115 8566056 31. Guzzle\Http\Curl\CurlMulti->executeHandles() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Curl/CurlMulti.php:215 56.8474 8588592 32. Guzzle\Http\Curl\CurlMulti->processMessages() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Curl/CurlMulti.php:231 56.8475 8589608 33. Guzzle\Http\Curl\CurlMulti->processResponse() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Curl/CurlMulti.php:248 56.8478 8593976 34. Guzzle\Http\Message\Request->setState() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Curl/CurlMulti.php:290 56.8478 8593976 35. Guzzle\Http\Message\Request->processResponse() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Message/Request.php:378 56.8478 8594960 36. Guzzle\Http\Message\Request->dispatch() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Message/Request.php:577 56.8479 8596720 37. Symfony\Component\EventDispatcher\EventDispatcher->dispatch() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/Message/Request.php:536 56.8480 8596720 38. Symfony\Component\EventDispatcher\EventDispatcher->doDispatch() /data/project/ia-upload/ia-upload/vendor/symfony/event-dispatcher/Symfony/Component/EventDispatcher/EventDispatcher.php:53 56.8480 8597232 39. call_user_func() /data/project/ia-upload/ia-upload/vendor/symfony/event-dispatcher/Symfony/Component/EventDispatcher/EventDispatcher.php:164 56.8480 8597304 40. Guzzle\Http\RedirectPlugin->onRequestSent() /data/project/ia-upload/ia-upload/vendor/symfony/event-dispatcher/Symfony/Component/EventDispatcher/EventDispatcher.php:0 56.8482 8597760 41. Guzzle\Http\Message\Request->setResponse() /data/project/ia-upload/ia-upload/vendor/guzzle/http/Guzzle/Http/RedirectPlugin.php:76

Event Timeline

Yann created this task.Mar 7 2016, 12:22 PM
Restricted Application added a project: Internet-Archive. · View Herald TranscriptMar 7 2016, 12:22 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Yann removed Tpt as the assignee of this task.Mar 7 2016, 12:23 PM
Yann edited subscribers, added: Tpt; removed: Aklapper.
Yann added a comment.Mar 7 2016, 6:22 PM

Now with a different message for https://archive.org/details/journalasiatique127sociuoft

The upload to WikimediaCommons failed: Client error response [status code] 413 [reason phrase] Request Entity Too Large [url] https://commons.wikimedia.org/w/api.php?action=upload&filename=Journal_asiatique%2C_s%C3%A9rie_7%2C_tome_1-2.djvu&format=json (file is 52.9 MB)

Yann added a comment.Mar 7 2016, 6:54 PM

Again with https://archive.org/details/s3journalasiatiq09sociuoft

The upload to WikimediaCommons failed: Client error response [status code] 413 [reason phrase] Request Entity Too Large [url] https://commons.wikimedia.org/w/api.php?action=upload&filename=Journal_asiatique%2C_s%C3%A9rie_3%2C_tome_9-10.djvu&format=json

Tpt added a comment.Mar 8 2016, 10:07 AM

The first error is probably because the downloaded DjVu files that makes PHP allocate more memory than allowed.

The 414 error is when the DjVu file weight is greater than 100MB (the current limit for direct uploads using the API). We should maybe output a more user friendly message.

Yann added a comment.Mar 9 2016, 8:48 PM

I don't see any file above 100 MB. The first case is 65.8 MB, the second is 52.9 MB, the third is 56.4 MB.

Samwilson subscribed.Feb 10 2017, 4:21 AM

Has this file been successfully uploaded now?

The s3journalasiatiq07sociuoft item was being referenced from https://commons.wikimedia.org/wiki/File:Journal_asiatique,_s%C3%A9rie_4,_tome_7-8.djvu as well as https://commons.wikimedia.org/wiki/File:Journal_asiatique,_s%C3%A9rie_3,_tome_7-8.djvu — the former was not correct, so I removed the IA identifier from the book template (I didn't know what identifier to put in its place). The latter appears to be the correctly uploaded version of the required DjVu.

I think this can be closed as resolved. Do you agree?

Samwilson added a project: All-and-every-Wikisource.Feb 10 2017, 4:31 AM
Samwilson moved this task from Backlog to IA Upload on the All-and-every-Wikisource board.Feb 10 2017, 4:33 AM
Samwilson triaged this task as Medium priority.Feb 10 2017, 4:42 AM
Yann added a comment.Feb 10 2017, 9:48 PM

This file is now uploaded, but is the bug source now fixed?

Samwilson added a comment.Feb 19 2017, 11:06 PM

@Yann, yes I think the new system of generating DjVus will not fail in this way.

I'll close this as resolved for now, and if anyone has similar issues they can open separate tasks.

Samwilson closed this task as Resolved.Feb 19 2017, 11:06 PM
Samwilson claimed this task.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL