Steps to replicate the issue (include links if applicable):
- Prepare a big (say, 2~3GiB) PDF file. e.g. https://hoiscript.toolforge.org/ynutcm/YNUTCM-294300021-5%E5%BE%A1%E7%BA%82%E9%86%AB%E5%AE%97%E9%87%91%E9%91%92%EF%BC%88%E5%8D%B7%E5%9B%9B%EF%BC%8972494.pdf .
- Upload it to Commons by mwclient with default configurations (Site::upload).
- Check log
What happens?:
The upload seldom succeeds. The server always returns 503 or 504.
What should have happened instead?:
The upload succeeds.
Software version (skip for WMF-hosted wikis like Wikipedia):
Wikimedia Commons
mwclient / 0.10.1
Other information (browser name/version, screenshots, etc.):
The problem exists for years. Dating back to some weeks in 2021, I cannot even upload files of > 200MiB files.
Until now, the failure rate of uploading files is proportional to the size of the files being uploaded. That is, uploading a ~ 1GiB file fails sometimes, ~ a 2GiB file fails frequently and ~ a 3GiB file fails almost all the time.
The failure appears to have nothing to do with the network. Uploading from my local box, a third-party Cloud server, Toolforge, or WMCS Cloud VPS yields the same result in general.
Sometimes, I manage to upload failed files manually via Special:UploadWizard. It is slightly more stable, but still suffers from the issue.
Logs (some repeated lines omitted):
INFO:__main__:Uploading File:YNUTCM-294300021-5御纂醫宗金鑒（卷四）72494.pdf (2325273937 B)
INFO:__main__:Retrying <function main.<locals>.do1 at 0x7fdb56345580>
WARNING:mwclient.client:Received 503 response: <!DOCTYPE html>
<html lang="en">
<meta charset="utf-8">
<title>Wikimedia Error</title>
...<h1>Error</h1>
<div class="content-text">
<p>Our servers are currently under maintenance or experiencing a technical problem.
...<div class="footer"><p>If you report this error to the Wikimedia System Administrators, please include the details below.</p><p class="text-muted"
><code>Request from 172.16.4.85 via cp1079 cp1079, Varnish XID 720477065<br>Error: 503, Backend fetch failed at Sat, 01 Jul 2023 04:59:00 GMT</cod
e></p>
</div>
</html>
. Retrying in a moment.
INFO:__main__:Retrying <function main.<locals>.do1 at 0x7fdb56345580>
...
File "/root/.cache/pypoetry/virtualenvs/uploader-URSXssvu-py3.11/lib/python3.11/site-packages/mwclient/client.py", line 679, in upload
return self.chunk_upload(file, filename, ignore, comment, text)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/uploader-URSXssvu-py3.11/lib/python3.11/site-packages/mwclient/client.py", line 760, in chunk_upload
if self.handle_api_result(info, kwargs=params, sleeper=sleeper):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/uploader-URSXssvu-py3.11/lib/python3.11/site-packages/mwclient/client.py", line 331, in handle_api_resul
t
raise errors.APIError(info['error']['code'],
mwclient.errors.APIError: ('chunk-too-small', 'Minimum chunk size is 1,024 bytes for non-final chunks.', 'See https://commons.wikimedia.org/w/api.
php for API usage. Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/postorius/lists/mediawiki-api-announce.
lists.wikimedia.org/> for notice of API deprecations and breaking changes.')
...