Commons API and Special:Upload fail to upload file under 100 MB (with cryptic "413 Request Entity Too Large" error)
Closed, ResolvedPublic
Actions

Assigned To

None

Authored By

	Billinghurst
	Jan 10 2015, 12:08 PM

Description

Using the labs tool ia-upload, it failed to upload a file to Commons with the error being reported being

"The upload to WikimediaCommons failed: Client error response [status code] 413 [reason phrase] Request Entity Too Large [url] https://commons.wikimedia.org/w/api.php?action=upload&filename=Old_and_New_London%2C_vol._1.djvu&format=json"

The file was 57MB in size. I talked to the toollabs developer, and passing on his assessment of why he believed it failed and I was directed to use the standard upload tool. That becomes quite a circuitous route for a file of that size, and it would be great to identify whether the root cause is the API failing or not. thanks.

Related Objects

Mentioned In: T111908: Chunked upload fails (for files >5 MB)
Mentioned Here: T175680: Switch to chunked uploading for large DjVus
T115984: Cryptic "413 Request Entity Too Large" error when uploading new version of >100MB file on Commons
P1957 bug86436 http log

Event Timeline

Billinghurst created this task.Jan 10 2015, 12:08 PM

Billinghurst raised the priority of this task from to Needs Triage.

Billinghurst updated the task description. (Show Details)

Billinghurst subscribed.

Billinghurst updated the task description. (Show Details)Jan 10 2015, 12:11 PM

Billinghurst set Security to None.

Krenair added a project: WMF-General-or-Unknown.Jan 10 2015, 12:24 PM

Krenair subscribed.

Steinsplitter subscribed.Jan 10 2015, 2:27 PM

Aklapper renamed this task from Commons API fails to upload file within 100MB threshold to Commons API fails (413 error) to upload file within 100MB threshold.Jan 12 2015, 1:43 PM

Aklapper triaged this task as Medium priority.

Wondering if either anomie could take a look at this, or if that's already Multimedia team territory...?

Neither the API nor anything else in MediaWiki that I can find will raise an HTTP 413 error. Some software handling the request (e.g. varnish, apache, hhvm, or possibly even the library that tool is using to make the request) is throwing that error before MediaWiki ever sees it, because the request is exceeding the maximum size it is configured to allow. I don't know which piece of the infrastructure is responsible or what the configured maximum for this might be.

The developer of that tool may find chunked uploading helpful to avoid this error.

In T86436#971090, @Anomie wrote:

Neither the API nor anything else in MediaWiki that I can find will raise an HTTP 413 error. Some software handling the request (e.g. varnish, apache, hhvm, or possibly even the library that tool is using to make the request) is throwing that error before MediaWiki ever sees it, because the request is exceeding the maximum size it is configured to allow. I don't know which piece of the infrastructure is responsible or what the configured maximum for this might be.

The developer of that tool may find chunked uploading helpful to avoid this error.

@Tpt Wondering whether you can review @Anomie's feedback here. Thanks.

@Tpt: Could you reply to the last comments?

Setting task status to STALLED as @Tpt has not replied yet.

• Reguyla moved this task from Backlog to Commons related on the WMF-General-or-Unknown board.Jun 16 2015, 2:17 PM

• Reguyla added a project: Commons.Jun 16 2015, 2:58 PM

• Reguyla added a project: Multimedia.Jun 16 2015, 4:52 PM

Restricted Application added a subscriber: Matanya. · View Herald TranscriptJun 16 2015, 4:52 PM

Steinsplitter moved this task from Incoming to Backlog on the Commons board.Jun 17 2015, 11:47 AM

saper changed the task status from Stalled to Open.Sep 1 2015, 1:08 AM

saper edited projects, added acl*sre-team; removed WMF-General-or-Unknown.

Here's a complete shell script using curl to reproduce the problem without any external tools:

https://github.com/saper/upload-bug86436

This script gives the following output to me in the http.log.02 file:

P1957 bug86436 http log

1	== Info: SSL connection using TLSv1.2 / ECDHE-ECDSA-AES128-GCM-SHA256
2	== Info: Server certificate:
3	== Info: subject: C=US; ST=California; L=San Francisco; O=Wikimedia Foundation, Inc.; CN=*.wikipedia.org
4	== Info: start date: 2015-06-23 18:37:07 GMT
5	== Info: expire date: 2017-02-19 12:00:00 GMT
6	== Info: subjectAltName: test2.wikipedia.org matched
7	== Info: issuer: C=BE; O=GlobalSign nv-sa; CN=GlobalSign Organization Validation CA - SHA256 - G2
8	== Info: SSL certificate verify ok.
9	=> Send header, 436 bytes (0x1b4)
10	0000: 50 4f 53 54 20 2f 77 2f 61 70 69 2e 70 68 70 20 POST /w/api.php
11	0010: 48 54 54 50 2f 31 2e 31 0d 0a 55 73 65 72 2d 41 HTTP/1.1..User-A
12	0020: 67 65 6e 74 3a 20 55 73 65 72 3a 53 61 70 65 72 gent: User:Saper
13	0030: 20 74 65 73 74 69 6e 67 20 66 6f 72 20 68 74 74 testing for htt
14	0040: 70 73 3a 2f 2f 70 68 61 62 72 69 63 61 74 6f 72 ps://phabricator
15	0050: 2e 77 69 6b 69 6d 65 64 69 61 2e 6f 72 67 2f 6d .wikimedia.org/m
16	0060: 61 6e 69 70 68 65 73 74 2f 74 61 73 6b 2f 65 64 aniphest/task/ed
17	0070: 69 74 2f 38 36 34 33 36 2f 0d 0a 48 6f 73 74 3a it/86436/..Host:
18	0080: 20 74 65 73 74 32 2e 77 69 6b 69 70 65 64 69 61 test2.wikipedia
19	0090: 2e 6f 72 67 0d 0a 41 63 63 65 70 74 3a 20 2a 2f .org..Accept: */
20	00a0: 2a 0d 0a 43 6f 6f 6b 69 65 3a 20 43 6f 6f 6b 69 *..Cookie: Cooki
21	00b0: 65 3a 20 74 65 73 74 32 77 69 6b 69 55 73 65 72 e: test2wikiUser
22	00c0: 4e 61 6d 65 3d 53 61 70 65 72 3b 74 65 73 74 32 Name=Saper;test2
23	00d0: 77 69 6b 69 55 73 65 72 49 44 3d 35 30 35 36 3b wikiUserID=XXXX;
24	00e0: 63 65 6e 74 72 61 6c 61 75 74 68 5f 55 73 65 72 centralauth_User
25	00f0: 3d 53 61 70 65 72 3b 63 65 6e 74 72 61 6c 61 75 =Saper;centralau
26	0100: 74 68 5f 54 6f 6b 65 6e 3d XXXXXXXXXXXXXXXXXXXX th_Token=XXXXXXX
27	0110: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXX
28	0120: XXXXXXXXXXXXXXXXXXXXXXXXXX 0d 0a 43 6f 6e 74 65 XXXXXXXXX..Conte
29	0130: 6e 74 2d 4c 65 6e 67 74 68 3a 20 33 35 36 38 34 nt-Length: 35684
30	0140: 34 31 33 39 0d 0a 45 78 70 65 63 74 3a 20 31 30 4139..Expect: 10
31	0150: 30 2d 63 6f 6e 74 69 6e 75 65 0d 0a 43 6f 6e 74 0-continue..Cont
32	0160: 65 6e 74 2d 54 79 70 65 3a 20 6d 75 6c 74 69 70 ent-Type: multip
33	0170: 61 72 74 2f 66 6f 72 6d 2d 64 61 74 61 3b 20 62 art/form-data; b
34	0180: 6f 75 6e 64 61 72 79 3d 2d 2d 2d 2d 2d 2d 2d 2d oundary=--------
35	0190: 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d ----------------
36	01a0: 65 64 39 66 66 63 34 36 65 38 64 39 38 66 32 34 ed9ffc46e8d98f24
37	01b0: 0d 0a 0d 0a ....
38	<= Recv header, 39 bytes (0x27)
39	0000: 48 54 54 50 2f 31 2e 31 20 34 31 33 20 52 65 71 HTTP/1.1 413 Req
40	0010: 75 65 73 74 20 45 6e 74 69 74 79 20 54 6f 6f 20 uest Entity Too
41	0020: 4c 61 72 67 65 0d 0a Large..
42	<= Recv header, 21 bytes (0x15)
43	0000: 53 65 72 76 65 72 3a 20 6e 67 69 6e 78 2f 31 2e Server: nginx/1.
44	0010: 39 2e 33 0d 0a 9.3..
45	<= Recv header, 37 bytes (0x25)
46	0000: 44 61 74 65 3a 20 54 75 65 2c 20 30 31 20 53 65 Date: Tue, 01 Se
47	0010: 70 20 32 30 31 35 20 30 31 3a 30 32 3a 34 32 20 p 2015 01:02:42
48	0020: 47 4d 54 0d 0a GMT..
49	<= Recv header, 25 bytes (0x19)
50	0000: 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 74 65 Content-Type: te
51	0010: 78 74 2f 68 74 6d 6c 0d 0a xt/html..
52	<= Recv header, 21 bytes (0x15)
53	0000: 43 6f 6e 74 65 6e 74 2d 4c 65 6e 67 74 68 3a 20 Content-Length:
54	0010: 31 39 38 0d 0a 198..
55	<= Recv header, 19 bytes (0x13)
56	0000: 43 6f 6e 6e 65 63 74 69 6f 6e 3a 20 63 6c 6f 73 Connection: clos
57	0010: 65 0d 0a e..
58	<= Recv header, 2 bytes (0x2)
59	0000: 0d 0a ..
60	<= Recv data, 198 bytes (0xc6)
61	0000: 3c 68 74 6d 6c 3e 0d 0a 3c 68 65 61 64 3e 3c 74 <html>..<head><t
62	0010: 69 74 6c 65 3e 34 31 33 20 52 65 71 75 65 73 74 itle>413 Request
63	0020: 20 45 6e 74 69 74 79 20 54 6f 6f 20 4c 61 72 67 Entity Too Larg
64	0030: 65 3c 2f 74 69 74 6c 65 3e 3c 2f 68 65 61 64 3e e</title></head>
65	0040: 0d 0a 3c 62 6f 64 79 20 62 67 63 6f 6c 6f 72 3d ..<body bgcolor=
66	0050: 22 77 68 69 74 65 22 3e 0d 0a 3c 63 65 6e 74 65 "white">..<cente
67	0060: 72 3e 3c 68 31 3e 34 31 33 20 52 65 71 75 65 73 r><h1>413 Reques
68	0070: 74 20 45 6e 74 69 74 79 20 54 6f 6f 20 4c 61 72 t Entity Too Lar
69	0080: 67 65 3c 2f 68 31 3e 3c 2f 63 65 6e 74 65 72 3e ge</h1></center>
70	0090: 0d 0a 3c 68 72 3e 3c 63 65 6e 74 65 72 3e 6e 67 ..<hr><center>ng
71	00a0: 69 6e 78 2f 31 2e 39 2e 33 3c 2f 63 65 6e 74 65 inx/1.9.3</cente
72	00b0: 72 3e 0d 0a 3c 2f 62 6f 64 79 3e 0d 0a 3c 2f 68 r>..</body>..</h
73	00c0: 74 6d 6c 3e 0d 0a tml>..
74	== Info: Closing connection 0
75	== Info: TLSv1.2, TLS alert, Client hello (1):
76	=> Send SSL data, 2 bytes (0x2)
77	0000: 01 00 ..

I was connecting this way:

m> traceroute6 2620:0:862:ed1a::1
traceroute6 to 2620:0:862:ed1a::1 (2620:0:862:ed1a::1) from 2a01:4f8:a0:7383::, 64 hops max, 12 byte packets
 1  ex3k7.rz1.hetzner.de  0.871 ms  1.196 ms  1.169 ms
 2  hos-tr1.juniper1.rz1.hetzner.de  0.275 ms
    hos-tr3.juniper2.rz1.hetzner.de  0.280 ms
    hos-tr2.juniper1.rz1.hetzner.de  0.256 ms
 3  core11.hetzner.de  0.313 ms
    core11.hetzner.de  0.430 ms
    core11.hetzner.de  0.537 ms
 4  juniper4.rz2.hetzner.de  0.489 ms  8.861 ms  0.443 ms
 5  2001:7f8:30:0:2:1:0:6939  18.131 ms  16.295 ms  9.661 ms
 6  10ge1-4.core1.prg1.he.net  15.062 ms  15.035 ms  20.294 ms
 7  10ge10-12.core1.fra1.he.net  27.239 ms  15.267 ms  44.788 ms
 8  100ge6-1.core1.ams1.he.net  27.199 ms  20.889 ms  20.953 ms
 9  ae2.cr1-esams.wikimedia.org  21.262 ms  21.643 ms  21.037 ms
10  text-lb.esams.wikimedia.org  22.001 ms  22.034 ms  21.704 ms

The pasted headers look like the TLS terminators (nginx 1.9.3) are returning the error, which is strange because they're configured with client_max_body_size 100m; in the http section of the primary config file, and that directive isn't set anywhere else. Google searches turn up some rumblings in various nginx versions about it only actually working in server context in spite of the documentation. Will experiment later and see if we need to move it there.

BBlack added a project: Traffic.Sep 1 2015, 1:30 AM

Yann subscribed.Sep 1 2015, 11:15 AM

Jdforrester-WMF moved this task from Untriaged to Backlog on the Multimedia board.Sep 4 2015, 6:23 PM

BBlack mentioned this in T111908: Chunked upload fails (for files >5 MB).Sep 9 2015, 11:26 PM

Same problem while overwriting an existing file with a cropped version via Special:Upload on Commons:

413 Request Entity Too Large
nginx/1.9.4

555 subscribed.Sep 24 2015, 7:29 PM

zhuyifei1999 moved this task from Backlog to Uploading (UploadWizard, cross-wiki upload, chunked uploading and API) on the Commons board.Nov 15 2015, 8:51 AM

The file was 57MB in size.

If the original DjVu is 57 MB, then it's likely that the upload was indeed over 100 MB, due to https://github.com/Tpt/ia-upload/issues/5

this is a problem for uploading books
with IA moving away from dejavu the bloated jpegs will be larger
important copyright information is unworkable
https://commons.wikimedia.org/wiki/File:Upload_file_too_large.png

Restricted Application added a subscriber: Poyekhali. · View Herald TranscriptApr 18 2016, 2:07 PM

ok i have uploaded this 69.29 MB volume using metadata from IA uploader and uploading using chunked uploads.
https://commons.wikimedia.org/w/index.php?title=File:1977_Books_and_Pamphlets_July-Dec.djvu
could someone please incorporate chunked upload functionality into IA uploader?

Nemo_bis removed BBlack as the assignee of this task.Apr 21 2016, 6:26 PM

Nemo_bis edited projects, added Tools; removed SRE, Traffic.

matmarex renamed this task from Commons API fails (413 error) to upload file within 100MB threshold to Commons API and Special:Upload fail to upload file over or almost 100 MB (with cryptic "413 Request Entity Too Large" error).Jul 1 2016, 8:52 PM

matmarex edited projects, added MediaWiki-Uploading; removed Tools.

matmarex added a project: WMF-General-or-Unknown.

matmarex merged a task: T115984: Cryptic "413 Request Entity Too Large" error when uploading new version of >100MB file on Commons.

matmarex added subscribers: MarkTraceur, StudiesWorld, matmarex and 4 others.

I merged T115984 since it's the same underlying issue of server configuration having a limit on POST size (or something) that is (presumably) precisely 100 MB, which means that files slightly under that limit fail to upload without ever reaching the MediaWiki code (depending on the encoding used by browser/tool, files around 60 MB may fail).

Quiddity removed a project: WMF-General-or-Unknown.Jul 16 2016, 6:47 PM

MarkTraceur unsubscribed.Jan 27 2017, 10:11 PM

It sounds like this is not a IA Upload issue after all, but we are working on T175680: Switch to chunked uploading for large DjVus anyway.

Aklapper removed a subscriber: Anomie.Oct 16 2020, 5:02 PM

Last update on this task is over 4 years ago and various part of the infrastructure have changed considerably since that last update. Is this problem still present? Do the reproduction instructions still apply ? Can we consider T175680 being resolved a solution to this one?

It looks like it might actually be fixed! (for files <100 MB – failures for files >100 MB are expected)

I tried to upload a copy of https://commons.wikimedia.org/wiki/File:Reliëf_op_de_Boroboedoer_bij_Magelang,_KITLV_D13566.tiff (90 MB), using Special:Upload and using API action=upload (without chunking). The upload to stash succeeded in both cases.

And just for completeness, I also tried to upload a copy of https://commons.wikimedia.org/wiki/File:EASTER_EGG_ROLLING,_WHITE_HOUSE_LCCN2016865362.tif (147 MB), and it failed with a "Wikimedia Error" as expected.

matmarex renamed this task from Commons API and Special:Upload fail to upload file over or almost 100 MB (with cryptic "413 Request Entity Too Large" error) to Commons API and Special:Upload fail to upload file under 100 MB (with cryptic "413 Request Entity Too Large" error).Jan 7 2022, 11:36 PM

Commons API and Special:Upload fail to upload file under 100 MB (with cryptic "413 Request Entity Too Large" error)Closed, ResolvedPublicActions

Description

Related Objects

Event Timeline

Commons API and Special:Upload fail to upload file under 100 MB (with cryptic "413 Request Entity Too Large" error)
Closed, ResolvedPublic
Actions