Page MenuHomePhabricator

Chunked upload fails (for files >5 MB)
Closed, ResolvedPublic

Description


Upload stops at "processing file". Filesize is 6 MB. Smaller files are uploaded.

Event Timeline

Sebaso created this task.Sep 9 2015, 7:43 AM
Sebaso raised the priority of this task from to Needs Triage.
Sebaso updated the task description. (Show Details)
Sebaso added a project: UploadWizard.
Sebaso added a subscriber: Sebaso.
Restricted Application added a project: Multimedia. · View Herald TranscriptSep 9 2015, 7:43 AM
Restricted Application added subscribers: Steinsplitter, Aklapper. · View Herald Transcript

Can you provide a copy of the file? You can upload it here to Phabricator by dragging-and-dropping it to the comment field, or at https://phabricator.wikimedia.org/file/upload/.

Restricted Application added a subscriber: Matanya. · View Herald TranscriptSep 9 2015, 2:51 PM
Steinsplitter set Security to None.
Steinsplitter moved this task from Incoming to Backlog on the Commons board.
Herzi.Pinki triaged this task as High priority.Sep 9 2015, 7:03 PM
Herzi.Pinki added a subscriber: Herzi.Pinki.

Sorry for interfering, but I can reproduce the behavior (different file, different ISP, different country), for small files it works like a charm, for larger files (mine was 6.5 Mb), it hangs. I did not try to verify the maximum size where it still does work.

As WLM is running at the moment worldwide, and especially UW is an offer to new and unexperienced users, please have a look how you can fix this. 6 Mb is not an unusal size.
I have changed the prio to High. (I think it is really urgent)

Thanks @Sebaso. The upload went through for me (I stopped before going further in the wizard), so this isn't ubiquitous, at least. But then I don't know how I could help you resolve this :/

Tsungam added a subscriber: Tsungam.Sep 9 2015, 8:06 PM

Could this be a problem with chunked uploads? With chunked uploads enabled, only 3 of 13 files succeeded to upload, these three were below 5 MB. After disabeling chunked uploads, all 13 files succeeded.

Herzi.Pinki added a comment.EditedSep 9 2015, 9:13 PM

Could this be a problem with chunked uploads? With chunked uploads enabled, only 3 of 13 files succeeded to upload, these three were below 5 MB. After disabeling chunked uploads, all 13 files succeeded.

Same for me, I can confirm: chunked uploads > 5 Mb (which is checked on the size truncated to Mb, so it is only true for files ≥ 6Mb, I suspect) will not work. When I switch off chunked uploads, then the file > 6 Mb was uploaded correctly.

An individual workaround would be for everyone to change UW properties correspondingly. But a more general approach would be to ignore the chunked-upload flag as long as the problem persists.

I can reproduce after enabling chunked uploads.

matmarex renamed this task from Upload failing (6 MB file) to Chunked upload fails (for files >5 MB).Sep 9 2015, 9:51 PM

{"servedby":"mw1135","error":{"code":"badparams","info":"Must supply a filekey when offset is non-zero","*":"See https://commons.wikimedia.org/w/api.php for API usage"}}

matmarex claimed this task.Sep 9 2015, 10:30 PM

Change 237285 had a related patch set uploaded (by Bartosz Dziewoński):
Always set 'offset' with chunked uploads, even for first chunk (offset == 0)

https://gerrit.wikimedia.org/r/237285

Investigation notes:

  • I can't reproduce this error locally, even after enabling chunked uploads. The two chunks of this file happily upload.
  • I verified that the above patch works by using the in-browser JS debugger at Commons to apply such a change there.

I have no idea why this is failing, nor why it started failing now.

  • I would blame @MarkTraceur's change 681e26a7eae9d0d58910f5b3909d93c132ba7c42 in UploadWizard, which changed the code not to send the 'offset' if it would be 0, but that was in January. Surely someone would have noticed some time over the last 8 months if chunked uploads were broken?
  • There were several more recent changes to the action=upload API (last one being 11893e476189ede8db3df9b21e8d369671b53a19, with the ominous message "Improve upload error reporting"), but at closer inspection they seem unrelated to this issue. Maybe @Anomie can guess something.
  • Since it works for me locally, perhaps this is caused by some Wikimedia servers' config weirdness? @BBlack, did you guys change anything related recently?

Anyway, for now I think we should merge and deploy that patch, and see what happens.

Tgr added a subscriber: Tgr.Sep 9 2015, 11:21 PM

Apparently the chunked upload browser tests have been failing consistently since May.

Well, I see a success in May, and then failures starting in August. No runs between?

Also, any chance this is related? T86436 - was originally reported back in Jan?

Tgr added a comment.Sep 9 2015, 11:36 PM

I'm not familiar with Jenkins; I just assumed Jenkins discarded old build logs apart from the last successful one. Maybe there was a CI change that disabled that job for a while?

matmarex moved this task from Untriaged to Prototyping on the Multimedia board.Sep 10 2015, 1:08 AM
matmarex moved this task from Prototyping to Doing on the Multimedia board.
matmarex moved this task from Doing to Needs code review on the Multimedia board.

I can reproduce after enabling chunked uploads.

thx for your work. just to confirm: yes, I'm using the chunked upload feature.

Change 237285 merged by jenkins-bot:
Always set 'offset' with chunked uploads, even for first chunk (offset == 0)

https://gerrit.wikimedia.org/r/237285

Change 237383 had a related patch set uploaded (by MarkTraceur):
Always set 'offset' with chunked uploads, even for first chunk (offset == 0)

https://gerrit.wikimedia.org/r/237383

Jarekt added a subscriber: Jarekt.Sep 10 2015, 2:31 PM

I had the same problem with a set of large photographs (5-30 Mb). I tried many times to upload them since this weekend but even overnight upload never finishes. I will retry without chunked upload. The photographs I was uploading can be seen at https://www.flickr.com/photos/15437873@N05/albums/72157658353009182, where I also uploaded them.

Change 237383 merged by jenkins-bot:
Always set 'offset' with chunked uploads, even for first chunk (offset == 0)

https://gerrit.wikimedia.org/r/237383

matmarex closed this task as Resolved.Sep 10 2015, 4:26 PM

This is fixed now, and the fix is deployed on Commons. I've learned that the root cause was a security patch which unintentionally made validation for 'offset' parameter stricter. Thank you for reporting this!