We should confirm whether this is purely timeout related (due to slow upload speeds), or something else, I guess by trying a large file from a known-fast source, or even from inside our networks somewhere?

In T128358#2072888, @BBlack wrote:

We should confirm whether this is purely timeout related (due to slow upload speeds), or something else, I guess by trying a large file from a known-fast source, or even from inside our networks somewhere?

The uploading script is within WMFLabs video cluster, there should be no reason why the network is slow.

@Krenair What's the typical time needed to upload these kind of files from server side?

I'm not sure if it's related, but action=purge on 3GB+ videos results in 503 as well. (Will file a separate task)

I don't really watch the clock (I tend to read email while those scripts are running), and importImages doesn't give a time. I've never seen it timeout when uploading from terbium, but it doesn't take more than a few minutes there.

In T128358#2074496, @Krenair wrote:

I don't really watch the clock (I tend to read email while those scripts are running), and importImages doesn't give a time. I've never seen it timeout when uploading from terbium, but it doesn't take more than a few minutes there.

Well, I don't think maintenance scripts times out at all. If it runs longer than 2 min, it might be a 503 if ran in a similar way as API upload.

The log above has most chunks uploading for 20-30 seconds, but the last chunk runs for 2:16 before receiving 503. I'm assuming there's some chunk-merging process happening.

A possible workaround is to use async chunked uploading, but pywikibot does not yet support so.

In T128358#2098525, @zhuyifei1999 wrote:

A possible workaround is to use async chunked uploading, but pywikibot does not yet support so.

Without async option, I would imagine the upload process should fail long before 1.2GB

Multimedia team bowing out of working on this because it seems like it might be a pywikibot problem (namely not using async), not a problem with our software.

Hello. I seem to have run up against this problem. This seems like a serious barrier to contributing core content (media files into Commons). Can this receive higher priority, please?

If this really is a Pywikibot problem, it probably helps with prioritization if the relevant project is tagged.

At a glance though pywikibot does seem to support chunked uploads (upload.py at least has a -chunked option) so maybe try using that?

Pywikibot does support chunked uploading, this task is about why such chunked uploading must be using async mode, and if it must be done, how to fail gracefully with an error message.

The difference between sync and async chunked uploading is that the file recombining and publishing (some large swift operations) is done on the job queue in async mode, and the client just polls the status from the job queue; sync mode does the swift operation on-the-spot and the request does not return until the operation is done.

On a side note, there are few tools/scripts that support async chunked uploading (UW, Rillke's script, and v2c are the only ones I know of), and there are a few tools/scripts that support chunked uploading only in sync mode, such as videoconvert, VicuñaUploader, vanilla pywikibot, and even our own documentation.

(@Tgr - my concern is for end-users not using Pywikibot, but the videoconvert tool. Thanks to @zhuyifei1999 and @Harej I learned that video2commons supports uploads (perhaps it used to not support uploads?), so there is a workaround for end-users for the moment. This, then, becomes a videoconvert tool limitation, rather than a blocker for end-user video uploads for people not familiar with the command line (ffmpeg etc.).)

(unfortunately, videoconvert does not seem to be tracking issues here on Phabricator...)

I doubt much can be done about this on the MediaWiki side (other than a better error message) so probably those tools need to switch to async uploading. Vicuna's issue tracker is here, Pywikipedia already has a task (T129216: Pywikibot should support async chunked uploading), couldn't find the place for videoconvert.