Project Name: video2commons

Purpose: For the backend of transferring videos from other sites to commons.

Wikitech Username of requestor: Zhuyifei1999

Why this can't be on Tool Labs?
The migration to k8s in order to leave NFS means not a lot of storage space could be allocated to each tool webservice. Webservice needs to be able to serve those files too large to upload directly to commons (>1000 MB) in order to make use of server side uploading (see Commons:Batch uploading/Public domain films for examples). Besides, the shared nature of tool labs exec nodes means not a lot of computational power could be given to the transcoding jobs, making transcoding slow. Also, the task queue video2commons planning to use is celery, with require a queuing service, such as redis. Celery do not have configuration to prefix redis keys, thus tools-redis cannot be used securely, and the redis must be hosted elsewhere.

Why this can't be on Nova Resource:Encoding01.video.eqiad.wmflabs?
Encoding01 has only the "default" service group, thus no usable open port to other labs projects. Thus, encoding01 lack the ability to serve the too-large files via http for server side uploading. Also, converting subtitles require ffmpeg instead of libav that encoding01 currently use, as the latter have no way of enabling an srt encoder. libav and ffmpeg are also incompatible: $ sudo apt-get install ffmpeg will remove libav. While workarounds exist for these problems, I've been having difficulty contacting @Matanya due to timezone issues, as when he's available it's usually like 3 or 4 o'clock in the morning for me.