Page MenuHomePhabricator

Spike test proposal: upload transformations/conversions
Open, Needs TriagePublic

Description

T211745 proposes automatic conversion of HEVC-baed HEIF/HEIC images to a free format (jpg/webp/avif/etc) with retention of the original files so they can be in the future referenced and made available for download. Similar proposals have been made before I believe for H.264 and HEVC .mp4 videos, based on peoples' past workflows with video2commons.

If we accept such files and "silently convert" them to a transformed format that, while lossy, maintains similar compression quality and the video format capabilities, it might resolve some of our difficulties with bringing that tool in-house.

Recommend a spike test working out what it would take to add a transform stage to the API uploadstash pipeline (and thus support in UploadWizard etc) where we could publish the original file into a holding area for such files, and direct the front-end to our replacement .avif or .webm file.

Encoding speed may be a factor for video, but for now we just need to work out the workflow in the MediaWiki internals.

Assigning to myself as my pet project: if result looks workable I'll push for more work time on it.

Event Timeline

I would suggest, by the way, that the original file be conserved on the server side, in part to allow for potential better re-encodes in the future (as well as release of the original, once the codecs are free).

Encoding speed may be a factor for video, but for now we just need to work out the workflow in the MediaWiki internals.

For reference, recent IGPs can transcode video with negligible power consumption very quickly. Closed source drivers not required. For example:

ffmpeg -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device /dev/dri/renderD128 -i testinput.mkv -c:v hevc_vaapi -vf scale_vaapi=1920:1080 -b:v 2M -b:a 256k -c:a libvorbis testoutput.mp4

This converts 3840x2160 30fps HEVC video to 1920x1080 30fps HEVC at almost twice the playback speed on my four-year old puny AMD Lucienne chip using the hardware for both decoding and encoding. With 4K HEVC output the speed is halved. (so roughly realtime) This chip is too old to support AV1. As newer chips are designed to handle 8K, they should be very fast with 4K material.

Image quality, compatibility and other factors would have to be researched if this is pursued, obviously.

Beware of https://github.com/GPUOpen-LibrariesAndSDKs/AMF/issues/423 before buying AMD though. As you mentioned, this is a headache for a later time. Just making notes for whom might find them useful.