Page MenuHomePhabricator

Move transcoded videos out of thumbs containers
Closed, ResolvedPublic

Description

We currently have two main types of containers, "originals" and "thumbs". My understanding is that uploaded videos got into "originals" and transcoded variants of those into "thumbs".

Transcoded videos are obviously not original content, however their characteristics are very different from those of thumbnails: they're on the opposite end of the file size spectrum and it's not that easy/cheap to just regenerate them.

This affects various choices that we make and will make on the infrastructure in the future: whether we should store thumbs in file storage or just cache them, how many replicas of thumbs we should keep, if we should have a separate caching layer for videos etc.

My idea was on having a third container type for "videos", for both original and transcoded content. I suppose could also have two, one for original videos and one for transcoded videos.

But I don't have a strong opinion on how it should be done. What do people think?


Version: unspecified
Severity: normal

Details

Reference
bz43343

Event Timeline

bzimport raised the priority of this task from to Normal.Nov 22 2014, 1:02 AM
bzimport added a project: TimedMediaHandler.
bzimport set Reference to bz43343.
faidon created this task.Dec 22 2012, 2:44 PM

jgerber wrote:

this would require changes to core, File right now does not allow setting any other location. for transcodes TMH uses getThumbPath this could be changed to point to another zone, not sure if this will cause issues in other places right now.

jgerber wrote:

https://gerrit.wikimedia.org/r/#/c/45102/
adds a "media" container to core that can be used for audio/video derivatives

https://gerrit.wikimedia.org/r/#/c/45101/
makes TMH use the media container if available to store derivatives,
it also comes with a maintenance script to move all derivatives to the media container for migration.

storing originals in another container is not possible since this would depend on the media handler and the current upload logic does not know about that.

Both of those were merged by Aaron Schulz on the 5th. Is this bug resolved now?

Let's assume it is then.

The patches were merged but a) there are more changes pending, b) the videos need to be actually moved via a maintenance script.

jgerber wrote:

bug 45294 is fixed now so the migration is done.

Gilles raised the priority of this task from Normal to Unbreak Now!.Dec 4 2014, 10:27 AM
Gilles moved this task from Untriaged to Done on the Multimedia board.
Gilles lowered the priority of this task from Unbreak Now! to Normal.Dec 4 2014, 11:23 AM