Page MenuHomePhabricator

Retool transcode table with a status column
Open, Needs TriagePublic

Description

The transcode table in TimedMediaHandler doesn't currently store the status in a straightforward way, which makes the table awkward to query in bulk at the scale of Wikimedia Commons.

Instead of implicitly storing the state of a transcode as the null/present state of various timestamp columns, add a column with suitable state constants and suitable indexes for pulling bulk info and most-recently-added N items to each state.

  • add column: transcode_status INT (eg 0 - 'missing', 1 - 'queued', 2 - 'active', 3 - 'failed')
  • add column: transcode_touched VARBINARY(14)
  • add index: `transcode_status_touched (transcode_status,transcode_touched)
  • update code to set transcode_state and transcode_touched on each update [switchable or carefully deployed]
  • update special:transcode_statistics to use the new fields [switchable or carefully deployed]
  • maintenance script to fill out the fields on existing rows after deployment
  • prep for deployment of the db update

Event Timeline

Yup, we shouldn't use ENUM for schema anymore, just use code constants (missing=0, queued=1, etc.)

@bvibber if we are going to run db updates anyways. Perhaps we should add a column for the file size of the derivs to the table as well ? Those are currently unknown and there's been a desire in the past to get better insight into that information. T57942: Expose file sizes of transcoded assets in API

Agreed let's do those together...

Yup, we shouldn't use ENUM for schema anymore, just use code constants (missing=0, queued=1, etc.)

You're right, enum is asking for trouble next time a state gets added. ;) Updated task description.

Reedy renamed this task from Retool transcode table with a status enum column to Retool transcode table with a status column.Tue, Apr 9, 7:44 PM

Change #1018347 had a related patch set uploaded (by Bvibber; author: Bvibber):

[mediawiki/extensions/TimedMediaHandler@master] WIP transcode table updates

https://gerrit.wikimedia.org/r/1018347