Page MenuHomePhabricator

Please mass-reset the video transcodes of tens of thousand videos stuck in "Unknown" state
Closed, ResolvedPublic

Description

Appears to still be somewhat broken, ex: https://commons.wikimedia.org/wiki/File:09._Гасење_вар_како_силно_егзотермент_процес.webm uploaded over a month ago did not auto-transcode anything (I believe it originally just said "Transcode not required") until I happened to purge the cache and kick off the 1080p transcode with the "Reset" link (which worked). I'm guessing perhaps it error-ed out? Reedy was helping me out and tried to find logs but couldn't.

On a side note, does anyone know how to link to the transcoded version of a video? is it even possible? when one uses link=https://upload.wikimedia.org/wikipedia/commons/transcoded/a/a7/09._Гасење_вар_како_силно_егзотермент_процес.webm/09._Гасење_вар_како_силно_егзотермент_процес.webm.1080p.ogv in the [[File:...|...]] tag, it seems to completely ignore the target.
My assumption was the player would auto-choose/play the version that is compatible with your browser, but it seems like the only ability that is given is to download any transcoded formats. Would be nice if it did the former...

Event Timeline

TheDJ created this task.Nov 21 2016, 9:21 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 21 2016, 9:21 AM
TheDJ added a comment.Nov 21 2016, 9:47 AM

If the table shows that the status is "unknown", then i think that means that all the column entries in the transcode table are still at NULL, or not present at all. That means that the Job was not started, or got lost. Since all entries are missing, I suspect that either startJobQueue (which dispatches all the jobs), either was never fired, or that an exception in the init of the first job, cause all subsequent jobs to not be added.

Now all jobs seem to have started, and they all seem to be stuck in some way. I wonder if this indicates an error with the parsing of the original file.

Jdforrester-WMF renamed this task from Video transcode jobs stuck in Unknown state. to Please mass-reset the video transcodes of tens of thousand videos stuck in "Unknown" state.Mar 17 2017, 11:29 PM

Looks like maintenance/resetTranscodes.php in TMH is what's needed, but presumably only run in small batches rather than all at once?

Revent added a subscriber: Revent.Mar 19 2017, 3:30 AM

I have actually been doing this (through the API) in reasonable-sized batches for several days now.

My impression is that these are mainly cases where additional transcode targets were added after the file was uploaded, and the 'new' targets were not added until Dispenser used a bot to purge, in mass, all of the file pages of videos.

It has also happened in the past when files were undeleted, or moved, but I think those bugs were fixed a while back.

brion added a subscriber: brion.Mar 20 2017, 6:49 PM

I've started a job running requeueTranscodes.php commonswiki --video --missing --throttle which should clean them all up.

resetTranscodes.php is obsolete and will be removed.

greg added a subscriber: greg.

I've started a job running requeueTranscodes.php commonswiki --video --missing --throttle which should clean them all up.
resetTranscodes.php is obsolete and will be removed.

Status?

brion closed this task as Resolved.May 19 2017, 2:54 PM
brion claimed this task.

@greg looks like the requeueTranscodes.php commonswiki --video --missing --throttle run finished April 2.

Closing this out for now; @Revent please feel free to open new tasks or ping me on other ones if the move-related bugs are still striking. (Some other transcode fail errors are also out there, but the large batch should be all done.)