Page MenuHomePhabricator

Investigate video transcoding race conditions with file revert/reupload
Open, Needs TriagePublic

Description

While investigating T114557, someone uploaded a new version of https://commons.wikimedia.org/wiki/File:The_President_Delivers_a_Statement_on_the_Shooting_in_Oregon.webm in the middle of testing, which I had to revert to finish testing.

There *appear* to have been transcode jobs from the previous version which survived the revert process, showing bogusly short encoding times like 42 seconds:

These appear to be being replaced over time with new transcodes from after the revert, but it's all very mysterious.

Ideal behavior is that uploading a new version should halt any running transcode jobs on old versions... but there's no infrastructure for halting jobs remotely. At the least, the existing job should cleanly die out when it finishes, or something.... argh. Need to run this in an isolated environment to make sure I can tell what's going on.

Event Timeline

brion created this task.Oct 6 2015, 6:33 PM
brion raised the priority of this task from to Needs Triage.
brion updated the task description. (Show Details)
brion added a project: TimedMediaHandler.
brion added a subscriber: brion.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 6 2015, 6:33 PM
Paladox set Security to None.Oct 6 2015, 7:09 PM
Paladox added a subscriber: Paladox.
brion added a comment.Oct 6 2015, 7:11 PM

Ok, it *looks* like the old jobs keep running and post their files, then the new jobs run and post *their* files. So it ends up correct in the end, but we've got some weird files in-flight in the meantime, which could be very confusing when doing a fast revert cycle...

TheDJ added a subscriber: TheDJ.Oct 6 2015, 7:15 PM

Before starting and finishing the job and delivering the new file, we should check for current revision id == revision id the transcode job was initiated with.

TheDJ moved this task from To sort to Transcoding on the TimedMediaHandler board.Oct 21 2015, 6:59 PM