Page MenuHomePhabricator

When several tasks are added to the transcoding queue at once, they seem to all get sent to the same machine
Open, Needs TriagePublic

Description

Even though my understanding is that it's not really 'load balancing'.

While trying to address the backlog of 'broken transcodes' on Commons, and watching the load on ganglia, I have noticed... when several tasks are added to the queue 'at once', they seem to all get sent to the same machine, even if the other one is sitting at idle. This can, easily, send one machine to well over 75% 'load' while leaving the other one at essentially nothing...the transcodes (if they are of 'large'.... HD, and over 45 min or so) files then error out. This does not seem to be a 'file problem'... the same files successfully work at other resolutions, and work if put through the queue when the servers are not loaded.

Event Timeline

Revent created this task.Nov 8 2016, 8:59 AM
Restricted Application added subscribers: Poyekhali, Aklapper. · View Herald TranscriptNov 8 2016, 8:59 AM
Aklapper renamed this task from load-balancing to When several tasks are added to the transcoding queue at once, they seem to all get sent to the same machine.Nov 8 2016, 1:34 PM