Page MenuHomePhabricator

integration-slave-trusty-1012, -1013, and -1015 unresponsive
Closed, ResolvedPublic


These three instances have been unresponsive for the last 24 hours. I'm also not able to establish an SSH connection.

integration-slave-precise-1011.eqiad.wmflabs: 1
integration-slave-precise-1012.eqiad.wmflabs: 1
integration-slave-precise-1013.eqiad.wmflabs: 1
integration-slave-precise-1014.eqiad.wmflabs: 1
integration-slave-trusty-1011.eqiad.wmflabs: 1
integration-slave-trusty-1014.eqiad.wmflabs: 1
integration-slave-trusty-1016.eqiad.wmflabs: 1
integration-slave-trusty-1017.eqiad.wmflabs: 1

Connection closed:

  • integration-slave-trusty-1012.eqiad.wmflabs
  • integration-slave-trusty-1013.eqiad.wmflabs
  • integration-slave-trusty-1015.eqiad.wmflabs

YuviPanda: Krinkle: I get "Connection closed by Unknown", usually this means something OOM'd
YuviPanda: Krinkle: no output in 'get console output' due to wikitech flakiness, I presume.

Event Timeline

Krinkle raised the priority of this task from to Unbreak Now!.
Krinkle updated the task description. (Show Details)
Krinkle added subscribers: Krinkle, hashar, yuvipanda. was on integration-slave-trusty-1012 and timed out after 30 minutes. Should we depool the stuck trusty slaves for now?

I marked all 3 slaves as offline in jenkins.

hashar claimed this task.

Seems it was some a transient labs issue. I have rebooted the three instances and repooled them:


Restricted Application added subscribers: Jay8g, TerraCodes. · View Herald Transcript