Page MenuHomePhabricator
Paste P6948

lvs1006-mathoid.log
ActivePublic
Actions

Authored by • ema on Apr 4 2018, 11:17 PM.
Tags
None
Referenced Files
F16683317: lvs1006-mathoid.log
Apr 4 2018, 11:17 PM
Subscribers
None
Apr 04 02:21:10 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1001.eqiad.wmnet (enabled/up/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 02:21:10 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1001.eqiad.wmnet because of too many down!
Apr 04 02:21:10 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1001.eqiad.wmnet (enabled/partially up/pooled): Connection to 10.64.0.121:10042 failed.
Apr 04 02:21:13 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1001.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 02:21:13 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1001.eqiad.wmnet pooled
Apr 04 06:23:45 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/up/pooled): Fetch failed, 0.002 s
Apr 04 06:23:45 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1002.eqiad.wmnet (enabled/up/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 06:23:45 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1002.eqiad.wmnet because of too many down!
Apr 04 06:23:45 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/up/pooled): Fetch failed, 0.005 s
Apr 04 06:23:45 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1003.eqiad.wmnet (enabled/up/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 06:23:45 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1003.eqiad.wmnet because of too many down!
Apr 04 06:23:45 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/up/pooled): Fetch failed, 0.007 s
Apr 04 06:23:45 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1004.eqiad.wmnet (enabled/up/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 06:23:45 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1004.eqiad.wmnet because of too many down!
Apr 04 06:23:55 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1002.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 06:23:55 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1002.eqiad.wmnet pooled
Apr 04 06:23:55 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1003.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 06:23:55 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1003.eqiad.wmnet pooled
Apr 04 06:23:55 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1004.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 06:23:55 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1004.eqiad.wmnet pooled
Apr 04 22:24:35 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.102 s
Apr 04 22:24:35 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1004.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:24:35 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1004.eqiad.wmnet because of too many down!
Apr 04 22:24:35 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.021 s
Apr 04 22:24:35 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1001.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:24:35 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1001.eqiad.wmnet because of too many down!
Apr 04 22:24:36 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.002 s
Apr 04 22:24:36 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1003.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:24:36 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1003.eqiad.wmnet because of too many down!
Apr 04 22:24:45 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1004.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:24:45 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1004.eqiad.wmnet pooled
Apr 04 22:24:45 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1001.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:24:45 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1001.eqiad.wmnet pooled
Apr 04 22:24:46 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1003.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:24:46 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1003.eqiad.wmnet pooled
Apr 04 22:24:47 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.171 s
Apr 04 22:24:47 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1002.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:24:47 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1002.eqiad.wmnet because of too many down!
Apr 04 22:24:59 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1002.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:24:59 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1002.eqiad.wmnet pooled
Apr 04 22:25:12 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.002 s
Apr 04 22:25:12 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1001.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:25:12 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1001.eqiad.wmnet because of too many down!
Apr 04 22:25:12 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.001 s
Apr 04 22:25:12 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1004.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:25:12 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1004.eqiad.wmnet because of too many down!
Apr 04 22:25:12 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.001 s
Apr 04 22:25:12 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1003.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:25:12 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1003.eqiad.wmnet because of too many down!
Apr 04 22:25:14 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.002 s
Apr 04 22:25:14 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1002.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:25:14 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1002.eqiad.wmnet because of too many down!
Apr 04 22:25:26 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1003.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:25:26 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1003.eqiad.wmnet pooled
Apr 04 22:25:26 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1002.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:25:26 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1002.eqiad.wmnet pooled
Apr 04 22:25:32 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 8.110 s
Apr 04 22:25:36 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 12.365 s
Apr 04 22:25:43 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1001.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:25:43 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1001.eqiad.wmnet pooled
Apr 04 22:25:51 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 5.001 s
Apr 04 22:25:56 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1004.eqiad.wmnet (enabled/down/pooled) down: Connection to the other side was lost in a non-clean fashion.
Apr 04 22:25:56 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] INFO: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.48.52:10042 lost.
Apr 04 22:25:58 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.001 s
Apr 04 22:25:58 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1001.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:25:58 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1001.eqiad.wmnet because of too many down!
Apr 04 22:25:58 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/up/pooled): Fetch failed, 8.024 s
Apr 04 22:25:58 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1002.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:25:58 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1002.eqiad.wmnet because of too many down!
Apr 04 22:26:08 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1001.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:26:08 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1001.eqiad.wmnet pooled
Apr 04 22:26:09 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1002.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:26:09 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1002.eqiad.wmnet pooled
Apr 04 22:26:14 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 12.118 s
Apr 04 22:26:20 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/up/pooled): Fetch failed, 30.001 s
Apr 04 22:26:20 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1003.eqiad.wmnet (enabled/up/pooled) down: User timeout caused connection failure.
Apr 04 22:26:20 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1003.eqiad.wmnet because of too many down!
Apr 04 22:26:23 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.001 s
Apr 04 22:26:23 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1001.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:26:23 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1001.eqiad.wmnet because of too many down!
Apr 04 22:26:29 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 5.006 s
Apr 04 22:26:30 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1003.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:26:30 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1003.eqiad.wmnet pooled
Apr 04 22:26:33 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1001.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:26:33 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1001.eqiad.wmnet pooled
Apr 04 22:26:45 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 5.115 s
Apr 04 22:26:45 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.015 s
Apr 04 22:26:45 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1003.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:26:45 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1003.eqiad.wmnet because of too many down!
Apr 04 22:26:46 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1002.eqiad.wmnet (enabled/up/pooled) down: Connection to the other side was lost in a non-clean fashion.
Apr 04 22:26:46 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1002.eqiad.wmnet because of too many down!
Apr 04 22:26:46 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] INFO: kubernetes1002.eqiad.wmnet (enabled/partially up/pooled): Connection to 10.64.16.75:10042 lost.
Apr 04 22:26:46 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1003.eqiad.wmnet (enabled/down/pooled) down: Connection to the other side was lost in a non-clean fashion.
Apr 04 22:26:46 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] INFO: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.32.23:10042 lost.
Apr 04 22:26:46 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 5.001 s
Apr 04 22:26:46 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1002.eqiad.wmnet (enabled/down/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:26:49 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.201 s
Apr 04 22:26:49 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1001.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:26:49 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1001.eqiad.wmnet because of too many down!
Apr 04 22:26:55 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1004.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:26:55 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1004.eqiad.wmnet pooled
Apr 04 22:26:55 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1003.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:26:55 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1003.eqiad.wmnet pooled
Apr 04 22:26:57 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1002.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:26:57 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1002.eqiad.wmnet pooled
Apr 04 22:26:59 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1002.eqiad.wmnet (enabled/up/pooled) down: Connection to the other side was lost in a non-clean fashion.
Apr 04 22:26:59 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1002.eqiad.wmnet because of too many down!
Apr 04 22:26:59 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] INFO: kubernetes1002.eqiad.wmnet (enabled/partially up/pooled): Connection to 10.64.16.75:10042 lost.
Apr 04 22:26:59 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1001.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:26:59 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1001.eqiad.wmnet pooled
Apr 04 22:27:02 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1002.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:27:02 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1002.eqiad.wmnet pooled
Apr 04 22:27:12 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.041 s
Apr 04 22:27:12 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1002.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:27:12 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1002.eqiad.wmnet because of too many down!
Apr 04 22:27:20 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.002 s
Apr 04 22:27:20 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1004.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:27:20 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1004.eqiad.wmnet because of too many down!
Apr 04 22:27:27 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 5.049 s
Apr 04 22:27:32 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1004.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:27:32 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1004.eqiad.wmnet pooled
Apr 04 22:27:37 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1002.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:27:37 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1002.eqiad.wmnet pooled
Apr 04 22:29:55 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.048 s
Apr 04 22:29:55 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1002.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:29:55 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1002.eqiad.wmnet because of too many down!
Apr 04 22:30:05 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1002.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:30:05 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1002.eqiad.wmnet pooled
Apr 04 22:30:18 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.001 s
Apr 04 22:30:18 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1003.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:30:18 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1003.eqiad.wmnet because of too many down!
Apr 04 22:30:19 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.002 s
Apr 04 22:30:19 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1001.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:30:19 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1001.eqiad.wmnet because of too many down!
Apr 04 22:30:19 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.002 s
Apr 04 22:30:19 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1004.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:30:19 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1004.eqiad.wmnet because of too many down!
Apr 04 22:30:20 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.001 s
Apr 04 22:30:20 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1002.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:30:20 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1002.eqiad.wmnet because of too many down!
Apr 04 22:30:33 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 5.001 s
Apr 04 22:30:34 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 5.001 s
Apr 04 22:30:34 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 5.001 s
Apr 04 22:30:35 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 5.001 s
Apr 04 22:30:43 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.085 s
Apr 04 22:30:44 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.049 s
Apr 04 22:30:44 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.050 s
Apr 04 22:30:45 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.001 s
Apr 04 22:30:46 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1001.eqiad.wmnet (enabled/down/pooled) down: Connection to the other side was lost in a non-clean fashion.
Apr 04 22:30:46 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] INFO: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.0.121:10042 lost.
Apr 04 22:30:46 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1002.eqiad.wmnet (enabled/down/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 22:30:46 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.16.75:10042 failed.
Apr 04 22:30:46 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1004.eqiad.wmnet (enabled/down/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 22:30:46 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.48.52:10042 failed.
Apr 04 22:30:49 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.16.75:10042 failed.
Apr 04 22:30:49 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.0.121:10042 failed.
Apr 04 22:30:50 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.48.52:10042 failed.
Apr 04 22:30:54 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.156 s
Apr 04 22:30:54 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.175 s
Apr 04 22:30:54 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.175 s
Apr 04 22:30:56 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1002.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:30:56 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1002.eqiad.wmnet pooled
Apr 04 22:31:04 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1003.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:31:04 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1003.eqiad.wmnet pooled
Apr 04 22:31:04 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1001.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:31:04 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1001.eqiad.wmnet pooled
Apr 04 22:31:04 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1004.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:31:04 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1004.eqiad.wmnet pooled
Apr 04 22:37:46 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.002 s
Apr 04 22:37:46 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1002.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:37:46 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1002.eqiad.wmnet because of too many down!
Apr 04 22:37:47 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.124 s
Apr 04 22:37:47 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1004.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:37:47 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1004.eqiad.wmnet because of too many down!
Apr 04 22:37:56 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1002.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:37:56 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1002.eqiad.wmnet pooled
Apr 04 22:37:57 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1004.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:37:57 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1004.eqiad.wmnet pooled
Apr 04 22:37:57 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.056 s
Apr 04 22:37:57 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1003.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:37:57 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1003.eqiad.wmnet because of too many down!
Apr 04 22:38:07 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1003.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:38:07 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1003.eqiad.wmnet pooled
Apr 04 22:44:13 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.047 s
Apr 04 22:44:13 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1001.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:44:13 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1001.eqiad.wmnet because of too many down!
Apr 04 22:44:22 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.012 s
Apr 04 22:44:22 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1003.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:44:22 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1003.eqiad.wmnet because of too many down!
Apr 04 22:44:24 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.071 s
Apr 04 22:44:24 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1002.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:44:24 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1002.eqiad.wmnet because of too many down!
Apr 04 22:44:28 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 5.003 s
Apr 04 22:44:35 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/up/pooled): Fetch failed, 6.012 s
Apr 04 22:44:35 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1004.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:44:35 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1004.eqiad.wmnet because of too many down!
Apr 04 22:44:40 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 7.735 s
Apr 04 22:44:40 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 6.237 s
Apr 04 22:44:46 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 6.140 s
Apr 04 22:44:50 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.001 s
Apr 04 22:44:50 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.001 s
Apr 04 22:44:51 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 6.237 s
Apr 04 22:44:56 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1002.eqiad.wmnet (enabled/down/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 22:44:56 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.16.75:10042 failed.
Apr 04 22:44:56 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1001.eqiad.wmnet (enabled/down/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 22:44:56 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.0.121:10042 failed.
Apr 04 22:44:56 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1003.eqiad.wmnet (enabled/down/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 22:44:56 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.32.23:10042 failed.
Apr 04 22:44:56 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.001 s
Apr 04 22:44:57 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1004.eqiad.wmnet (enabled/down/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 22:44:57 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.48.52:10042 failed.
Apr 04 22:44:59 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.32.23:10042 failed.
Apr 04 22:44:59 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.16.75:10042 failed.
Apr 04 22:44:59 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.0.121:10042 failed.
Apr 04 22:45:00 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.48.52:10042 failed.
Apr 04 22:45:00 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.001 s
Apr 04 22:45:00 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.001 s
Apr 04 22:45:01 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.014 s
Apr 04 22:45:07 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1001.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:45:07 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1001.eqiad.wmnet pooled
Apr 04 22:45:10 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1003.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:45:10 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1003.eqiad.wmnet pooled
Apr 04 22:45:10 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1002.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:45:10 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1002.eqiad.wmnet pooled
Apr 04 22:45:11 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1004.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:45:11 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1004.eqiad.wmnet pooled
Apr 04 22:50:29 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.024 s
Apr 04 22:50:29 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1002.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:50:29 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1002.eqiad.wmnet because of too many down!
Apr 04 22:50:30 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.002 s
Apr 04 22:50:30 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1001.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:50:30 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1001.eqiad.wmnet because of too many down!
Apr 04 22:50:30 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.002 s
Apr 04 22:50:30 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1004.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:50:30 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1004.eqiad.wmnet because of too many down!
Apr 04 22:50:30 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.021 s
Apr 04 22:50:30 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1003.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:50:30 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1003.eqiad.wmnet because of too many down!
Apr 04 22:50:44 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 5.001 s
Apr 04 22:50:45 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 5.001 s
Apr 04 22:50:45 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 5.002 s
Apr 04 22:50:45 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 5.002 s
Apr 04 22:50:54 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.002 s
Apr 04 22:50:55 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.001 s
Apr 04 22:50:55 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.001 s
Apr 04 22:50:55 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.001 s
Apr 04 22:51:09 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.384 s
Apr 04 22:51:09 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.361 s
Apr 04 22:51:09 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.361 s
Apr 04 22:51:09 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.357 s
Apr 04 22:51:16 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1004.eqiad.wmnet (enabled/down/pooled) down: Connection to the other side was lost in a non-clean fashion.
Apr 04 22:51:16 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] INFO: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.48.52:10042 lost.
Apr 04 22:51:16 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1003.eqiad.wmnet (enabled/down/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 22:51:16 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.32.23:10042 failed.
Apr 04 22:51:16 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1002.eqiad.wmnet (enabled/down/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 22:51:16 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.16.75:10042 failed.
Apr 04 22:51:16 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1001.eqiad.wmnet (enabled/down/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 22:51:16 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.0.121:10042 failed.
Apr 04 22:51:19 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.48.52:10042 failed.
Apr 04 22:51:19 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.32.23:10042 failed.
Apr 04 22:51:19 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.0.121:10042 failed.
Apr 04 22:51:19 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.003 s
Apr 04 22:51:19 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.003 s
Apr 04 22:51:19 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.003 s
Apr 04 22:51:19 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.004 s
Apr 04 22:51:19 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.16.75:10042 failed.
Apr 04 22:51:29 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1002.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:51:29 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1002.eqiad.wmnet pooled
Apr 04 22:51:29 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1001.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:51:29 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1001.eqiad.wmnet pooled
Apr 04 22:51:29 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1003.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:51:29 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1003.eqiad.wmnet pooled
Apr 04 22:51:29 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1004.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:51:29 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1004.eqiad.wmnet pooled
Apr 04 22:52:19 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.124 s
Apr 04 22:52:19 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1002.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:52:19 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1002.eqiad.wmnet because of too many down!
Apr 04 22:52:19 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.127 s
Apr 04 22:52:19 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1001.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:52:19 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1001.eqiad.wmnet because of too many down!
Apr 04 22:52:19 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.128 s
Apr 04 22:52:19 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1003.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:52:19 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1003.eqiad.wmnet because of too many down!
Apr 04 22:52:19 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.128 s
Apr 04 22:52:19 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1004.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:52:19 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1004.eqiad.wmnet because of too many down!
Apr 04 22:52:20 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1003.eqiad.wmnet (enabled/down/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 22:52:20 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.32.23:10042 failed.
Apr 04 22:52:29 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.001 s
Apr 04 22:52:29 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.001 s
Apr 04 22:52:29 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.002 s
Apr 04 22:52:29 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.001 s
Apr 04 22:52:39 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.001 s
Apr 04 22:52:39 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.001 s
Apr 04 22:52:39 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.001 s
Apr 04 22:52:39 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.001 s
Apr 04 22:52:49 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.001 s
Apr 04 22:52:49 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.001 s
Apr 04 22:52:49 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.001 s
Apr 04 22:52:49 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.001 s
Apr 04 22:52:56 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1003.eqiad.wmnet (enabled/down/pooled) down: Connection to the other side was lost in a non-clean fashion.
Apr 04 22:52:56 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] INFO: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.32.23:10042 lost.
Apr 04 22:52:56 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1002.eqiad.wmnet (enabled/down/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 22:52:56 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.16.75:10042 failed.
Apr 04 22:52:56 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1004.eqiad.wmnet (enabled/down/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 22:52:56 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.48.52:10042 failed.
Apr 04 22:52:56 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1001.eqiad.wmnet (enabled/down/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 22:52:56 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.0.121:10042 failed.
Apr 04 22:52:59 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.16.75:10042 failed.
Apr 04 22:52:59 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.001 s
Apr 04 22:52:59 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.000 s
Apr 04 22:52:59 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.001 s
Apr 04 22:52:59 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.001 s
Apr 04 22:52:59 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.0.121:10042 failed.
Apr 04 22:52:59 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.48.52:10042 failed.
Apr 04 22:52:59 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.32.23:10042 failed.
Apr 04 22:53:04 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.16.75:10042 failed.
Apr 04 22:53:09 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1001.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:53:09 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1001.eqiad.wmnet pooled
Apr 04 22:53:09 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1003.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:53:09 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1003.eqiad.wmnet pooled
Apr 04 22:53:09 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1004.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:53:09 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1004.eqiad.wmnet pooled
Apr 04 22:53:19 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1002.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:53:19 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1002.eqiad.wmnet pooled
Apr 04 22:53:25 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1004.eqiad.wmnet (enabled/up/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 22:53:25 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1004.eqiad.wmnet because of too many down!
Apr 04 22:53:25 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1004.eqiad.wmnet (enabled/partially up/pooled): Connection to 10.64.48.52:10042 failed.
Apr 04 22:53:25 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1001.eqiad.wmnet (enabled/up/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 22:53:25 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1001.eqiad.wmnet because of too many down!
Apr 04 22:53:25 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1001.eqiad.wmnet (enabled/partially up/pooled): Connection to 10.64.0.121:10042 failed.
Apr 04 22:53:28 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1004.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:53:28 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1004.eqiad.wmnet pooled
Apr 04 22:53:28 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1001.eqiad.wmnet (enabled/partially up/pooled): Connection to 10.64.0.121:10042 failed.
Apr 04 22:53:35 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1001.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:53:35 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1001.eqiad.wmnet pooled
Apr 04 22:53:45 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.001 s
Apr 04 22:53:45 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1002.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:53:45 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1002.eqiad.wmnet because of too many down!
Apr 04 22:53:45 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/up/pooled): Fetch failed, 5.002 s
Apr 04 22:53:45 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1004.eqiad.wmnet (enabled/up/pooled) down: Getting http://localhost/_info took longer than 5 seconds.
Apr 04 22:53:45 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1004.eqiad.wmnet because of too many down!
Apr 04 22:53:57 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1002.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 22:53:57 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1002.eqiad.wmnet pooled
Apr 04 22:54:03 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 8.037 s
Apr 04 22:54:05 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1001.eqiad.wmnet (enabled/up/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 22:54:05 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1001.eqiad.wmnet because of too many down!
Apr 04 22:54:05 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1001.eqiad.wmnet (enabled/partially up/pooled): Connection to 10.64.0.121:10042 failed.
Apr 04 22:54:07 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/up/pooled): Fetch failed, 0.001 s
Apr 04 22:54:07 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1002.eqiad.wmnet (enabled/up/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 22:54:07 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1002.eqiad.wmnet because of too many down!
Apr 04 22:54:08 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1001.eqiad.wmnet (enabled/partially up/pooled): Connection to 10.64.0.121:10042 failed.
Apr 04 22:54:10 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/up/pooled): Fetch failed, 30.001 s
Apr 04 22:54:10 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1003.eqiad.wmnet (enabled/up/pooled) down: User timeout caused connection failure.
Apr 04 22:54:10 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Could not depool server kubernetes1003.eqiad.wmnet because of too many down!
Apr 04 22:54:10 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 30.000 s
Apr 04 22:54:10 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance ProxyFetch reports server kubernetes1001.eqiad.wmnet (enabled/down/pooled) down: User timeout caused connection failure.
Apr 04 22:54:13 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.001 s
Apr 04 22:54:16 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.0.121:10042 failed.
Apr 04 22:54:17 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.001 s
Apr 04 22:54:24 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.347 s
Apr 04 22:54:24 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.347 s
Apr 04 22:54:24 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/partially up/pooled): Fetch failed, 0.360 s
Apr 04 22:54:26 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1003.eqiad.wmnet (enabled/down/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 22:54:26 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.32.23:10042 failed.
Apr 04 22:54:26 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1002.eqiad.wmnet (enabled/down/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 22:54:26 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.16.75:10042 failed.
Apr 04 22:54:26 lvs1006 pybal[15265]: [mathoid_10042] ERROR: Monitoring instance IdleConnection reports server kubernetes1004.eqiad.wmnet (enabled/down/pooled) down: Connection was refused by other side: 111: Connection refused.
Apr 04 22:54:26 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.48.52:10042 failed.
Apr 04 22:54:27 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.001 s
Apr 04 22:54:32 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.16.75:10042 failed.
Apr 04 22:54:32 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.32.23:10042 failed.
Apr 04 22:54:33 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.48.52:10042 failed.
Apr 04 22:54:40 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.0.121:10042 failed.
Apr 04 22:54:43 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.32.23:10042 failed.
Apr 04 22:54:44 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.16.75:10042 failed.
Apr 04 22:54:44 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.48.52:10042 failed.
Apr 04 22:55:04 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.030 s
Apr 04 22:55:04 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.030 s
Apr 04 22:55:04 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:55:07 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:55:08 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.32.23:10042 failed.
Apr 04 22:55:14 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.48.52:10042 failed.
Apr 04 22:55:17 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.16.75:10042 failed.
Apr 04 22:55:44 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:55:44 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:55:44 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:55:47 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:55:47 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.0.121:10042 failed.
Apr 04 22:56:21 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.32.23:10042 failed.
Apr 04 22:56:24 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.48.52:10042 failed.
Apr 04 22:56:24 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:56:24 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:56:24 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:56:27 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:56:52 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.16.75:10042 failed.
Apr 04 22:57:04 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.027 s
Apr 04 22:57:04 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.027 s
Apr 04 22:57:05 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.036 s
Apr 04 22:57:07 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.045 s
Apr 04 22:57:44 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:57:44 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:57:45 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:57:47 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:58:24 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:58:24 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:58:25 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:58:27 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:58:28 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.0.121:10042 failed.
Apr 04 22:59:01 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.48.52:10042 failed.
Apr 04 22:59:04 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:59:04 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:59:05 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:59:07 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 22:59:31 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.32.23:10042 failed.
Apr 04 22:59:49 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 34.772 s
Apr 04 22:59:49 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 34.773 s
Apr 04 22:59:49 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 34.633 s
Apr 04 22:59:49 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 32.768 s
Apr 04 23:00:29 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.176 s
Apr 04 23:00:29 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.176 s
Apr 04 23:00:29 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.172 s
Apr 04 23:00:29 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:01:04 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.16.75:10042 failed.
Apr 04 23:01:09 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:01:09 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:01:09 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:01:09 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:01:49 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:01:49 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:01:49 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:01:49 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:02:30 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:02:30 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:02:30 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:02:30 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.002 s
Apr 04 23:03:10 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:03:10 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:03:10 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:03:10 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:03:38 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.0.121:10042 failed.
Apr 04 23:03:50 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:03:50 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:03:50 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:03:50 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:04:25 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.32.23:10042 failed.
Apr 04 23:04:29 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.48.52:10042 failed.
Apr 04 23:04:30 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:04:30 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:04:30 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:04:30 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:05:10 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:05:10 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:05:10 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:05:10 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:05:50 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:05:50 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.000 s
Apr 04 23:05:50 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.000 s
Apr 04 23:05:50 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.000 s
Apr 04 23:06:19 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.16.75:10042 failed.
Apr 04 23:06:30 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.078 s
Apr 04 23:06:30 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.079 s
Apr 04 23:06:30 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.080 s
Apr 04 23:06:30 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.081 s
Apr 04 23:07:10 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:07:10 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:07:10 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:07:10 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:07:53 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:07:53 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:07:53 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.000 s
Apr 04 23:07:53 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.002 s
Apr 04 23:08:33 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:08:33 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:08:33 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:08:33 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 30.001 s
Apr 04 23:08:59 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 15.298 s
Apr 04 23:08:59 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 15.295 s
Apr 04 23:08:59 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 15.299 s
Apr 04 23:08:59 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 15.299 s
Apr 04 23:09:09 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.002 s
Apr 04 23:09:09 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.002 s
Apr 04 23:09:09 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.002 s
Apr 04 23:09:09 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.002 s
Apr 04 23:09:09 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.32.23:10042 failed.
Apr 04 23:09:19 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1003.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.001 s
Apr 04 23:09:19 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1002.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.001 s
Apr 04 23:09:19 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1001.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.001 s
Apr 04 23:09:19 lvs1006 pybal[15265]: [mathoid_10042 ProxyFetch] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Fetch failed, 0.002 s
Apr 04 23:09:23 lvs1006 pybal[15265]: [mathoid_10042 IdleConnection] WARN: kubernetes1004.eqiad.wmnet (enabled/down/pooled): Connection to 10.64.48.52:10042 failed.
Apr 04 23:10:00 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1001.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 23:10:00 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1001.eqiad.wmnet pooled
Apr 04 23:11:04 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1002.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 23:11:04 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1002.eqiad.wmnet pooled
Apr 04 23:13:38 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1004.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 23:13:38 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1004.eqiad.wmnet pooled
Apr 04 23:14:13 lvs1006 pybal[15265]: [mathoid_10042] INFO: Server kubernetes1003.eqiad.wmnet (enabled/partially up/pooled) is up
Apr 04 23:14:13 lvs1006 pybal[15265]: [mathoid_10042] INFO: Leaving previously pooled but down server kubernetes1003.eqiad.wmnet pooled
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL