Inbound errors on interface lsw1-d4-eqiad:ethernet-1/19 (an-worker1230 {#5330})
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	phaultfinder
	Sat, Apr 18, 1:39 AM

Description

dashboard: https://grafana.wikimedia.org/d/5p97dAASz/queue-and-error-stats-by-network-device?var-site=eqiad+prometheus%2Fops&var-device=lsw1-d4-eqiad&var-interface=ethernet-1%2F19
description:
runbook: https://wikitech.wikimedia.org/wiki/Network_monitoring#Inbound/outbound_interface_errors
summary: Inbound errors on interface lsw1-d4-eqiad:ethernet-1/19 (an-worker1230 {#5330})

dashboard: https://grafana.wikimedia.org/d/5p97dAASz/queue-and-error-stats-by-network-device?var-site=eqiad+prometheus%2Fops&var-device=lsw1-d4-eqiad&var-interface=ethernet-1%2F19
description:
runbook: https://wikitech.wikimedia.org/wiki/Network_monitoring#Inbound/outbound_interface_errors
summary: Inbound errors on interface lsw1-d4-eqiad:ethernet-1/19 (an-worker1230 {#5330})
alertname: InboundInterfaceErrors
instance: lsw1-d4-eqiad:9804
interface_description: an-worker1230 {#5330}
interface_name: ethernet-1/19
job: gnmi
prometheus: ops
scope: network
severity: task
site: eqiad
source: prometheus
team: dcops
Source

@BTullis Hey Ben, we can replace this cable in order to clear up this error. Can you please let us know when the best time to swap it would be.

Hi @VRiley-WMF - Yes, please feel free to swap this cable any time. If it's a short outage, we can just let Hadoop deal with the network errors.

If you think that it's going to be a more sustained thing, more than a few minutes, then we can make a patch to exclude this host from the cluster.

This cable has been swapped out.

Great! Did it fix the issue?

It'll take time in order to see if it fixes it. We'll see if it doesn't throw the error again