In T429116#12018393, @Jclark-ctr wrote:

I believe this is the link causing the error: https://netbox.wikimedia.org/dcim/cables/5727/

@cmooney @ayounsi — can you confirm? If so, would you like us to schedule replacement of the optic or the fiber?

Hey @Jclark-ctr that is correct, it seems it's been bad for a while:

https://grafana-rw.wikimedia.org/d/5p97dAASz/queue-and-error-stats-by-network-device?orgId=1&from=now-24h&to=now&timezone=utc&var-site=000000006&var-device=cr1-eqiad:9804&var-interface=xe-3%2F2%2F3&refresh=30s

The rate is quite low but it's got more consistent of late:

I think probably the easiest first step to do is to swap the optic in asw2-b2-eqiad port 45. Should be a 10GBase-LR SFP+ if we have a spare. You can just pull the fibres the AE interface will keep working off the other three links. Though ping me on irc when doing it as it will likely alert so I will downtime the host or at least advice on call.

If that doesn't do it we can try the optic in the CR, and finally the patch cable if it's not those.

Verified cables seated properly and checked serials yesterday Errors seemed to of cleared will monitor for a week before closing

@cmooney, this link continues to experience errors. If you're available tomorrow and would like me to swap the optics, I can ping you.

In T429116#12041312, @Jclark-ctr wrote:

@cmooney, this link continues to experience errors. If you're available tomorrow and would like me to swap the optics, I can ping you.

Hey. Yep happy to look at this one whenever you are on site just ping me. Let's start by swapping the asw2-b2-eqiad one.

I swapped the optic on Switch B2 and also replaced the cable between the core router and the ToR switch.

New Cable ID: G2210253241007206.

I decided to swap the cable after noticing that the interface errors had increased quite a bit over the past few days. Since the new switch was recently installed and the cabling would of been moved during that work, I thought it was possible the cable had been affected and was contributing to the increase in errors.