Page MenuHomePhabricator

cr2-magru errors on xe-0/1/0 (EdgeUno Transit)
Closed, ResolvedPublic

Description

For the past few weeks we have begun to see incrementing inbound errors on cr2-magru xe-0/1/0, connected to EdgeUno transit circuit there.

image.png (583×1 px, 81 KB)

The problem could be the edge-uno side, but experience tells me the carrier will request us to check our end before taking any action. So I think the first thing to do is arrange remote hands in magru to clean the fibre patch from the panel to our router, and replace the optic in the port. Optic type is 10G-Base-LR.

The port is the one circled below, labelled '4' on the front:

max204_xe_0_1_0.png (200×1 px, 407 KB)

We can have a look at it after that and if it persists raise an issue with edgeuno to swap the module their side / clean fibers. Please loop in netops so we can drain this circuit before the work takes place.

Right now the error-rate is low and we are doing a decent level of traffic so I am not putting the circuit of of service, but I'll keep an eye on it.

Event Timeline

cmooney triaged this task as High priority.

@cmooney,

Can we have the carrier check their end first, since we'll incur hourly billing for our check?

@cmooney,

Can we have the carrier check their end first, since we'll incur hourly billing for our check?

I usually prefer to avoid a Mexican standoff like that, but this is Brazil so maybe. I'll ask on the portal.

Mentioned in SAL (#wikimedia-operations) [2025-02-24T12:17:31Z] <XioNoX> disable BGP to edgeuno in magru - T387006

Mentioned in SAL (#wikimedia-operations) [2025-02-24T13:52:18Z] <XioNoX> re-enable BGP to edgeuno in magru - T387006

Our datacenter engineering team has concluded the on-site activity, and no problems were found on our side. Could you please confirm if this has improved the situation and ceased the errors on your side?
We performed a fiber cleaning and checked all the physical components.

I'm re-enabling the link to see if it has improved.

ayounsi claimed this task.

No more errors.