Page MenuHomePhabricator

Interface errors on cr1-eqiad:xe-3/2/1
Closed, ResolvedPublic

Description

Now that T263019 is done, traffic is flowing on that interface, and revealing CRC errors:
https://librenms.wikimedia.org/graphs/id=11615/type=port_errors/

Faulty cable or optic. Feel free to replace without draining traffic away. Ping me if any issue.

Thanks

Event Timeline

ayounsi triaged this task as High priority.Nov 10 2020, 6:46 PM
ayounsi created this task.
Restricted Application added a project: SRE. · View Herald TranscriptNov 10 2020, 6:46 PM
ayounsi mentioned this in Unknown Object (Task).Nov 10 2020, 6:52 PM
ayounsi closed this task as Resolved.Nov 16 2020, 5:09 PM
ayounsi claimed this task.

This is not showing errors anymore.

ayounsi reopened this task as Open.Nov 18 2020, 8:29 AM

They're back...

ayounsi reassigned this task from ayounsi to Cmjohnson.Nov 20 2020, 9:54 AM

Didn't see it was assigned to me, oops.

Mentioned in SAL (#wikimedia-operations) [2020-11-24T16:01:19Z] <cmjohnson1> replacing the sfp at cr1-eqiad xe-3/2/1 T267672

I replaced the sfp+ at cr1-eqiad xe-3/2/1 cleared the interface statistics on that port. Let's leave this open a few days to see if anything changes.

noticed this today

Statistics last cleared: 2020-11-24 16:04:52 UTC (1d 04:14 ago)

Input rate     : 1272 bps (1 pps)
Output rate    : 3536 bps (4 pps)
Input errors: 2, Output errors: 0
Active alarms  : None
Active defects : None
PCS statistics                      Seconds
  Bit errors                          8496
  Errored blocks                     29484

@XioNoX it appears a new fiber will need to be run. I will get to this next week. Do we need to schedule any downtime? The total time offline is 1 minute (long enough for me to swap out fiber on both ends).

No need to schedule anything. Thanks.

replaced the cable, gave it the same cable number, removed the old fiber. Cleared the interface statistics on cr1.

Cmjohnson closed this task as Resolved.Dec 18 2020, 5:34 PM

No new errors since I replace the fiber. Resolving this