lvs1015 interface errors
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	ayounsi
	Jan 18 2021, 8:17 AM

Description

asw2-b-eqiad:xe-2/0/3, lvs1015:enp4s0f1

Similar to T271087 so it could quickly lead to the interface failing completely.

Please sync up with traffic to depool the host.

ayounsi triaged this task as High priority.Jan 18 2021, 8:17 AM

ayounsi created this task.

ayounsi updated the task description. (Show Details)

Vgutierrez subscribed.

@Cmjohnson - let me know when you're ready to deal with this, and we'll stop service on the node and fail things over to lvs1016.

@BBlack Can you schedule this for this afternoon?

@Cmjohnson Yes, just let me know a timeframe and we'll get it ready

@BBlack Okay, 2 hours from now. 1300 EST

@Cmjohnson - we'll have it ready then.

Mentioned in SAL (#wikimedia-operations) [2021-01-20T17:54:27Z] <bblack> lvs1015: stopping pybal with puppet disabled for T272258

Mentioned in SAL (#wikimedia-operations) [2021-01-20T18:01:34Z] <bblack> lvs1015 - shutdown for T272258

@BBlack I swapped both optics on lvs1015 and b2 xe-2/0/3

resolving this task, if the issue persists please re-open and ping me. Thanks!

Mentioned in SAL (#wikimedia-operations) [2021-01-20T18:29:41Z] <bblack> lvs1015: re-enabling puppet + pybal - T272258

All appears healthy now and downtimes are removed, and librenms isn't showing those errors on the interface anymore, either. Thanks!