Page MenuHomePhabricator

lvs1016 interface down
Closed, ResolvedPublic



Usual optics/patch replacement dance :)

Please sync up with the traffic team so they depool the host first.

Related Objects

Event Timeline

ayounsi triaged this task as Medium priority.Jan 4 2021, 9:53 AM
ayounsi created this task.

@BBlack can we schedule this on Monday? 1530/1600UTC

The interface seems having trouble at the moment, we have some icinga alerts about pybal not reaching row-a hosts:

elukey@asw2-a-eqiad> show interfaces descriptions | match lvs1016    
xe-4/0/7        up    down lvs1016:enp4s0f1 {#3917}

Also some updown event logged in the syslog of the switch..

ayounsi raised the priority of this task from Medium to High.Jan 13 2021, 2:47 PM

The lvs is a secondary so not taking traffic, added a day of downtime :)

ayounsi renamed this task from Interface errors on asw2-a-eqiad:xe-4/0/7 (lvs1016) to lvs1016 interface down.Jan 13 2021, 4:22 PM

@Cmjohnson - Please do it at your earliest convenience. It's not in the flow of live traffic and doesn't need any "depool" AFAIK (but it is problematic that we don't have it as a reliable backup option!).

I added a week of downtime, the alarm popped up again, remember it if the issue gets solved sooner!

@elukey @BBlack swapped both the optics at the switch on a4 and on the server. It appears that the server side optic was the issue, the link is backup

xe-4/0/7 up up lvs1016:enp4s0f1 {#3917}