Page MenuHomePhabricator

asw1-eqsin vcp port flapping
Closed, ResolvedPublic

Description

The following is happening between 5 and 10 times per hour on asw1-eqsin:

%-: Member 0, interface vcp-255/0/24.32768 went down
%-VCCPD_PROTOCOL_ADJDOWN: Lost adjacency to 5800.bb29.bb60 on vcp-255/0/25.32768, 
%-CHASSISD_VCHASSIS_MEMBER_UPDATE_NOTICE: Membership update: Member 1->1, Mode Master->Master, 1M 0B, Master Unchanged, Members Unchanged
%-: self ID 0 self mode 0 master ch 0 master de 0 back ch 0 back de 0 memb ch 0
%-CHASSISD_VCHASSIS_MEMBER_UPDATE_NOTICE: Membership update: Member 1->1, Mode Master->Master, 1M 0B, Master Unchanged, Members Unchanged
%-: self ID 0 self mode 0 master ch 0 master de 0 back ch 0 back de 0 memb ch 0
%-: Member 0, interface vcp-255/0/24.32768 came up

Link flapping is between member0 (rack 603) port 24 and member1 (rack 605) port 25.
Probably needs one of the following (ordered by complexity):
1/ re-seat
2/ replace cable
3/ use different ports

Depending on the timeline of T187157 we could have remote-hands investigate the issue as well.

And probably worth skipping 1/ to optimize remote-hands time, especially as we're not going to know instantly if the issue is solved.

Related Objects

Event Timeline

ayounsi triaged this task as High priority.Apr 13 2018, 12:54 AM
ayounsi created this task.
Restricted Application added a project: Operations. · View Herald TranscriptApr 13 2018, 12:54 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
ayounsi mentioned this in Unknown Object (Task).Apr 17 2018, 8:13 PM
ayounsi mentioned this in Unknown Object (Task).Apr 17 2018, 8:22 PM

Mentioned in SAL (#wikimedia-operations) [2018-04-23T22:56:24Z] <XioNoX> disabling flapping VCP on asw1-eqsin - T192125

Disabled with:
ayounsi@asw1-eqsin# run request virtual-chassis vc-port set interface member 1 vcp-255/0/25 disable

Should be re-enabled with:
# run request virtual-chassis vc-port set interface member 1 vcp-255/0/25

ayounsi closed this task as Resolved.May 8 2018, 9:01 PM

Cable has been replaced by remote-hands.
vc-port re-enabled. Will reopen task if still flapping.