Page MenuHomePhabricator

Servers exposing incorrect LLDP info
Open, LowPublic

Description

There are two different scenarios:

Broadcom system name

ayounsi@asw1-eqsin# run show lldp neighbors 
Local Interface    Parent Interface    Chassis Id          Port info          System Name
[...]
xe-0/0/9           -                   d0:94:66:81:90:90   NIC 10Gb SFP+ DA   Broadcom Adv. Dual 10G SFP+ Ethernet fw_version:AFW_214.0.200.0
[...]

xe-0/0/9 (ganeti5001) is not reporting its System and Port name properly.

More details:

ayounsi@asw1-eqsin# run show lldp neighbors interface xe-0/0/9 
LLDP Neighbor Information:
Local Information:
Index: 3830 Time to live: 120 Time mark: Thu Apr 16 10:37:33 2020 Age: 22 secs 
Local Interface    : xe-0/0/9
Parent Interface   : -
Local Port ID      : 649
Ageout Count      : 0

Neighbour Information:
Chassis type       : Mac address
Chassis ID         : d0:94:66:81:90:90
Port type          : Mac address
Port ID            : d0:94:66:81:90:90
Port description   : NIC 10Gb SFP+ DA
System name        : Broadcom Adv. Dual 10G SFP+ Ethernet fw_version:AFW_214.0.200.0
  
System Description : fw_version:AFW_214.0.200.0


System capabilities 
        Supported: Station Only 
        Enabled  : Station Only 

Management address 
        Address Type      : 802 MAC(6)
        Address           : d0:94:66:81:90:90
        Interface Number  : 0
        Interface Subtype : Unknown(1)

As of today there are 90 hosts with that issue, querying LibreNMS:

SELECT ifAlias FROM librenms.links, librenms.ports where remote_hostname like "%Broadcom%" and links.local_port_id = ports.port_id;
"cloudvirt1023 eth1"
"cloudcephosd1002 eth1"
"cloudcephosd1003 eth1"
lvs2010:enp175s0f1d1
lvs2009:enp175s0f0
lvs2010:enp175s0f0
lvs2009:enp59s0f1d1
lvs2010:enp59s0f1d1
lvs2009:enp175s0f1d1
cloudvirt1024-eth1
"cloudcephmon1001 eth0"
"cloudcephmon1001 eth1"
"cloudcephmon1002 eth1"
"cloudcephmon1003 eth1"
"cloudcephosd1001 eth1"
"cloudcephmon1003 eth0"
"cloudcephosd1001 eth0"
"ganeti4003:ens3f0np0 {#1053}"
lvs2008enp175s0f0
lvs2008enp59s0f1d1
lvs2008enp175s0f1d1
lvs2007:enp175s0f0
lvs2007:enp59s0f1d1
lvs2007:enp175s0f1d1.
"lvs3007:enp175s0f0 {#20050}"
"cp3056:enp59s0f0 {#20031}"
ms-be1050
ms-be1044
logstash1026
cp1077
elastic2058
elastic2060
elastic2050
cp2042
lvs2010:enp59s0f0
"bast3004:enp175s0f0 {#20052}"
"cp3061:enp59s0f0 {#20053}"
"cp3065:enp59s0f0 {#20057}"
kafka-main1001
ms-be1048
logstash1029
dns1002
cp1088
cp2031
elastic2038
backup2002
kafka-main2004
backup2001
mc-gp2003
"cp3050:enp59s0f0 {#20023}"
"ganeti3002:enp175s0f0 {#20028}"
"ganeti5003:ens1f0np0 {#}"
ms-be1049
cp1084
cloudvirt1024-eth0
"cloudcephosd1003 eth0"
cp1090
mc-gp1003
ms-be2048
cp2035
cp2034
elastic2056
elastic2054
"cp3064:enp59s0f0 {#20056}"
"lvs3006:enp175s0f0 {#20027}"
"ganeti3003:enp175s0f0 {#20051}"
cp1083
dbprov1002
ms-be1047
cp1079
ms-be1045
cp1089
backup1001
elastic2059
ms-be2049
elastic2045
cp2036
elastic2057
dbprov2002
kafka-main2002
elastic2043
elastic2044
ms-be2044
kafka-main2001
cp2027
authdns2001
ms-be2050
kafka-main2005
elastic2051
"ganeti4001:ens3f0np0 {#1052}"

MAC Port info (no System name)

ayounsi@asw1-eqsin# run show lldp neighbors 
Local Interface    Parent Interface    Chassis Id          Port info          System Name
[...]
xe-1/0/8           -                   f4:e9:d4:d0:21:60   f4:e9:d4:d0:21:60
[...]

xe-1/0/8 is cp5002, but show:

ayounsi@asw1-eqsin# run show lldp neighbors interface xe-1/0/8 
LLDP Neighbor Information:
Local Information:
Index: 3838 Time to live: 120 Time mark: Thu Apr 16 10:38:28 2020 Age: 9 secs 
Local Interface    : xe-1/0/8
Parent Interface   : -
Local Port ID      : 559
Ageout Count      : 0

Neighbour Information:
Chassis type       : Mac address
Chassis ID         : f4:e9:d4:d0:21:60
Port type          : Mac address
Port ID            : f4:e9:d4:d0:21:60

Organization Info
       OUI      : 0.1b.21
       Subtype  : 2
       Info     : 020A0000000000030000000004110000C0000001000032320000000000000306060000C0000802080A0000C0008906001B2108 
       Index    : 1

Event Timeline

ayounsi created this task.

So for hosts that say something like: Port description : NIC 10Gb SFP+ DA one theory is that the NIC has an embedded LLDP daemon that prevents the host one to work properly.
There are some flags mentioned https://github.com/torvalds/linux/commit/7d63818a35851cf00867248d5ab50a8fe8df5943 but not sure if they can be queried from Linux.

This BCM NIC/driver doesn't seem to support priv-flags via ethtool (where I believe that should show up):

cmooney@lvs2010:~$ sudo /sbin/ethtool -i ens3f1np1
driver: bnxt_en
version: 1.9.2
firmware-version: 214.0.253.1/pkg 21.40.25.31
expansion-rom-version: 
bus-info: 0000:af:00.1
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: no
supports-priv-flags: no
cmooney@lvs2010:~$ sudo /sbin/ethtool --show-priv-flags ens3f1np1
No private flags defined

This VMware post does seem to back up your hypothesis though:

https://communities.vmware.com/t5/ESXi-Discussions/LLDP-Broadcom-10-25g/td-p/2270395

This doc from Broadcom suggests it might be possible to disable it using a "HII (Human Interface Infrastrucutre)" menu at system boot time? See section 9 / page 169:

https://docs.broadcom.com/doc/netxtreme-e-user-guide