Page MenuHomePhabricator

netflow hosts spamming /var/log
Closed, DuplicatePublic

Description

Investigating T249176 I've noticed that in /var/log both messages and user.log files are quite spammy with the same message repeated over and over:

Apr  1 22:01:59 netflow2001 fastnetmon: 1585778519 INFO  : Received ipfix options flowset id, which is not supported
Apr  1 22:01:59 netflow2001 fastnetmon: 1585778519 INFO  : We don't have a template for flowset_id: 512 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it!
Apr  1 22:01:59 netflow2001 fastnetmon: 1585778519 INFO  : Received ipfix options flowset id, which is not supported
Apr  1 22:01:59 netflow2001 fastnetmon: 1585778519 INFO  : We don't have a template for flowset_id: 513 but it's not an error if this message disappears in 5-10 seconds. We need some time to learn it!

And the flowset_id are always the same two: 512 and 513:

# egrep -o 'have a template for flowset_id: [0-9]+' messages.1 | sort | uniq -c
 272614 have a template for flowset_id: 512
 272621 have a template for flowset_id: 513

Looking at all hosts, here's the count:

$ sudo cumin 'A:netflow' 'grep -c "have a template for flowset_id" /var/log/messages'
5 hosts will be targeted:
netflow2001.codfw.wmnet,netflow1001.eqiad.wmnet,netflow5001.eqsin.wmnet,netflow3001.esams.wmnet,netflow4001.ulsfo.wmnet
Confirm to continue [y/n]? y
===== NODE GROUP =====
(1) netflow2001.codfw.wmnet
----- OUTPUT of 'grep -c "have a ...var/log/messages' -----
544785
===== NODE GROUP =====
(1) netflow4001.ulsfo.wmnet
----- OUTPUT of 'grep -c "have a ...var/log/messages' -----
135126
===== NODE GROUP =====
(1) netflow3001.esams.wmnet
----- OUTPUT of 'grep -c "have a ...var/log/messages' -----
270279
===== NODE GROUP =====
(1) netflow5001.eqsin.wmnet
----- OUTPUT of 'grep -c "have a ...var/log/messages' -----
135203
===== NODE GROUP =====
(1) netflow1001.eqiad.wmnet
----- OUTPUT of 'grep -c "have a ...var/log/messages' -----
544420