Can easily imagine that this isn't possible, but asking anyway.
Was trying to track down this peak in ingress https://librenms.wikimedia.org/graphs/to=1582678200/id=8209/type=port_bits/from=1582656600/ but it's hard to find in netflow when all you know is "entered via this link on this core router". Would be cool if I could filter by cr2-codfw traffic.
It's possible by adding the keys in_iface, out_iface and export_proto_sysid to modules/pmacct/templates/nfacctd.conf.erb
See: https://github.com/pmacct/pmacct/blob/master/CONFIG-KEYS#L53
The difficulty is that it exports the interfaces SNMP ID, and not the interface names themselves. So somewhere in Analytics-land, we then have to maintain a mapping of source:snmpid -> hostname (optional):portname.
This is usually done via SNMP, but could also be done by querying LibreNMS.
If done via SNMP, we can also fed it bgp_neighbors_file to know which hosts to query.
Having the sysid would also be useful to filter by sites/routers, mapping it to their hostnames would make it easier to use.
Note that also means having 3 more (non-PII) dimensions to store Druid, and would thus required Analytics's ok and help.
About this specific peak, we could also configure pmacct to send BGP events to Kafka using BMP, see https://github.com/pmacct/pmacct/blob/master/QUICKSTART#L2230
Change 577298 had a related patch set uploaded (by Ayounsi; owner: Ayounsi):
[operations/puppet@production] Nfacct, add export_proto_sysid
Change 577298 merged by Ayounsi:
[operations/puppet@production] Nfacct, add export_proto_sysid
Change 577305 had a related patch set uploaded (by Ayounsi; owner: Ayounsi):
[operations/puppet@production] Nfacct, export proper field
Change 577305 merged by Ayounsi:
[operations/puppet@production] Nfacct, export proper field
The new field is being exported properly to Kafka, see: "peer_ip_src": "103.102.166.129"
In Turnilo it would be convenient to rename it "Exporter IP" if possible.
Thanks!
If the cardinality of the three new dimensions are not too big we could definitely add them to Druid.
One note - adding the new info to pmacct, and hence to kafka messages, will be handled transparently up to Hive ingestion (Druid indexation happens right after). Since the Analytics team is experimenting with Presto (https://wikitech.wikimedia.org/wiki/Analytics/Systems/Presto#Usage_on_analytics_cluster), we could think about using it to create dashboards with Superset to consult (Presto would fetch data directly from Hive, but way quicker than regular tools). We are working with Superset upstream on this, I hope to have Superset working with Presto very soon.
Let's proceed with peer_ip_src for now, it's only ~15 different possible values (IPs) max.
It's already being exported by pmacct, so now we would only need it in the existing Turnilo dashboard.
Change 585377 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] Add peer_ip_src dimension for netflow in Druid and Turnilo
Change 585377 merged by Elukey:
[operations/puppet@production] Add peer_ip_src dimension for netflow in Druid and Turnilo
Mentioned in SAL (#wikimedia-analytics) [2020-04-02T08:23:30Z] <elukey> kill/restart netflow realtime druid indexation with a new dimension (peer_ip_src) - T246186
Change 585432 had a related patch set uploaded (by Elukey; owner: Elukey):
[analytics/refinery@master] Update Netflow's realtime indexation druid spec with a new dimension
Change 585432 merged by Elukey:
[analytics/refinery@master] Update Netflow's realtime indexation druid spec with a new dimension