Page MenuHomePhabricator

Upgrade Fastnetmon to 1.1.7
Closed, ResolvedPublic

Description

From https://github.com/pavel-odintsov/fastnetmon/releases/tag/v1.1.7

The version we're running is working fine, but 1.1.6 brings interesting features (mostly GoBGP, API and CLI).

EDIT: 1.1.7 brings improvements of the IPFIX processing.

I don't know yet the steps needed to package it for our infra

Edit: upstream provides binary packages, so the upgrade should be very easy: https://community-downloads.fastnetmon.com/releases/1.1.7/debian/10/

Event Timeline

ayounsi triaged this task as Low priority.Jul 3 2020, 11:01 AM
ayounsi created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 3 2020, 11:01 AM
ayounsi updated the task description. (Show Details)Jul 3 2020, 12:52 PM

Steps to update the existing package on deneb:

apt-get source fastnetmon
cd fastnetmon- 1.1.4-1~deb10u1
uupdate ../1.1.6.orig.tar.xz
(some patches might be merged, drop those from debian/patches/series)
build the new deb
ayounsi renamed this task from Upgrade Fastnetmon to 1.1.6 to Upgrade Fastnetmon to 1.1.7.Sep 14 2020, 9:25 AM
ayounsi updated the task description. (Show Details)

I have upgraded the existing 1.1.4 package to 1.1.7, I needed to fix up all patches for 1.1.7 (except one for luajit, which was obsoleted by upstream dropping support for luajit in 1.1.5 "Disabled LUA Jit support due to very low demand". I haven't uploaded the new package to apt.wikimedia.org, but you can find the updated binary on deneb.codfw.wmnet in /var/cache/pbuilder/result/buster-amd64/fastnetmon_1.1.7-1~wmf1_amd64.deb

I've made a quick upgrade and installability test in a Buster nspawn container, but I'm not familiar enough with fastnetmon to do a practical test on netflow*. Let me know if the new versions works for you and I'll upload to apt.wikimedia.org

Maybe I can get Debian updated to 1.1.7 in unstable as well, we'll see.

Mentioned in SAL (#wikimedia-operations) [2020-09-21T07:05:37Z] <XioNoX> upgrade FNM to 1.1.7 in ulsfo - T257035

Thanks @MoritzMuehlenhoff I installed in on netflow4001 and it is working fine.

Surprisingly though one new CLI tool fastnetmon_api_client is missing from the DEB. Was there any issues during the build?

@CDanis, let me know if you agree with the following:
There are 2 relevant commits to the config file since the version we're running:
https://github.com/pavel-odintsov/fastnetmon/commit/91599d87da5c21bdd02a074795b5918d898c477d
https://github.com/pavel-odintsov/fastnetmon/commit/9bb0cd085be355283547e2ad8ae588745780935e

  • enable_connection_tracking = on

If I understand the code correctly this would require to turn on process_outgoing_traffic which is a whole new can of worms

  • average_calculation_time_for_subnets = 5

This is to align it with average_calculation_time = 5. We're already aligned on 20s, see https://github.com/wikimedia/puppet/commit/601c72e9ac68dfd7072b8b9a2cef2ed8786ec7f0

  • speed_calculation_delay = 1

Even though I don't understand what it does, as it's a new configuration option, I suggest we add it to the config.

+1 for leaving the default speed_calculation_delay = 1

+1 for setting average_calculation_time_for_subnets to the same as our overall average_calculation_time

Just to make sure I understand, you were thinking we don't enable_connection_tracking? I agree with that, it rather expensive on CPU.

Change 629119 had a related patch set uploaded (by Ayounsi; owner: Ayounsi):
[operations/puppet@production] FNM: add new default speed_calculation_delay

https://gerrit.wikimedia.org/r/629119

Mentioned in SAL (#wikimedia-operations) [2020-09-22T13:59:15Z] <XioNoX> add fastnetmon_1.1.7 to buster-wikimedia repo - T257035

Just to make sure I understand, you were thinking we don't enable_connection_tracking? I agree with that, it rather expensive on CPU.

Correct.

Surprisingly though one new CLI tool fastnetmon_api_client is missing from the DEB. Was there any issues during the build?

We can go without the API client for now, as the package has dependencies issues.

Mentioned in SAL (#wikimedia-operations) [2020-09-22T14:06:37Z] <XioNoX> upgrade FNM on netflow3001 - T257035

Mentioned in SAL (#wikimedia-operations) [2020-09-22T14:09:02Z] <XioNoX> upgrade FNM on netflow1001 - T257035

Mentioned in SAL (#wikimedia-operations) [2020-09-22T14:10:50Z] <XioNoX> upgrade FNM on netflow5001 - T257035

Mentioned in SAL (#wikimedia-operations) [2020-09-22T14:15:10Z] <XioNoX> upgrade FNM on netflow2001 - T257035

Change 629119 merged by Ayounsi:
[operations/puppet@production] FNM: add new default speed_calculation_delay

https://gerrit.wikimedia.org/r/629119

ayounsi closed this task as Resolved.Sep 22 2020, 2:19 PM
ayounsi claimed this task.

All done! Thanks a lot and we can revisit when we need the API client.