Page MenuHomePhabricator

Make bpfcc-tools available fleet-wide
Open, MediumPublic

Description

I built a bpfcc backport for stretch for https://jira.mariadb.org/browse/MDEV-22809, which provides bpfcc-memleak (among many other tools). The same tool was also used in T260281, so I'm planning to install it on all stretch and buster hosts (the latter ships it natively, no backport needed).

It needs the kernel headers and debug packages, which we don't install by default (and the latter are insanely big), systemtap also needs them. I think installing them by default will do more harm than good (they are not used frequently, but would use disproportionate disk space especially on smaller VMs), so I'm thinking of of simply installing a small wrapper like "wmf-kernel-tracing install|cleanup" which installs/removes the necessary kernel packages for the current kernel (and allows to clean up after analysis).

Event Timeline

Thanks for opening this!

When it comes to systemtap, user-space tracing requires the linux-headers package for the currently running kernel, plus the debug symbols for whatever software is under scrutiny (eg: coreutils-dbgsym) . Kernel-level tracing also requires the kernel debug symbols, which as mentioned in this ticket are huge. The script stap-prep, shipped by the systemtap package, takes care of installing all the required packages for kernel tracing (kernel headers and kernel debug symbols), so it might be useful to use it for wmf-kernel-tracing install, or just as an inspiration maybe.

CDanis triaged this task as Medium priority.Aug 25 2020, 1:06 PM
CDanis added a subscriber: wkandek.

Thanks for opening this! Really happy to see it (and was also talking to @wkandek just yesterday about making bpfcc generally available in the fleet).

+1 to the wrapper idea. In my case what I did was install the packages, then lazily reimage the machine when done. A cleanup wrapper would be good to have.

Maybe the cleanup script should also prompt to uninstall any debug symbol packages? dpkg -l | awk '$1 ~ /ii/ && $2 ~ /-dbg/ { print $2 }' ought to make a list.

Change 622190 had a related patch set uploaded (by CDanis; owner: CDanis):
[operations/puppet@production] package_builder: add support for 'sloppy' backports

https://gerrit.wikimedia.org/r/622190

Change 622190 abandoned by CDanis:
[operations/puppet@production] package_builder: add support for 'sloppy' backports

Reason:

https://gerrit.wikimedia.org/r/622190