Page MenuHomePhabricator

rsyslog-kubernetes missing in buster-wikimedia
Open, MediumPublic

Description

While working on T245272, it came up that rsyslog-kubernetes is not in buster-wikimedia yet. This is the current status for rsyslog:

elukey@apt1001:/srv/wikimedia$ sudo reprepro lsbycomponent rsyslog
rsyslog | 8.1901.0-1~bpo8+wmf1 |  jessie-wikimedia |              main | amd64, source
rsyslog | 8.1901.0-1~bpo9+wmf1 | stretch-wikimedia |              main | amd64, source
rsyslog |   8.2008.0-1~bpo10+1 |  buster-wikimedia | component/rsyslog | amd64, source

We don't have yet a buster-wikimedia branch in our rsyslog gerrit repo, so we'd need to figure out what to do now :)

After a chat with Filippo, IIUC 8.2008.0-1 is used only on centrallog nodes (hence the component - see T259780) but we might want to use 8.19011 provided on Buster and add the custom bits for rsyslog-kubernetes, uploading all to main. The alternative would be to modify 8.2008.0 and use the component instead (so adding rsyslog-kubernetes to it).

Any preference? :)

Long term https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=911299 will save us from this headache (hopefully).

Event Timeline

After a chat with Filippo, IIUC 8.2008.0-1 is used only on centrallog nodes (hence the component) but we might want to use 8.19011 provided on Buster and add the custom bits for rsyslog-kubernetes, uploading all to main. The alternative would be to modify 8.2008.0 and use the component instead (so adding rsyslog-kubernetes to it).

Any preference? :)

I have no idea why 8.2008.0-1 is used on centrallog , but if it is compatible with 8.1901 and can have kubernetes support, I 'd just upload it to main.

I am a little confused from the debian/stretch-wikimedia branch of the rsyslog repo, because the debian changelog seems to mention 8.38.0-1~bpo9+1wmf1 (it corresponds to a commit from Filippo to enable mmkubernetes).

I am a little confused from the debian/stretch-wikimedia branch of the rsyslog repo, because the debian changelog seems to mention 8.38.0-1~bpo9+1wmf1 (it corresponds to a commit from Filippo to enable mmkubernetes).

Self answering: I tried apt-get source rsyslog and its changelog mentions:

rsyslog (8.1901.0-1~bpo9+wmf1) stretch-backports; urgency=medium

  * Enable mmkubernetes (build depends on libcurl and liblognorm) and build
    rsyslog-kubernetes

 -- Filippo Giunchedi <filippo@wikimedia.org>  Mon, 22 Oct 2018 14:40:32 +0200

rsyslog (8.1901.0-1~bpo9+1) stretch-backports; urgency=medium

So I guess that we took the rsyslog in stretch-backports and applied Filippo's patch on top, but it wasn't saved in our repository. I would do the same for Buster as well, seems the easiest:

  1. get the source of https://packages.debian.org/buster/rsyslog
  2. Apply Filippo's patch
  3. upload to buster wikimedia with +wmf1

I am a little confused from the debian/stretch-wikimedia branch of the rsyslog repo, because the debian changelog seems to mention 8.38.0-1~bpo9+1wmf1 (it corresponds to a commit from Filippo to enable mmkubernetes).

Self answering: I tried apt-get source rsyslog and its changelog mentions:

rsyslog (8.1901.0-1~bpo9+wmf1) stretch-backports; urgency=medium

  * Enable mmkubernetes (build depends on libcurl and liblognorm) and build
    rsyslog-kubernetes

 -- Filippo Giunchedi <filippo@wikimedia.org>  Mon, 22 Oct 2018 14:40:32 +0200

rsyslog (8.1901.0-1~bpo9+1) stretch-backports; urgency=medium

So I guess that we took the rsyslog in stretch-backports and applied Filippo's patch on top, but it wasn't saved in our repository. I would do the same for Buster as well, seems the easiest:

  1. get the source of https://packages.debian.org/buster/rsyslog
  2. Apply Filippo's patch
  3. upload to buster wikimedia with +wmf1

+1

It probably makes sense to figure out what to do with the repo then though. archive it? sync it?

After a chat with Filippo, IIUC 8.2008.0-1 is used only on centrallog nodes (hence the component) but we might want to use 8.19011 provided on Buster and add the custom bits for rsyslog-kubernetes, uploading all to main. The alternative would be to modify 8.2008.0 and use the component instead (so adding rsyslog-kubernetes to it).

Any preference? :)

I have no idea why 8.2008.0-1 is used on centrallog , but if it is compatible with 8.1901 and can have kubernetes support, I 'd just upload it to main.

I've tried it on centrallog at upstream' suggestion to see if it'd fix T199406 (it doesn't) and T259780, however it was decided to leave the rest of the fleet unchanged unless we have to (and thus needed to upgrade the fleet). The package needed no changes though, as it was a straight backport from the version in testing (now at 8.2102.0).

I am a little confused from the debian/stretch-wikimedia branch of the rsyslog repo, because the debian changelog seems to mention 8.38.0-1~bpo9+1wmf1 (it corresponds to a commit from Filippo to enable mmkubernetes).

Indeed, it looks like we've upgraded rsyslog fleetwide 8.1901.0 but never committed the change to operations/debs/rsyslog

I am a little confused from the debian/stretch-wikimedia branch of the rsyslog repo, because the debian changelog seems to mention 8.38.0-1~bpo9+1wmf1 (it corresponds to a commit from Filippo to enable mmkubernetes).

Self answering: I tried apt-get source rsyslog and its changelog mentions:

rsyslog (8.1901.0-1~bpo9+wmf1) stretch-backports; urgency=medium

  * Enable mmkubernetes (build depends on libcurl and liblognorm) and build
    rsyslog-kubernetes

 -- Filippo Giunchedi <filippo@wikimedia.org>  Mon, 22 Oct 2018 14:40:32 +0200

rsyslog (8.1901.0-1~bpo9+1) stretch-backports; urgency=medium

So I guess that we took the rsyslog in stretch-backports and applied Filippo's patch on top, but it wasn't saved in our repository. I would do the same for Buster as well, seems the easiest:

  1. get the source of https://packages.debian.org/buster/rsyslog
  2. Apply Filippo's patch
  3. upload to buster wikimedia with +wmf1

+1

It probably makes sense to figure out what to do with the repo then though. archive it? sync it?

I'm +1 on keeping the repo since we have to carry the patch anyways. On which version to enable mmkubernetes I'm not sure, I see the following solutions though:

  1. Import 8.2102.0 in operations/debs/rsyslog (to have a version likely shipped in a Debian release) and enable mmkubernetes, upload to component/rsyslog, then enable the component on kubernetes hosts as well.
  2. Enable mmkubernetes on 8.2008.0 and upload to component/rsyslog, then enable the component on kubernetes hosts as well.
  3. Enable mmkubernetes on 8.1901.0 and upload to main in buster-wikimedia, and upgrade the fleet.

I think overall I prefer solution 1, and can of course assist with its implementation.

This is what I did on deneb before reading Filippo's answer :)

  • apt-get source rsyslog -t buster
  • applied Filippo's patch manually (git diff HEAD~1 HEAD on the debian/stretch-wikimedia branch of the rsyslog gerrit repo)
  • DIST=buster pdebuild (all tests passed)
elukey@deneb:~/rsyslog-8.1901.0$ ls /var/cache/pbuilder/result/buster-amd64/rsyslog*8.1901.0-1+wmf1*  | grep kubernetes
/var/cache/pbuilder/result/buster-amd64/rsyslog-kubernetes_8.1901.0-1+wmf1_amd64.deb
/var/cache/pbuilder/result/buster-amd64/rsyslog-kubernetes-dbgsym_8.1901.0-1+wmf1_amd64.deb

Update after a chat with Filippo:

  • a fleetwide update of rsyslog is painful, we can avoid it with the component solution (extremely wise point)
  • update the rsyslog repo seems good - master branch to version 8.2102 (like centrallog / bulleye / unstable) and then custom buster-wikimedia branch with Filippo's patch

I have updated the operations/debs/rsyslog from salsa.debian.org, now it contains 8.2102.0. I then tried something simple:

  • created a local branch debian/buster-wikimedia from master
  • applied Filippo's patch to create 8.2102.0+wmf1
  • ran DIST=buster pdebuild (relying on the .orig file from apt-get source rsyslog -t unstable)

It failed, due to debhelper-compact=13, so I lowered it down in control and kicked off the build again. A new error came up:

The following packages have unmet dependencies:
pbuilder-satisfydepends-dummy : Depends: librelp-dev (>= 1.4.0) but it is not going to be installed

Filippo must have dealt with this when creating 8.2008.0-1~bpo10+1 for buster-backports, since I see:

elukey@centrallog1001:~$ apt-cache policy librelp-dev
librelp-dev:
  Installed: (none)
  Candidate: 1.7.0-1~bpo10+1
  Version table:
     1.7.0-1~bpo10+1 1001
       1001 http://apt.wikimedia.org/wikimedia buster-wikimedia/component/rsyslog amd64 Packages
     1.3.0-1 500
        500 http://mirrors.wikimedia.org/debian buster/main amd64 Packages

I am pretty sure that the problem here is my limited debian packaging knowledge, so how can I build rsyslog 8.2102.0 using librelp-dev 1.7.0-1~bpo10+1 ?

Change 673442 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] aptrepo: add a new rsyslog-k8s component for buster-wikimedia

https://gerrit.wikimedia.org/r/673442

Change 673442 merged by Elukey:
[operations/puppet@production] aptrepo: add a new rsyslog-k8s component for buster-wikimedia

https://gerrit.wikimedia.org/r/673442

After a chat with Moritz we decided to create a specific component with 8.1901 for buster:

root@apt1001:/srv/wikimedia# reprepro lsbycomponent rsyslog
rsyslog | 8.1901.0-1~bpo8+wmf1 |  jessie-wikimedia |                  main | amd64, source
rsyslog | 8.1901.0-1~bpo9+wmf1 | stretch-wikimedia |                  main | amd64, source
rsyslog |   8.2008.0-1~bpo10+1 |  buster-wikimedia |     component/rsyslog | amd64, source
rsyslog |      8.1901.0-1+wmf1 |  buster-wikimedia | component/rsyslog-k8s | amd64, source

I'll file another code review to apply it for kubernetes on buster, and then we'll need to do something similar for bullseye. Filing a change for https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=911299 ourselves could be good as well, so post bullseye we'll not have this issue.

Change 673450 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::rsyslog::kubernetes: add component for buster

https://gerrit.wikimedia.org/r/673450

Change 673450 merged by Elukey:
[operations/puppet@production] profile::rsyslog::kubernetes: add component for buster

https://gerrit.wikimedia.org/r/673450

jijiki triaged this task as Medium priority.Mon, Mar 29, 8:59 PM