Page MenuHomePhabricator

Package logstash-logback-encoder for Debian
Open, LowPublic

Description

Our Cassandra clusters use a third-party jar to support logging to logstash, and we deploy that jar (and its dependencies) via a scap repository. This is problematic for a few reasons:

  1. Relies on git-fat, which as of yet has not been ported to Python 3 (see: T279509)
  2. Requires that we maintain a separate list of deployment targets (duplicative, error-prone, and frequently forgotten)
  3. Lots of moving parts (Archiva, rsync, git-fat, oh my!); It is complex, brittle, and often fails in opaque ways
NOTE: Unless something changes between now and then, item #1 alone will be a blocker on Bookworm.

If nothing else, a Debian package of the dependencies would work well (and they rarely require updating).

Update: Migrated to git-lfs (priority moved to low)

Event Timeline

A Debian package already exists (in testing):

$ apt policy liblogstash-logback-encoder-java
liblogstash-logback-encoder-java:
  Installed: (none)
  Candidate: 7.2-2
  Version table:
     7.2-2 500
        500 http://deb.debian.org/debian testing/main amd64 Packages
        500 http://deb.debian.org/debian unstable/main amd64 Packages
$

It looks like this could be easy as a) installing liblogstash-logback-encoder-java (and deps¹), and then b) adding the jars to extra_classpath in hiera:

profile::cassandra::settings:
  extra_classpath:
  - /usr/share/java/logstash-logback-encoder.jar
  - /usr/share/java/disruptor.jar
  - /usr/share/java/jackson-annotations.jar
  - /usr/share/java/jackson-core.jar
  - /usr/share/java/jackson-databind.jar

¹: libdisruptor-java, libjackson2-annotations-java, libjackson2-core-java, libjackson2-databind-java

Change 1007671 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[All-Projects@refs/meta/config] Add LFS config for operations/software/logstash-logback-encoder

https://gerrit.wikimedia.org/r/1007671

Change 1007671 merged by Ahmon Dancy:

[All-Projects@refs/meta/config] Add LFS config for operations/software/logstash-logback-encoder

https://gerrit.wikimedia.org/r/1007671

Change 1007701 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[operations/software/logstash-logback-encoder@master] Switch from git-fat to git-lfs

https://gerrit.wikimedia.org/r/1007701

@Eevans To take care of the immediate concern of getting away from git-fat, I prepared https://gerrit.wikimedia.org/r/c/operations/software/logstash-logback-encoder/+/1007701. Let me know what you think.

Change 1007701 merged by Eevans:

[operations/software/logstash-logback-encoder@master] Switch from git-fat to git-lfs

https://gerrit.wikimedia.org/r/1007701

dancy updated the task description. (Show Details)
Eevans lowered the priority of this task from Medium to Low.

I think that ultimately we may still want to do this (even if our backs are no longer up against the wall). We don't routinely update these, and having a separate deployment like this just adds moving parts.

Eevans renamed this task from Find alternative to scap for deployment of logstash-logback-encoder to Package logstash-logback-encoder for Debian.Mar 7 2024, 9:07 PM
Eevans updated the task description. (Show Details)
dancy removed dancy as the assignee of this task.Mar 7 2024, 9:07 PM
dancy subscribed.
Eevans removed dancy as the assignee of this task.
Eevans updated the task description. (Show Details)