Page MenuHomePhabricator

Upgrade prometheus-jmx-exporter
Closed, ResolvedPublic

Description

Logstash 7.10.x does not work with prometheus-jmx-exporter 0.3.0:

Mar  4 23:06:45 pontoon-logstash7-03 logstash[25121]: /usr/share/logstash/logstash-core-plugin-api/lib/logstash-core-plugin-api/version.rb:22: warning: The Psych YAML extension failed to load.
Mar  4 23:06:45 pontoon-logstash7-03 logstash[25121]: Check your env for conflicting versions of SnakeYAML
Mar  4 23:06:45 pontoon-logstash7-03 logstash[25121]: See https://github.com/jruby/jruby/wiki/FAQs#why-does-the-psych-yaml-extension-fail-to-load-in-my-environment
Mar  4 23:06:45 pontoon-logstash7-03 logstash[25121]: [5.655s][info ][safepoint] Application time: 0.0524442 seconds
Mar  4 23:06:45 pontoon-logstash7-03 logstash[25121]: [5.655s][info ][safepoint] Entering safepoint region: RevokeBias
Mar  4 23:06:45 pontoon-logstash7-03 logstash[25121]: [5.655s][info ][safepoint] Leaving safepoint region
Mar  4 23:06:45 pontoon-logstash7-03 logstash[25121]: [5.655s][info ][safepoint] Total time for which application threads were stopped: 0.0006017 seconds, Stopping threads took: 0.0000053 seconds
Mar  4 23:06:45 pontoon-logstash7-03 logstash[25121]: [ERROR] 2021-03-04 23:06:45.911 [main] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (GemspecError)
Mar  4 23:06:45 pontoon-logstash7-03 logstash[25121]: [!] There was an error while loading `logstash-core-plugin-api.gemspec`: bad SnakeYAML version 1.16, required 1.21 or higher; check your CLASSPATH for a conflicting jar
Mar  4 23:06:45 pontoon-logstash7-03 logstash[25121]: Does it try to require a relative path? That's been removed in Ruby 1.9. Bundler cannot continue.
Mar  4 23:06:45 pontoon-logstash7-03 logstash[25121]:  #  from /usr/share/logstash/logstash-core-plugin-api/logstash-core-plugin-api.gemspec:23
Mar  4 23:06:45 pontoon-logstash7-03 logstash[25121]:  #  -------------------------------------------
Mar  4 23:06:45 pontoon-logstash7-03 logstash[25121]:  #
Mar  4 23:06:45 pontoon-logstash7-03 logstash[25121]:  >  require "logstash-core-plugin-api/version"
Mar  4 23:06:45 pontoon-logstash7-03 logstash[25121]:  #
Mar  4 23:06:45 pontoon-logstash7-03 logstash[25121]:  #  -------------------------------------------

It appears to only surface in jruby 9.2.13.0 (and likely greater). Logstash starts correctly when prometheus-jmx-exporter is removed from the startup options.

Tested prometheus-jmx-exporter 0.15.0 on pontoon and it appears to work. We ought to upgrade that.

Tagging other users of prometheus-jmx-exporter for visibility.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
colewhite renamed this task from Logstash 7.10.x does not work with prometheus-jmx-exporter 0.3.0 to Upgrade prometheus-jmx-exporter.Mar 5 2021, 5:38 PM
colewhite updated the task description. (Show Details)

Change 669894 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/debs/prometheus-jmx-exporter@master] Upgrade to upstream version 0.15.0

https://gerrit.wikimedia.org/r/669894

Milimetric moved this task from temporary to Incoming on the Analytics board.
crusnov triaged this task as Medium priority.Mar 8 2021, 8:11 PM

Hello! Does Analytics have to upgrade too? :)

Hello! Does Analytics have to upgrade too? :)

The updated jar will be deployed to to our apt repo which will make the old version unavailable for newly-provisioned hosts. I recommend upgrading so there are no surprises on new host creation.

Change 669894 merged by Cwhite:
[operations/debs/prometheus-jmx-exporter@master] Upgrade to upstream version 0.15.0

https://gerrit.wikimedia.org/r/669894

colewhite claimed this task.

prometheus-jmx-exporter 0.15.0 is deployed to our apt repo.