Page MenuHomePhabricator

installation of logstash plugins should be more robust
Closed, DeclinedPublic

Description

During deployment of lgostash on deployment-prep, plugins were deployed on disk, but were not loaded into logstash. The current puppet code deploying the plugins does not seem robust enough. The [[ URL | equivalent puppet code written by elastic ]] seems more robust as it uses logstash-plugin to check if plugins are actually installed.

Note that the elastic approach requires deploying individual plugins and not a bundle as we do at the moment. So the change is not trivial. Still, it make sense to dig a bit to see if we can improve our current situation.

Event Timeline

The bundling was chosen based off the choices in the offline plugins documentation. This is basically the options for if you want to install plugins without reaching out to the internet. To install individual plugins rather than a plugin pack the logstash-plugin command can be pointed at a local url for fetching the plugin itself, but it will still reach out to a rubygems server to fetch the dependencies of the plugin. For this reason the documentation says you need to install a local Rubygems server. There are numerous options, and they don't seem particularly complicated. When originally building this though it seemed more straight forward to build the plugin pack as opposed to setting up a Rubygems server in a VM in prod.

I'm boldly resolving this in favor of using Debian packages: T217340: Change logstash plugin deployment to use deb packaging and deployment, although feel free to reopen!