Change 594460 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] profile: initial tests for logstash filters

https://gerrit.wikimedia.org/r/594460

cc Release-Engineering-Team: I'd like to have puppet CI run a golang binary and logstash for logstash regression tests, what's the recommended way to achieve that ? I outlined a couple of options in the task description, what do you think ?

I gave the naive option (add logstash to operations-puppet docker image) and I realize it might be a tall order for the size difference (1.12GB vs 1.69GB) but it would work as expected, logstash tests would of course run only when needed

diff --git a/dockerfiles/operations-puppet/Dockerfile.template b/dockerfiles/operations-puppet/Dockerfile.template
index b6f22382..b67f49c8 100644
--- a/dockerfiles/operations-puppet/Dockerfile.template
+++ b/dockerfiles/operations-puppet/Dockerfile.template
@@ -6,11 +6,20 @@ ENV PUPPET_DIR='/srv/workspace/puppet'
 
 {% set pkgs_to_install = """build-essential bundler  rubygems-integration \
 rake ruby ruby-dev ca-certificates default-libmysqlclient-dev mtail=3.0.0~rc5-1~bpo9+1wmf2 \
-isc-dhcp-server unzip lua-busted file python3-dev python3-pip tox python-dev python-pip python3-wheel python-wheel systemd"""
+isc-dhcp-server unzip lua-busted file python3-dev python3-pip tox python-dev python-pip \
+python3-wheel python-wheel systemd logstash-oss logstash-filter-verifier"""
 %}
 
 ARG PIP_DISABLE_PIP_VERSION_CHECK=1
 
+# Install logstash repositories and JRE. The latter is required for Logstash to
+# not fail at install time if java isn't found or hasn't been configured by
+# dpkg yet. See also https://github.com/elastic/logstash/issues/6394
+USER root
+RUN echo "deb http://apt.wikimedia.org/wikimedia buster-wikimedia thirdparty/elastic74" \
+    > /etc/apt/sources.list.d/wikimedia-elastic.list && \
+    {{ "default-jre-headless" | apt_install }}
+
 USER root
 RUN {{ pkgs_to_install | apt_install }} \
     && pip3 install --no-cache-dir setuptools \

Change 597227 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[integration/config@master] operations-puppet: ship logstash and logstash-filter-verifier

https://gerrit.wikimedia.org/r/597227

I think we're at the point where it would be best if we could change the logic of our testing, and use docker directly, so that we can split tests into different images.

But I think that should be a separated task and should not block you right now.

Change 598703 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Job to run logstash-filter-verifier on puppet.git

https://gerrit.wikimedia.org/r/598703

Change 597227 merged by jenkins-bot:
[integration/config@master] dockerfiles: image for logstash-filter-verifier

https://gerrit.wikimedia.org/r/597227

Mentioned in SAL (#wikimedia-releng) [2020-05-26T12:26:48Z] <hashar> Built docker-registry.discovery.wmnet/releng/logstash-filter-verifier:0.0.1 # T251869

Change 598703 merged by jenkins-bot:
[integration/config@master] Job to run logstash-filter-verifier on puppet.git

https://gerrit.wikimedia.org/r/598703

Change 594460 merged by Filippo Giunchedi:
[operations/puppet@production] profile: initial tests for logstash filters

https://gerrit.wikimedia.org/r/594460

Change 608061 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] logstash: add tests for filter-mediawiki

https://gerrit.wikimedia.org/r/608061

Change 608061 merged by Filippo Giunchedi:
[operations/puppet@production] logstash: add tests for filter-mediawiki

https://gerrit.wikimedia.org/r/c/operations/puppet/ /608061