Page MenuHomePhabricator

Puppet broken on deployment-cache-text04 due to varnishkafka issues
Closed, ResolvedPublic

Description

E.g. missing hieradata, and this error:
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Function Call, undefined method `[]' for nil:NilClass at /etc/puppet/modules/profile/manifests/cache/kafka/statsv.pp:25:22 on node deployment-cache-text04.deployment-prep.eqiad.wmflabs

Event Timeline

Krenair triaged this task as Medium priority.Jan 5 2018, 1:39 AM
Krenair created this task.

hiera part:

diff --git a/hieradata/labs/deployment-prep/host/deployment-cache-text04.yaml b/hieradata/labs/deployment-prep/host/deployment-cache-text04.yaml
index a4e902a4ea..908d7c7ed7 100644
--- a/hieradata/labs/deployment-prep/host/deployment-cache-text04.yaml
+++ b/hieradata/labs/deployment-prep/host/deployment-cache-text04.yaml
@@ -1,3 +1,4 @@
+profile::cache::kafka::statsv::kafka_cluster_name: main-eqiad
 profile::cache::base::varnish_version: 5
 nginx::variant: extras
 cache::lua_support: true

no idea wtf that method `[]' for nil:NilClass error means

Mentioned in SAL (#wikimedia-releng) [2018-03-29T14:26:20Z] <elukey> removed from deployment-puppetmaster02's ops repo "1ffd6e72a9 WIP Bug T184234" due to git sync failures

EddieGP subscribed.

Seems fixed.

eddie@deployment-cache-text04:~$ sudo puppet agent -tv
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for deployment-cache-text04.deployment-prep.eqiad.wmflabs
Notice: /Stage[main]/Base::Environment/Tidy[/var/tmp/core]: Tidying 0 files
Info: Applying configuration version '1523316846'
Notice: tlsproxy::localssl instance unified with server name beta.wmflabs.org is the default server.
Notice: /Stage[main]/Profile::Cache::Ssl::Unified/Tlsproxy::Localssl[unified]/Notify[tlsproxy localssl default_server]/message: defined 'message' as 'tlsproxy::localssl instance unified with server name beta.wmflabs.org is the default server.'
Notice: /Stage[main]/Profile::Cache::Ssl::Unified/Tlsproxy::Localssl[unified]/Letsencrypt::Cert::Integrated[beta.wmflabs.org]/Exec[acme-setup-acme-beta_wmflabs_org]/returns: executed successfully
Notice: /Stage[main]/Mtail/Systemd::Service[mtail]/Service[mtail]/ensure: ensure changed 'stopped' to 'running'
Info: /Stage[main]/Mtail/Systemd::Service[mtail]/Service[mtail]: Unscheduling refresh on Service[mtail]
Notice: /Stage[main]/Confd/Base::Service_unit[confd]/Service[confd]/ensure: ensure changed 'stopped' to 'running'
Info: /Stage[main]/Confd/Base::Service_unit[confd]/Service[confd]: Unscheduling refresh on Service[confd]
Notice: Applied catalog in 43.82 seconds
eddie@deployment-cache-text04:~$