Page MenuHomePhabricator

No Puppet resources found on instance deployment-urldownloader04 on project deployment-prep
Closed, ResolvedPublicBUG REPORT

Description

Common information

  • summary: No Puppet resources found on instance deployment-urldownloader04 on project deployment-prep
  • alertname: PuppetAgentNoResources
  • instance: deployment-urldownloader04
  • job: node
  • project: deployment-prep
  • severity: warning

Firing alerts


  • summary: No Puppet resources found on instance deployment-urldownloader04 on project deployment-prep
  • alertname: PuppetAgentNoResources
  • instance: deployment-urldownloader04
  • job: node
  • project: deployment-prep
  • severity: warning
  • Source

Event Timeline

bd808@deployment-urldownloader04:~$ sudo -i puppet agent -tv
Info: Using environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Function lookup() did not find a value for the name 'profile::hcaptcha::proxy::wikimedia_domains' (file: /srv/puppet_code/environments/production/modules/profile/manifests/hcaptcha/proxy.pp, line: 10) on node deployment-urldownloader04.deployment-prep.eqiad1.wikimedia.cloud
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

https://gerrit.wikimedia.org/r/plugins/gitiles/cloud/instance-puppet/+/370453a723871c08d849a011d8d28bb85542d8a5%5E%21/#F0

diff --git a/deployment-prep/deployment-urldownloader.yaml b/deployment-prep/deployment-urldownloader.yaml
index a172d06..389539c 100644
--- a/deployment-prep/deployment-urldownloader.yaml
+++ b/deployment-prep/deployment-urldownloader.yaml

@@ -4,3 +4,4 @@
 profile::hcaptcha::proxy::nginx_ipblinding_conf: ''
 profile::hcaptcha::proxy::nginx_private_conf: proxy_set_header X-Clacks-Overhead "GNU
   Terry Pratchett";
+profile::hcaptcha::proxy::wikimedia_domains: '%{alias(''wikimedia_domains'')}'
bd808 changed the task status from Open to In Progress.Aug 26 2025, 9:50 PM
bd808 claimed this task.
bd808 triaged this task as Medium priority.
bd808 moved this task from To Triage to Puppet errors on the Beta-Cluster-Infrastructure board.
bd808 changed the subtype of this task from "Task" to "Bug Report".
$ sudo -i puppet agent -tv
Info: Using environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for deployment-urldownloader04.deployment-prep.eqiad1.wikimedia.cloud
Info: Applying configuration version '(926ba1f6cb) gitpuppet - varnish: Implement new direct routing for mobile views'
Error: Systemd start for nginx failed!
journalctl log for nginx:
Aug 27 18:06:30 deployment-urldownloader04 systemd[1]: Starting nginx.service - A high performance web server and a reverse proxy server...
Aug 27 18:06:30 deployment-urldownloader04 nginx[115838]: 2025/08/27 18:06:30 [emerg] 115838#115838: unknown directive "header_filter_by_lua_file" in /etc/nginx/sites-enabled/assets-deployment-urldownloader04:42
Aug 27 18:06:30 deployment-urldownloader04 nginx[115838]: nginx: configuration file /etc/nginx/nginx.conf test failed
Aug 27 18:06:30 deployment-urldownloader04 systemd[1]: nginx.service: Control process exited, code=exited, status=1/FAILURE
Aug 27 18:06:30 deployment-urldownloader04 systemd[1]: nginx.service: Failed with result 'exit-code'.
Aug 27 18:06:30 deployment-urldownloader04 systemd[1]: Failed to start nginx.service - A high performance web server and a reverse proxy server.

Error: /Stage[main]/Nginx/Service[nginx]/ensure: change from 'stopped' to 'running' failed: Systemd start for nginx failed!
journalctl log for nginx:
Aug 27 18:06:30 deployment-urldownloader04 systemd[1]: Starting nginx.service - A high performance web server and a reverse proxy server...
Aug 27 18:06:30 deployment-urldownloader04 nginx[115838]: 2025/08/27 18:06:30 [emerg] 115838#115838: unknown directive "header_filter_by_lua_file" in /etc/nginx/sites-enabled/assets-deployment-urldownloader04:42
Aug 27 18:06:30 deployment-urldownloader04 nginx[115838]: nginx: configuration file /etc/nginx/nginx.conf test failed
Aug 27 18:06:30 deployment-urldownloader04 systemd[1]: nginx.service: Control process exited, code=exited, status=1/FAILURE
Aug 27 18:06:30 deployment-urldownloader04 systemd[1]: nginx.service: Failed with result 'exit-code'.
Aug 27 18:06:30 deployment-urldownloader04 systemd[1]: Failed to start nginx.service - A high performance web server and a reverse proxy server.
 (corrective)
Notice: /Stage[main]/Prometheus::Nginx_exporter/Service[prometheus-nginx-exporter]: Dependency Service[nginx] has failures: true
Warning: /Stage[main]/Prometheus::Nginx_exporter/Service[prometheus-nginx-exporter]: Skipping because of failed dependencies
Notice: Applied catalog in 7.86 seconds

https://gerrit.wikimedia.org/r/plugins/gitiles/cloud/instance-puppet/+/7fdfb72c7b762c001c6f215c1ea8229324e4f6c1%5E%21/#F0

diff --git a/deployment-prep/deployment-urldownloader.yaml b/deployment-prep/deployment-urldownloader.yaml
index 389539c..db50f6f 100644
--- a/deployment-prep/deployment-urldownloader.yaml
+++ b/deployment-prep/deployment-urldownloader.yaml

@@ -5,3 +5,4 @@
 profile::hcaptcha::proxy::nginx_private_conf: proxy_set_header X-Clacks-Overhead "GNU
   Terry Pratchett";
 profile::hcaptcha::proxy::wikimedia_domains: '%{alias(''wikimedia_domains'')}'
+profile::nginx::variant: extras

https://gerrit.wikimedia.org/r/c/operations/puppet/+/1164432 changed that hiera setting upstream.