Common information
- summary: No Puppet resources found on instance deployment-urldownloader04 on project deployment-prep
- alertname: PuppetAgentNoResources
- instance: deployment-urldownloader04
- job: node
- project: deployment-prep
- severity: warning
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | • Dreamy_Jazz | T397793 Implement hCaptcha enterprise trial configuration | |||
| Resolved | • sguebo_WMF | T388891 Decide hCaptcha enterprise trial configuration | |||
| Resolved | kostajh | T382148 Enable hCaptcha on test2wiki | |||
| Resolved | kostajh | T402366 hCaptcha account creation trial deployment tracker | |||
| Restricted Task | |||||
| Restricted Task | |||||
| Restricted Task | |||||
| Resolved | jijiki | T397841 Implement an hCaptcha IP blinding proxy prototype | |||
| Resolved | ssingh | T404407 No Puppet resources found on instance deployment-urldownloader04 on project deployment-prep |
Event Timeline
Comment Actions
bd808@deployment-urldownloader04:~$ sudo -i puppet agent -tv Info: Using environment 'production' Info: Retrieving pluginfacts Info: Retrieving plugin Info: Loading facts Info: Caching catalog for deployment-urldownloader04.deployment-prep.eqiad1.wikimedia.cloud Info: Applying configuration version '(5d2d1eab5e) gitpuppet - varnish: Add "Vary: User-Agent" during delivery of pageviews' Error: Systemd start for nginx failed! journalctl log for nginx: Sep 12 15:40:40 deployment-urldownloader04 systemd[1]: Starting nginx.service - A high performance web server and a reverse proxy server... Sep 12 15:40:40 deployment-urldownloader04 nginx[822041]: 2025/09/12 15:40:40 [emerg] 822041#822041: a duplicate listen 127.0.0.1:4260 in /etc/nginx/sites-enabled/assets-deployment-urldownloader04:12 Sep 12 15:40:40 deployment-urldownloader04 nginx[822041]: nginx: configuration file /etc/nginx/nginx.conf test failed Sep 12 15:40:40 deployment-urldownloader04 systemd[1]: nginx.service: Control process exited, code=exited, status=1/FAILURE Sep 12 15:40:40 deployment-urldownloader04 systemd[1]: nginx.service: Failed with result 'exit-code'. Sep 12 15:40:40 deployment-urldownloader04 systemd[1]: Failed to start nginx.service - A high performance web server and a reverse proxy server. Sep 12 15:41:22 deployment-urldownloader04 systemd[1]: Starting nginx.service - A high performance web server and a reverse proxy server... Sep 12 15:41:22 deployment-urldownloader04 nginx[822456]: 2025/09/12 15:41:22 [emerg] 822456#822456: a duplicate listen 127.0.0.1:4260 in /etc/nginx/sites-enabled/assets-deployment-urldownloader04:12 Sep 12 15:41:22 deployment-urldownloader04 nginx[822456]: nginx: configuration file /etc/nginx/nginx.conf test failed Sep 12 15:41:22 deployment-urldownloader04 systemd[1]: nginx.service: Control process exited, code=exited, status=1/FAILURE Sep 12 15:41:22 deployment-urldownloader04 systemd[1]: nginx.service: Failed with result 'exit-code'. Sep 12 15:41:22 deployment-urldownloader04 systemd[1]: Failed to start nginx.service - A high performance web server and a reverse proxy server. Error: /Stage[main]/Nginx/Service[nginx]/ensure: change from 'stopped' to 'running' failed: Systemd start for nginx failed! journalctl log for nginx: Sep 12 15:40:40 deployment-urldownloader04 systemd[1]: Starting nginx.service - A high performance web server and a reverse proxy server... Sep 12 15:40:40 deployment-urldownloader04 nginx[822041]: 2025/09/12 15:40:40 [emerg] 822041#822041: a duplicate listen 127.0.0.1:4260 in /etc/nginx/sites-enabled/assets-deployment-urldownloader04:12 Sep 12 15:40:40 deployment-urldownloader04 nginx[822041]: nginx: configuration file /etc/nginx/nginx.conf test failed Sep 12 15:40:40 deployment-urldownloader04 systemd[1]: nginx.service: Control process exited, code=exited, status=1/FAILURE Sep 12 15:40:40 deployment-urldownloader04 systemd[1]: nginx.service: Failed with result 'exit-code'. Sep 12 15:40:40 deployment-urldownloader04 systemd[1]: Failed to start nginx.service - A high performance web server and a reverse proxy server. Sep 12 15:41:22 deployment-urldownloader04 systemd[1]: Starting nginx.service - A high performance web server and a reverse proxy server... Sep 12 15:41:22 deployment-urldownloader04 nginx[822456]: 2025/09/12 15:41:22 [emerg] 822456#822456: a duplicate listen 127.0.0.1:4260 in /etc/nginx/sites-enabled/assets-deployment-urldownloader04:12 Sep 12 15:41:22 deployment-urldownloader04 nginx[822456]: nginx: configuration file /etc/nginx/nginx.conf test failed Sep 12 15:41:22 deployment-urldownloader04 systemd[1]: nginx.service: Control process exited, code=exited, status=1/FAILURE Sep 12 15:41:22 deployment-urldownloader04 systemd[1]: nginx.service: Failed with result 'exit-code'. Sep 12 15:41:22 deployment-urldownloader04 systemd[1]: Failed to start nginx.service - A high performance web server and a reverse proxy server. (corrective) Notice: /Stage[main]/Prometheus::Nginx_exporter/Service[prometheus-nginx-exporter]: Dependency Service[nginx] has failures: true Warning: /Stage[main]/Prometheus::Nginx_exporter/Service[prometheus-nginx-exporter]: Skipping because of failed dependencies Notice: Applied catalog in 9.03 seconds
Comment Actions
bd808@deployment-urldownloader04:/etc/nginx/sites-enabled$ grep 4260 * assets-deployment-urldownloader04: listen 127.0.0.1:4260 ssl; assets-deployment-urldownloader04: listen [::1]:4260 ssl; assets-deployment-urldownloader04: listen 127.0.0.1:4260 ssl; deployment-urldownloader04: listen 127.0.0.1:4260 ssl default_server; deployment-urldownloader04: listen [::1]:4260 ssl default_server; deployment-urldownloader04: listen 127.0.0.1:4260 ssl default_server; imgs-deployment-urldownloader04: listen 127.0.0.1:4260 ssl; imgs-deployment-urldownloader04: listen [::1]:4260 ssl; imgs-deployment-urldownloader04: listen 127.0.0.1:4260 ssl; report-deployment-urldownloader04: listen 127.0.0.1:4260 ssl; report-deployment-urldownloader04: listen [::1]:4260 ssl; report-deployment-urldownloader04: listen 127.0.0.1:4260 ssl;
Comment Actions
It seems to be by design in the Puppet config from https://gerrit.wikimedia.org/r/c/operations/puppet/+/1164432 that there will be multiple nginx vhosts that are all listening on 127.0.0.1:4260. Maybe @ssingh, @hnowlan, or @kamila can help figure out what is not quite right here?
Comment Actions
Ouch sorry. We were so busy rolling this out again in the morning that we forgot. I can fix this. What is happening here is essentially that the dummy service IP I added to keep Puppet happy is also the same IP we added for listening on the loopback.
Comment Actions
This should now be resolved. The urldownloaders are probably not the right hosts for this anyway, but since this is a prototype, we are using them and it is on our plan to fix this later.
Sorry about the error again.