Page MenuHomePhabricator

Important nagios-nrpe-server errors not showing up in unit journal
Open, MediumPublic

Description

A few services are not starting properly at boot time on cp5011 after having reimaged the host to the text_ats role in the context of T227432.

Among those, nrpe fails as follows:

Nov 01 16:22:45 cp5011 nrpe[1030]: Starting up daemon
Nov 01 16:22:45 cp5011 nrpe[1030]: Bind to port 5666 on 10.132.0.111 failed: Cannot assign requested address.
Nov 01 16:22:45 cp5011 nrpe[1030]: Cannot bind to any address.
Nov 01 16:22:45 cp5011 systemd[1]: nagios-nrpe-server.service: PID 1030 read from file /var/run/nagios/nrpe.pid does not exist or is a zombie.
Nov 01 16:22:45 cp5011 systemd[1]: Failed to start Nagios Remote Plugin Executor.
Nov 01 16:22:45 cp5011 systemd[1]: nagios-nrpe-server.service: Unit entered failed state.
Nov 01 16:22:45 cp5011 systemd[1]: nagios-nrpe-server.service: Failed with result 'resources'.

The most interesting of the error messages above isn't shown when filtering journalctl's output by unit name:

$ sudo journalctl -u nagios-nrpe-server.service
-- Logs begin at Fri 2019-11-01 16:22:43 UTC, end at Mon 2019-11-04 10:38:17 UTC. --
Nov 01 16:22:45 cp5011 systemd[1]: Starting Nagios Remote Plugin Executor...
Nov 01 16:22:45 cp5011 nrpe[1030]: Starting up daemon
Nov 01 16:22:45 cp5011 systemd[1]: nagios-nrpe-server.service: PID 1030 read from file /var/run/nagios/nrpe.pid does not exist or is a zombie.
Nov 01 16:22:45 cp5011 systemd[1]: Failed to start Nagios Remote Plugin Executor.
Nov 01 16:22:45 cp5011 systemd[1]: nagios-nrpe-server.service: Unit entered failed state.
Nov 01 16:22:45 cp5011 systemd[1]: nagios-nrpe-server.service: Failed with result 'resources'.

Filtering by executable name is likewise misleading:

$ sudo journalctl /usr/sbin/nrpe
-- Logs begin at Fri 2019-11-01 16:22:43 UTC, end at Mon 2019-11-04 10:40:39 UTC. --
Nov 01 16:22:45 cp5011 nrpe[1030]: Starting up daemon
Nov 01 16:27:48 cp5011 nrpe[6266]: Starting up daemon
Nov 01 16:27:48 cp5011 nrpe[6266]: Server listening on 10.132.0.111 port 5666.
Nov 01 16:27:48 cp5011 nrpe[6266]: Listening for connections on port 5666
Nov 01 16:27:48 cp5011 nrpe[6266]: Allowing connections from: 208.80.154.84,2620:0:861:3:208:80:154:84,208.80.153.74,2620:0:860:3:208:80:153:74

Event Timeline

ema created this task.Nov 4 2019, 10:42 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 4 2019, 10:42 AM
MoritzMuehlenhoff triaged this task as Medium priority.Nov 5 2019, 10:48 AM