With the current Puppetization for Netbox, Postgres is not able to start. This happened today after a reboot for the kernel security update.
To make it running again I had to manually modify the configurations as follows:
- comment include 'tuning.conf' in /etc/postgresql/9.6/main/postgresql.conf
- comment checkpoint_segments = 64 in /etc/postgresql/9.6/main/master.conf
Few action items here:
- Puppet is broken on netmon2001 because postgres is not installed
- Postgres DB was empty after I was able to have it restarted. No tables defined, but a netbox DB is defined.
- Icinga doesn't have any check that Postgres is up and running, see T185504
- Icinga doens't have any check that Netbox is up and running, see T185505
- The Postgres module uses a master.conf.erb file that defines checkpoint_segments = <%= @checkpoint_segments %>, but the checkpoint_segments setting was deprecated in 9.5 and removed in 9.6, see https://www.postgresql.org/docs/9.6/static/release-9-5.html.
- Netbox profile call the Postgres one with includes => ['tuning.conf'],, but it doesn't install the tuning.conf file, as required by our current puppetization. Does it need it? See modules/postgresql/manifests/server.pp:
# includes # An array of files that will be included in the config. It is # the caller's responsibility to provide these
- A bit unrelated to this error, but came out while debugging, in modules/role/manifests/postgres/common.pp the version 9.4 of Postgresql is hardcoded
- Puppet was failing because the command defined in $userexists inside modules/postgresql/manifests/user.pp was failing, and the real reason (postgres not started) is a bit hidden in the middle of the message. I was wondering if it should had fail earlier. Here the log:
Notice: /Stage[main]/Profile::Netbox/Postgresql::User[replication@netmon2001]/Exec[create_user-replication@netmon2001]/returns: could not change directory to "/root": Permission denied Notice: /Stage[main]/Profile::Netbox/Postgresql::User[replication@netmon2001]/Exec[create_user-replication@netmon2001]/returns: createuser: could not connect to database postgres: could not connect to server: No such file or directory Notice: /Stage[main]/Profile::Netbox/Postgresql::User[replication@netmon2001]/Exec[create_user-replication@netmon2001]/returns: Is the server running locally and accepting Notice: /Stage[main]/Profile::Netbox/Postgresql::User[replication@netmon2001]/Exec[create_user-replication@netmon2001]/returns: connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"? Error: /usr/bin/createuser --no-superuser --no-createdb --no-createrole replication returned 1 instead of one of [0] Error: /Stage[main]/Profile::Netbox/Postgresql::User[replication@netmon2001]/Exec[create_user-replication@netmon2001]/returns: change from notrun to 0 failed: /usr/bin/createuser --no-superuser --no-createdb --no-createrole replication returned 1 instead of one of [0]