Page MenuHomePhabricator

Failure to resolve 'puppet' in DNS on bookworm
Closed, ResolvedPublic

Description

The cloud-vps dns resolver has a hacked in entry for 'puppet' for the purpose of bootstrapping new VMs. This works on Bullseye but a brand new bookworm install can't run puppet because 'puppet' doesn't resolve properly.

DNS seems to otherwise work.

root@buildvm-a03feb21-71d1-4205-ac5b-ffcf53aa20a3:~# host en.wikipedia.org
en.wikipedia.org is an alias for dyna.wikimedia.org.
dyna.wikimedia.org has address 208.80.154.224
dyna.wikimedia.org has IPv6 address 2620:0:861:ed1a::1
root@buildvm-a03feb21-71d1-4205-ac5b-ffcf53aa20a3:~# ping en.wikipedia.org
PING dyna.wikimedia.org (208.80.154.224) 56(84) bytes of data.

root@buildvm-a03feb21-71d1-4205-ac5b-ffcf53aa20a3:~# host puppet
puppet has address 172.16.0.38
puppet has address 172.16.0.38
puppet has address 172.16.0.38
root@buildvm-a03feb21-71d1-4205-ac5b-ffcf53aa20a3:~# ping puppet
ping: puppet: Name or service not known

It's weird! 'host', 'nslookup,' and 'dig' resolve it just fine, but apparently the actual system call used by puppet and/or ping does not.

This seems to be somewhat related to this release note:

5.2.4. systemd-resolved has been split into a separate package

The new systemd-resolved package will not be installed automatically on upgrades. If you were using the systemd-resolved system service, please install the new package manually after the upgrade, and note that until it has been installed, DNS resolution might no longer work since the service will not be present on the system. Installing this package will automatically give systemd-resolved control of /etc/resolv.conf. For more information about systemd-resolved, consult the official documentation. Note that systemd-resolved was not, and still is not, the default DNS resolver in Debian. If you have not configured your machine to use systemd-resolved as the DNS resolver, no action is required.

If I manually uninstall systemd-resolved (and libnss-resolve which relies on it) then I can resolve 'puppet' just fine.

I'm not sure what the right way forward is. I'm pretty sure systemd-resolved is included in the upstream base image; at any rate so far I don't see where cloud-init would be installing it.

Related Objects

Event Timeline

Andrew updated the task description. (Show Details)

Hi, @jbond have you run across this by any chance or have thoughts on it?

This is apparently known behavior:

https://wiki.archlinux.org/title/systemd-resolved#systemd-resolved_does_not_resolve_hostnames_without_suffix

I'm going to try removing systemd-resolved and see if everything works without it.

Change 927317 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] nove vendordata: purge systemd-resolved libnss-resolve during first boot

https://gerrit.wikimedia.org/r/927317

Change 927317 merged by Andrew Bogott:

[operations/puppet@production] nove vendordata: purge systemd-resolved libnss-resolve during first boot

https://gerrit.wikimedia.org/r/927317

I think the above patch is adequate for this.