By default, node.js does not cache DNS resolution at all. This can be a performance issue especially for UDP. To avoid this for statsd, we configured raw IP addresses. However, this means that statsd hosts can't be easily switched by updating DNS.
Instead, we should set up generic DNS caching in node, possibly using the in-process cache described in T128015. TTLs should either respect DNS responses, or default to a conservative TTL of 5s. Since this should apply to all services, it might make sense to add this to service-runner.
Alternatively or additionally, we could set up a local caching DNS resolver service, and configure this as the local DNS server. However, since this would still involve DNS queries on each packet, it is likely that performance would not be as good as the in-process cache.