Background
While working through some of the Cloud DNS stuff with @Andrew we noticed that cloudservices1006 is using private IP 10.64.151.4 to send DNS NOTIFY messages to cloudservices1005. We probably should have anticipated this, as that's the IP cloudservices1006 will use by default when trying to reach any external address. Including the current cloudservices1005 IP of 208.80.154.148.
In an ideal scenario mdns would use the local public IP for all comms. We've configured the local-address and query-local-address pdns options to force that, but it seems they don't affect DNS NOTIFYs. Checking through the pdns options I couldn't see a similar way to control the source IP for NOTIFYs (someone with more pdns experience might want to double check). (EDIT: mdns is producing the updates / notify messages, and sending them to pdns, pdns is not creating them).
Right now, when cloudservices1006 sends a NOTIFY to itself, it's using 185.15.56.163 ('lo' interface). But when sending them to cloudservices1005 it picks 10.64.151.4 ('eno12399np0' interface). This is probably just normal kernel IP selection based on outbound interface. Either way it's not using the same IP for all NOTIFYs.
Current Setup
Andrew updated the list of IPs defined as "master" on each host earlier, to ensure updates were allowed both from the local system and its remote peer. This cleared up errors we observed whereby cloudservices1006 was rejecting its own NOTIFYs sent from 185.15.56.163. Right now the masters are set up as follows and all updates are being accepted:
| Host | Allowed Masters |
|---|---|
| cloudservices1005 | 208.80.154.148 (itself), 10.64.151.4 (cloudservices1006) |
| cloudservices1006 | 185.15.56.163 (itself), 208.80.154.148 (cloudservices1005) |
Eventual Setup
When we have moved and reimaged cloudservices1005, making it live again for the new ns0 IP 185.15.56.162, we probably need to have the following setup:
| Host | Allowed Masters |
|---|---|
| cloudservices1005 | 185.15.56.162 (itself), 172.20.1.5 (cloudservices1006) |
| cloudservices1006 | 185.15.56.163 (itself), 172.20.2.4 (cloudservices1005) |
Ideally we could just have 185.15.56.162 and 185.15.56.163 on both, covering the local and remote system in either case. But instead we need a different pair of IPs on each, as the systems are using different source addresses for local vs. remote updates. Perhaps we could include all 4 IPs on both, if it didn't cause any other issue?