Due to some Puppet/Facter limitations, we currently have at least the following issues and workarounds:
- The ssh module uses the ipaddress6 fact to export the hosts' IPv6 address, but in some cases (cp*, authns) this is a service iP bound on the loopback interface. This results in SSH known hosts authentication failures over IPv6.
- On some hosts, like e.g. copper, that run docker, $ipaddress defaults to the 172.16.0.0/12 IP that the docker0 interface has, resulting in all kinds of weird behavior across the tree.
- Probably because of the above, realm.pp exports a puppet-global variable $main_ipaddress (but not $main_ipaddress6), which is set to the the first fact that is non-undef from (ipaddress_bond0, ipaddress_eth0, ipaddress). This is used very seldomly, in a couple of places in the tree.
- interface::add_ip6_mapped has code to figure out the "primary" interface, by taking the first out of the interfaces fact.
- Because "the first interface" this is not always the right one (and probably because of copy/paste), we have 79 definitions of interface::add_ip6_mapped that explicitly pass eth0 as the argument.
- Finally, over at T158429 we have been discussing switching to systemd/stretch's predictable interface names, which means that eth0 may soon not be the primary and in turn means that both these interface::add_ip6_mapped definitions and $main_ipaddress will be wrong.
For these reasons, I recently pushed abf0e49c32acbf99a993e47dd482e4d194d23318 which created these three facts:
- interface_primary: the interface that is being used to reach the default gateway (ip -4 route list 0/0). Inspired and named out of Facter 3's networking['primary'] fact. I'd name it the same for forward-compatibility but unfortunately we don't have Puppet 3.8's structured facts enabled :(
- ipaddress_primary: basically an alias for $ipaddress_${interface_primary}.
- ipaddress6_primary: Ditto.
These are not being used yet mainly due to lack of testing.
So, to fix the ipaddress/6 situation:
- Compare $ipaddress(6) with $ipaddress(6)_primary and $main_ipaddress across the fleet and audit manually all the differences. Fix any $ipaddress_primary bugs found during the audit.
- Rename ipaddress(6)_primary to ipaddress(6) with weight 100 (using Facter's precedence rules) [change merged]
- Replace $main_ipaddress by $::ipaddress (or $facts[$ipaddress]) across the tree. [change staged]
- Remove the hardcoded ipaddress_eth0 and ipaddress6_eth0 calls across the tree.
And to fix the interface_primary situation:
- Audit whether $interface_primary is the same to the first interface of $interfaces for all the hosts where add_ip6_mapped calls is applied without an explit interface parameter passed.
- Change add_ip6_mapped to use interface_primary and ipaddress [change staged]
- Remove the explicit interface parameter from all the add_ip6_mapped calls where is redundant i.e. where interface_primary equals the parameter, which is (almost?) always set to eth0.