After yesterday move to cloudsw2-d5, puppet is no longer reachable on ipv6 for this host.
The issue is that by default puppet uses ipv6, so puppetruns are failing.
Some debugging:
When trying to ping from cloudcephosd1015 -> puppetmaster1001 packets reach puppetmaster1001, but they don't come back
to cloudcephosd1015:
PING puppet(puppetmaster1001.eqiad.wmnet (2620:0:861:102:10:64:16:73)) 56 data bytes ^C --- puppet ping statistics --- 14 packets transmitted, 0 received, 100% packet loss, time 332ms dcaro@puppetmaster1001:~$ sudo tcpdump -nvvi any host 2620:0:861:11e:10:64:20:66 tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes 07:05:34.338686 IP6 (flowlabel 0xfa1e8, hlim 62, next-header TCP (6) payload length: 40) 2620:0:861:11e:10:64:20:66.56164 > 2620:0:861:102:10:64:16:73.8140: Flags [S], cksum 0xe42e (correct), seq 3374414322, win 35760, options [mss 8940,sackOK,TS val 1508753819 ecr 0,nop,wscale 9], length 0 07:05:34.338722 IP6 (flowlabel 0x36c43, hlim 64, next-header TCP (6) payload length: 40) 2620:0:861:102:10:64:16:73.8140 > 2620:0:861:11e:10:64:20:66.56164: Flags [S.], cksum 0x6147 (incorrect -> 0x5dc7), seq 3327565436, ack 3374414323, win 42840, options [mss 1440,sackOK,TS val 910059398 ecr 1508722430,nop,wscale 9], length 0 07:05:34.491236 IP6 (flowlabel 0xa3f9a, hlim 62, next-header ICMPv6 (58) payload length: 64) 2620:0:861:11e:10:64:20:66 > 2620:0:861:102:10:64:16:73: [icmp6 sum ok] ICMP6, echo request, seq 1 07:05:34.491294 IP6 (flowlabel 0xa9558, hlim 64, next-header ICMPv6 (58) payload length: 64) 2620:0:861:102:10:64:16:73 > 2620:0:861:11e:10:64:20:66: [icmp6 sum ok] ICMP6, echo reply, seq 1 07:05:35.522677 IP6 (flowlabel 0xa3f9a, hlim 62, next-header ICMPv6 (58) payload length: 64) 2620:0:861:11e:10:64:20:66 > 2620:0:861:102:10:64:16:73: [icmp6 sum ok] ICMP6, echo request, seq 2 07:05:35.522729 IP6 (flowlabel 0xa9558, hlim 64, next-header ICMPv6 (58) payload length: 64) 2620:0:861:102:10:64:16:73 > 2620:0:861:11e:10:64:20:66: [icmp6 sum ok] ICMP6, echo reply, seq 2 07:05:36.546686 IP6 (flowlabel 0xa3f9a, hlim 62, next-header ICMPv6 (58) payload length: 64) 2620:0:861:11e:10:64:20:66 > 2620:0:861:102:10:64:16:73: [icmp6 sum ok] ICMP6, echo request, seq 3 07:05:36.546737 IP6 (flowlabel 0xa9558, hlim 64, next-header ICMPv6 (58) payload length: 64) 2620:0:861:102:10:64:16:73 > 2620:0:861:11e:10:64:20:66: [icmp6 sum ok] ICMP6, echo reply, seq 3 07:05:37.570681 IP6 (flowlabel 0xa3f9a, hlim 62, next-header ICMPv6 (58) payload length: 64) 2620:0:861:11e:10:64:20:66 > 2620:0:861:102:10:64:16:73: [icmp6 sum ok] ICMP6, echo request, seq 4
Hmm... it seems to have an ip from the old vlan configured:
root@cloudcephosd1015:~# ip a
...
2: ens3f0np0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
...
inet6 2620:0:861:11e:10:64:20:66/64 scope global dynamic mngtmpaddr
valid_lft 2542487sec preferred_lft 555287sec
...I'll try rebooting from scratch so it boots fresh from the new vlan.