Page MenuHomePhabricator

Neutron networking not working for cloudnet200[5,6]-dev.codfw.wmnet
Closed, ResolvedPublic

Description

I just now tried to fail over neutron network service from cloudnet2004 to cloudnet2006 and everything broke.

Event Timeline

@ayounsi I have barely investigated this but I'm guessing that there's some kind of switch binding that needs to be done for these hosts to get them talking on cloud-hosts1-codfw. Can you take a quick look? These are brand new servers, just puppetized today.

@Andrew the racking task for the cloudnet nodes said to setup "cloud-gw-transport and cloud-instance-transport" on the second interface but looking at the settings on cloudnet2004 on the second interface we have cloud-gw-transport cloud-instance-transport and cloud-instance2 so i added the third Vlan.Please tried again and let me know.

Thanks

[edit interfaces ge-1/0/24 unit 0 family ethernet-switching vlan]
-       members [ cloud-gw-transport-codfw cloud-instance-transport1-b-codfw ];
+       members [ cloud-gw-transport-codfw cloud-instance-transport1-b-codfw cloud-instances2-b-codfw ];
[edit interfaces ge-1/0/26 unit 0 family ethernet-switching vlan]
-       members [ cloud-gw-transport-codfw cloud-instance-transport1-b-codfw ];
+       members [ cloud-gw-transport-codfw cloud-instance-transport1-b-codfw cloud-instances2-b-codfw ];

@Andrew the racking task for the cloudnet nodes said to setup "cloud-gw-transport and cloud-instance-transport" on the second interface but looking at the settings on cloudnet2004 on the second interface we have cloud-gw-transport cloud-instance-transport and cloud-instance2 so i added the third Vlan.Please tried again and let me know.

All working now. Thank you!

I spoke too soon! I now see that VMs can't talk to cloudserviceshosts:

root@tools-codfw1dev-k8s-worker-1:/var/log/sssd# ping 208.80.153.47
PING 208.80.153.47 (208.80.153.47) 56(84) bytes of data.

This seems to be a general issue; when routing through the new cloudnet hosts traffic is blocked between codfw1dev VMs and public production IPs.

... I'm now less sure about what's happening here, so stand by :)

Whatever this is, it wasn't what I thought it was. Working now.