We're chronically thwarted by codf1w-dev's differences from eqiad1. After much discussion we've mostly agreed that we should just make this like eqiad1.
|operations/puppet : production||nova firstboot.sh: remove a no-longer-needed apt hack for codf1dev|
|operations/dns : master||wikimediacloud.org: introduce FQDN for routing_source_ip in codfw1dev|
|operations/puppet : production||openstack: codfw1dev: update routing_source_ip|
|operations/homer/public : master||Start advertising 22.214.171.124/24 from codfw/eqdfw|
|Resolved||aborrero||T217891 CloudVPS: rework codfw deployments|
|Open||None||T229441 CloudVPS: codfw1dev: missing bits|
|Resolved||aborrero||T239347 create a 'normal' network for codf1dev neutron w/public IPs|
First step I think would be to have a proper routing_source_ip address in codfw1dev.
Current status in eqiad1:
- routing_source_ip is 126.96.36.199
- belongs to CIDR 188.8.131.52/25 https://netbox.wikimedia.org/ipam/prefixes/2/
- FQDN nat.openstack.eqiad1.wikimediacloud.org
- core routers have routing to direct this CIDR to 184.108.40.206/29 which is cloudinstances2b-gw.openstack.eqiad1.wikimediacloud.org (the neutron virtual router in eqiad1)
Current status in codfw1dev:
- routing_source_ip is 172.16.129.254
- belongs to CIDR 172.16.129.0/24 https://netbox.wikimedia.org/ipam/prefixes/307/
- no FQDN
Thus, we would need to:
- allocate a public IPv4 CIDR (probably a /29 should be enough). This provides us with 8 addresses. We can use 1 for routing_source_ip and the other 7 for floating_ips we may need in this deployment.
- routing in core routers. The whole CIDR should be directed to 220.127.116.11/29, which is cloudinstances2b-gw.openstack.codfw1dev.wikimediacloud.org (the neutron virtual router in codfw1dev)
- introduce FQDN nat.openstack.codfw1dev.wikimediacloud.org for the first address in the CIDR.
- update neutron net/subnet objects in codf1dev, making sure we trim down the related IP pool to account for the routing_source_ip address.
- review and double-check core router filtering, although it should be already very similar to the one for eqiad1.
cc @ayounsi for actionable items on his side (allocating the CIDR and adding routing).
If a /25 works for you, then it works for us too :-) It is very unlikely that we will use such amount of addresses though.
We don't have a strong timeline (this is not part of a Q goal, yet). Would like to see this moving forward but can wait until holidays season is over.
Added the following routes on cr1/2-codfw:
set routing-options static route 18.104.22.168/29 next-hop 22.214.171.124
As well as starting advertising 126.96.36.199/24 from codfw/eqdfw:
set routing-options aggregate route 188.8.131.52/24 policy BGP_aggregate_contributors
set policy-options prefix-list bgp-out 184.108.40.206/24
Another option would be to advertise 220.127.116.11/23 from both eqiad/codfw.
This should be all done. @Andrew I think you were finding difficulties doing some operations without this setting being completed. Could you please confirm whatever you were doing is working now?
Was it doing apt-get stuff? We don't seem to have any specific filter in core-routers.
Closing task now, please reopen if required. Thanks @ayounsi