Page MenuHomePhabricator

Floating IP allocation allows compat-net
Closed, DeclinedPublic

Description

<Krenair> we're supposed to allocate floating IPs on wan-transport-eqiad not compat-net right?
<andrewbogott> Krenair: I think it's a bug that that dialog shows you two options. arturo may know more
<andrewbogott> but, yes :)
<Krenair> ok good
<arturo> yes I know a workaround (not a proper fix, but hey...)
<arturo> Krenair: please open a phab ticket so I dont forget

Event Timeline

Krenair created this task.Nov 20 2018, 2:48 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 20 2018, 2:48 PM

Mentioned in SAL (#wikimedia-cloud) [2018-11-21T10:48:34Z] <arturo> re-created compat-net as not shared in labtestn to test stuff related to T209954

aborrero triaged this task as Normal priority.Nov 21 2018, 10:48 AM
aborrero moved this task from Inbox to Doing on the cloud-services-team (Kanban) board.

There is no clean way to tell neutron that it shouldn't allocate floating IPs from a given net/subet object :-( hopefully this will improve in future openstack versions. A related issues was detected in T202115: cloudvps: eqiad1: review floating IP mechanisms.

However, I investigated if changing some bits in how we declare the compat network could do the trick this time, and it seems it can be done by declaring the compat net as router:external=False (as opposed to True, check https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Bootstrap#neutron).
So, we could implement this in eqiad1, but doing this change will likely impact connections main<-->eqiad1, so I didn't do yet, and we will have to coordinate and schedule this change.

For reference, this is my test in labtestn/labtest

root@labtestcontrol2003:~# nova floating-ip-pool-list
+----------------+
| name           |
+----------------+
| flattransportb |
| compat-net     |
+----------------+
root@labtestcontrol2003:~# neutron router-interface-delete cloudinstances2b-gw port=compat-port
Removed interface from router cloudinstances2b-gw.
root@labtestcontrol2003:~# neutron subnet-delete compat-subnet
Deleted subnet: compat-subnet
root@labtestcontrol2003:~# neutron net-delete compat-net
Deleted network: compat-net
root@labtestcontrol2003:~# neutron net-create compat-net --router:external=false --provider:network_type=flat --provider:physical_network=br-compat
Created a new network:
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| availability_zone_hints   |                                      |
| availability_zones        |                                      |
| created_at                | 2018-11-21T11:39:01                  |
| description               |                                      |
| id                        | 3a3bfff3-d602-43c7-9178-89d7a90545a9 |
| ipv4_address_scope        |                                      |
| ipv6_address_scope        |                                      |
| mtu                       | 1500                                 |
| name                      | compat-net                           |
| port_security_enabled     | True                                 |
| provider:network_type     | flat                                 |
| provider:physical_network | br-compat                            |
| provider:segmentation_id  |                                      |
| router:external           | False                                |
| shared                    | False                                |
| status                    | ACTIVE                               |
| subnets                   |                                      |
| tags                      |                                      |
| tenant_id                 | admin                                |
| updated_at                | 2018-11-21T11:39:01                  |
+---------------------------+--------------------------------------+
root@labtestcontrol2003:~# neutron subnet-create --gateway 10.196.16.1 --name compat-subnet --ip-version 4 --disable-dhcp --allocation-pool start=10.196.16.3,end=10.196.16.3 compat-net 10.196.16.0/24
Created a new subnet:
+-------------------+------------------------------------------------+
| Field             | Value                                          |
+-------------------+------------------------------------------------+
| allocation_pools  | {"start": "10.196.16.3", "end": "10.196.16.3"} |
| cidr              | 10.196.16.0/24                                 |
| created_at        | 2018-11-21T11:39:29                            |
| description       |                                                |
| dns_nameservers   |                                                |
| enable_dhcp       | False                                          |
| gateway_ip        | 10.196.16.1                                    |
| host_routes       |                                                |
| id                | 79c339b3-94ff-4d89-829c-44acfc9ef5cc           |
| ip_version        | 4                                              |
| ipv6_address_mode |                                                |
| ipv6_ra_mode      |                                                |
| name              | compat-subnet                                  |
| network_id        | 3a3bfff3-d602-43c7-9178-89d7a90545a9           |
| subnetpool_id     |                                                |
| tenant_id         | admin                                          |
| updated_at        | 2018-11-21T11:39:29                            |
+-------------------+------------------------------------------------+
root@labtestcontrol2003:~# neutron port-create --name compat-port --fixed-ip subnet_id=compat-subnet,ip_address=10.196.16.3 compat-net
Created a new port:
+-----------------------+------------------------------------------------------------------------------------+
| Field                 | Value                                                                              |
+-----------------------+------------------------------------------------------------------------------------+
| admin_state_up        | True                                                                               |
| allowed_address_pairs |                                                                                    |
| binding:host_id       |                                                                                    |
| binding:profile       | {}                                                                                 |
| binding:vif_details   | {}                                                                                 |
| binding:vif_type      | unbound                                                                            |
| binding:vnic_type     | normal                                                                             |
| created_at            | 2018-11-21T11:39:54                                                                |
| description           |                                                                                    |
| device_id             |                                                                                    |
| device_owner          |                                                                                    |
| dns_name              |                                                                                    |
| extra_dhcp_opts       |                                                                                    |
| fixed_ips             | {"subnet_id": "79c339b3-94ff-4d89-829c-44acfc9ef5cc", "ip_address": "10.196.16.3"} |
| id                    | 586de0ea-52cf-4429-804b-bc4b535feec9                                               |
| mac_address           | fa:16:3e:5b:f0:05                                                                  |
| name                  | compat-port                                                                        |
| network_id            | 3a3bfff3-d602-43c7-9178-89d7a90545a9                                               |
| port_security_enabled | True                                                                               |
| security_groups       | 6167edf8-87a0-457a-935d-7fd7fb6e22b2                                               |
| status                | DOWN                                                                               |
| tenant_id             | admin                                                                              |
| updated_at            | 2018-11-21T11:39:54                                                                |
+-----------------------+------------------------------------------------------------------------------------+
root@labtestcontrol2003:~# neutron router-interface-add cloudinstances2b-gw port=compat-port
Added interface 586de0ea-52cf-4429-804b-bc4b535feec9 to router cloudinstances2b-gw.
root@labtestcontrol2003:~# nova floating-ip-pool-list
+----------------+
| name           |
+----------------+
| flattransportb |
+----------------+

(of course, labtest<-->labtestn connectivity works after the changes)

aborrero moved this task from Doing to Needs discussion on the cloud-services-team (Kanban) board.

Needs discussion because:

  • if we want to fix the horizon thing we need to delete and re-create the compat net in eqiad1
  • that operation would cause some brief downtime for cross-region connections/traffic main<-->eqiad1
  • this has been already tested in the labtest/labtestn deploymetns, seems to work. But we may find additional difficulties with the new config in the 'live' deployments.
Andrew changed the task status from Open to Stalled.Nov 27 2018, 4:41 PM
Andrew added a subscriber: Andrew.

Probably we'll just leave this wrong -- it'll get fixed when we turn of the eqiad region regardless.

aborrero lowered the priority of this task from Normal to Low.Nov 27 2018, 4:41 PM
aborrero moved this task from Needs discussion to Graveyard on the cloud-services-team (Kanban) board.

We agreed on waiting to deprecate the main deployment, and then wipe the compat network.

Krenair closed this task as Declined.Nov 27 2018, 4:43 PM

Let's just decline this then.