Neutron API not properly exposed?
Closed, ResolvedPublic

Description

Example code
from keystoneclient.session import Session as KeystoneSession
from keystoneclient.auth.identity.v3 import Password as KeystonePassword
from neutronclient.v2_0 import client as neutronclient

def get_keystone_session(project):
    return KeystoneSession(auth=KeystonePassword(
        auth_url="http://cloudcontrol1003.wikimedia.org:5000/v3",
        username="novaobserver",
        password='Fs6Dq2RtG8KwmM2Z', # read-only guest account, password is public
        project_name=project,
        user_domain_name='default',
        project_domain_name='default'
    ))

session = get_keystone_session('bastion')
client = neutronclient.Client(session=session, region_name='eqiad1-r')
print(client)
print(client.list_quotas())

Gives this:

<neutronclient.v2_0.client.Client object at 0x7f0d975d3510>
Traceback (most recent call last):
  File "testNeutronStuff.py", line 18, in <module>
    print(client.list_quotas())
  File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 97, in with_params
    ret = self.function(instance, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 612, in list_quotas
    return self.get(self.quotas_path, params=_params)
  File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 358, in get
    headers=headers, params=params)
  File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 335, in retry_request
    headers=headers, params=params)
  File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 286, in do_request
    resp, replybody = self.httpclient.do_request(action, method, body=body)
  File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 306, in do_request
    return self.request(url, method, **kwargs)
  File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 294, in request
    resp = super(SessionClient, self).request(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 98, in request
    return self.session.request(url, method, **kwargs)
  File "/usr/lib/python2.7/dist-packages/positional/__init__.py", line 94, in inner
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 405, in request
    resp = send(**kwargs)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 449, in _send_request
    raise exceptions.ConnectionRefused(msg)
keystoneauth1.exceptions.connection.ConnectFailure: Unable to establish connection to http://cloudcontrol1003.wikimedia.org:9696/v2.0/quotas.json

I'm guessing either the neutron API is not properly exposed, or keystone is telling us the wrong port for neutron, or something.

Krenair created this task.Thu, Dec 6, 11:10 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptThu, Dec 6, 11:10 PM
bd808 added a subscriber: bd808.

Probably a couple of issues to fix here. I'm guessing that @Krenair tested this from a host in Cloud Services. It doesn't look like the firewall rules for port 9696 on cloudcontrol1003 allow 172.16.0.0/21 or 10.196.16.0/21 like ports 9292, 5000, and 8744 do. profile::openstack::base::neutron::service has a ferm rule for prod networks, but none for public networks:

ferm::rule{'neutron-server-api':
    ensure => 'present',
    rule   => "saddr (${prod_networks}) proto tcp dport (9696) ACCEPT;",
}

I tried running the example script from a host in an allowed network and got a different error:

neutronclient.common.exceptions.Forbidden: User does not have admin privileges:
Only admin can view or configure quota.

A bit of googling to try and find out what RBAC rule to add to neutron's policy.json led me to https://bugs.launchpad.net/neutron/+bug/1671448 and two unmerged patches (https://review.openstack.org/#/c/507446 & https://review.openstack.org/#/c/505155). I'm tagging as Upstream because of that half of the problem.

I have just (accidentally) assigned myself to https://bugs.launchpad.net/neutron/+bug/1671448. In the meantime I'm not sure how best to approach this :( I can fix the firewall rules for neutron but that won't get us the ability to actually query quota usage.

Yeah let's treat the ACL thing as a separate problem and focus on getting the networking/iptables part correct here.

Change 478786 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Neutron: allow VMs to access the neutron API

https://gerrit.wikimedia.org/r/478786

aborrero claimed this task.Wed, Dec 12, 2:00 PM
aborrero moved this task from Inbox to Doing on the cloud-services-team (Kanban) board.

Change 478786 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] Neutron: allow VMs to access the neutron API

https://gerrit.wikimedia.org/r/478786

The original issue has been solved, however:

<neutronclient.v2_0.client.Client object at 0x7f81d0325b50>
Traceback (most recent call last):
  File "t.py", line 18, in <module>
    print(client.list_quotas())
  File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 97, in with_params
    ret = self.function(instance, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 612, in list_quotas
    return self.get(self.quotas_path, params=_params)
  File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 358, in get
    headers=headers, params=params)
  File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 335, in retry_request
    headers=headers, params=params)
  File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 298, in do_request
    self._handle_fault_response(status_code, replybody, resp)
  File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 273, in _handle_fault_response
    exception_handler_v20(status_code, error_body)
  File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 84, in exception_handler_v20
    request_ids=request_ids)
neutronclient.common.exceptions.Forbidden: User does not have admin privileges: Only admin can view or configure quota.
Neutron server returns request_ids: ['req-4fd56660-d911-46e2-90a6-1e014911e5ee']
aborrero closed this task as Declined.Wed, Dec 12, 2:27 PM
aborrero triaged this task as Low priority.

I don't think we can allow the GET in the API policy because doing so will also allow PUT and DELETE:

https://developer.openstack.org/api-ref/network/v2/?expanded=#update-quota-for-a-project

And we don't want projects to be able to update their own quotas.

Feel free to reopen if you think we can do anything else.

Krenair changed the task status from Declined to Resolved.Wed, Dec 12, 3:34 PM

This task is about the ability to contact the Neutron API only. That problem has been solved. I have not yet opened a ticket about the permissions around viewing quotas, but when I do it will be Upstream and not declined.