Page MenuHomePhabricator

'Tool Labs instance distribution' check failing on cloudvirt1003
Closed, ResolvedPublic

Description

root@cloudcontrol1003:/etc# /usr/local/bin/spreadcheck.py --config /usr/local/etc/spreadcheck-tools.yaml
Version 1.1 is deprecated, using alternative version 2 instead.
Found more than one valid endpoint. Use a more restrictive filter
Traceback (most recent call last):
  File "/usr/local/bin/spreadcheck.py", line 92, in <module>
    config['classifier'])
  File "/usr/local/bin/spreadcheck.py", line 39, in classify_instances
    servers = client.servers.list()
  File "/usr/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 750, in list
    "servers")
  File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 242, in _list
    resp, body = self.api.client.get(url)
  File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 452, in get
    return self._cs_request(url, 'GET', **kwargs)
  File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 408, in _cs_request
    self.authenticate()
  File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 565, in authenticate
    auth_url = self._v2_auth(auth_url)
  File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 656, in _v2_auth
    return self._authenticate(url, body)
  File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 671, in _authenticate
    return self._extract_service_catalog(url, resp, respbody)
  File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 494, in _extract_service_catalog
    self.management_url = self.get_service_url(self.service_type)
  File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 471, in get_service_url
    volume_service_name=self.volume_service_name,)
  File "/usr/lib/python2.7/dist-packages/novaclient/service_catalog.py", line 87, in url_for
    endpoints=matching_endpoints)
novaclient.exceptions.AmbiguousEndpoints: AmbiguousEndpoints: [{u'adminURL': u'http://labnet1001.eqiad.wmnet:8774/v2.1', u'region': u'eqiad', u'id': u'<redacted>', 'serviceName': u'nova', u'internalURL': u'http://labnet1001.eqiad.wmnet:8774/v2.1', u'publicURL': u'http://labnet1001.eqiad.wmnet:8774/v2.1'}, {u'adminURL': u'http://cloudcontrol1003.wikimedia.org:8774/v2.1', u'region': u'eqiad1-r', u'id': u'<redacted>', 'serviceName': u'nova', u'internalURL': u'http://cloudcontrol1003.wikimedia.org:8774/v2.1', u'publicURL': u'http://cloudcontrol1003.wikimedia.org:8774/v2.1'}]

Event Timeline

aborrero triaged this task as Medium priority.
aborrero moved this task from Inbox to Doing on the cloud-services-team (Kanban) board.
aborrero subscribed.

I think adding a region selector could help. I will take a look.

Change 479673 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudvps: make spreadcheck monitor region-aware

https://gerrit.wikimedia.org/r/479673

Change 479673 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudvps: make spreadcheck monitor region-aware

https://gerrit.wikimedia.org/r/479673

I think this is solved. Feel free to reopen if you find any additional issue.