cloudvirt1033 (and others?) can't reach keystone
Closed, DuplicatePublic
Actions

Assigned To

Authored By

	Andrew
	Sep 20 2023, 1:01 PM

Description

2023-09-20 12:59:15.329 3910567 WARNING nova.compute.manager [-] [instance: 58d695cd-09f9-4b43-8eea-5c494a4f3d83] Failed to deallocate network for instance; retrying. Error: Unable to establish connection to https://openstack.eqiad1.wikimediacloud.org:29696/v2.0/ports?device_id=58d695cd-09f9-4b43-8eea-5c494a4f3d83: HTTPSConnectionPool(host='openstack.eqiad1.wikimediacloud.org', port=29696): Max retries exceeded with url: /v2.0/ports?device_id=58d695cd-09f9-4b43-8eea-5c494a4f3d83 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5884185df0>: Failed to establish a new connection: [Errno 110] ETIMEDOUT')): keystoneauth1.exceptions.connection.ConnectFailure: Unable to establish connection to https://openstack.eqiad1.wikimediacloud.org:29696/v2.0/ports?device_id=58d695cd-09f9-4b43-8eea-5c494a4f3d83: HTTPSConnectionPool(host='openstack.eqiad1.wikimediacloud.org', port=29696): Max retries exceeded with url: /v2.0/ports?device_id=58d695cd-09f9-4b43-8eea-5c494a4f3d83 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5884185df0>: Failed to establish a new connection: [Errno 110] ETIMEDOUT'))

Related Objects

Mentioned In: T346651: cloudvirt: eqiad1: connect them to cloud-private

Event Timeline

Andrew created this task.Sep 20 2023, 1:01 PM

Restricted Application edited projects, added cloud-services-team; removed Patch-For-Review, cloud-services-team (FY2023/2024-Q1-Q2). · View Herald TranscriptSep 20 2023, 1:01 PM

The next place I checked (cloudvirt1052) seems to have the same issue:

2023-09-20 13:00:58.761 2602525 INFO keystoneauth.session [None req-d558ff8e-b1bb-453e-a0e4-31caade01280 - - - - - -] Failure: Unable to establish connection to https://openstack.eqiad1.wikimediacloud.org:29696/v2.0/ports?device_id=01c81b24-d2a6-4323-9bc9-98b7ff73382c&fields=binding%3Ahost_id&fields=binding%3Avif_type: HTTPSConnectionPool(host='openstack.eqiad1.wikimediacloud.org', port=29696): Max retries exceeded with url: /v2.0/ports?device_id=01c81b24-d2a6-4323-9bc9-98b7ff73382c&fields=binding%3Ahost_id&fields=binding%3Avif_type (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f03c038f580>: Failed to establish a new connection: [Errno 110] ETIMEDOUT')). Retrying in 0.5s.

curl does not work either, so a restart would not help xd

taavi claimed this task.Sep 20 2023, 1:07 PM

taavi added a subtask: T346651: cloudvirt: eqiad1: connect them to cloud-private.

taavi edited projects, added Cloud-VPS; removed Goal.

taavi mentioned this in T346651: cloudvirt: eqiad1: connect them to cloud-private.

aborrero removed a parent task: T346439: cloudlb: eqiad1: change openstack.eqiad1.wikimediacloud.org endpoint to point to cloudlb.Sep 20 2023, 2:20 PM

aborrero removed a subtask: T346651: cloudvirt: eqiad1: connect them to cloud-private.

aborrero closed this task as a duplicate of T346651: cloudvirt: eqiad1: connect them to cloud-private.