Page MenuHomePhabricator

Deploy paws-dev trove db
Closed, ResolvedPublic

Description

To keep paws-dev closer to paws (prod) we will want it to use a trove db. Though attempting to create on is currently giving an error:

Error creating DNS entry for instance: abce52b6-9921-47d9-bfcc-066fd8db4800 An unknown exception occurred

@Andrew is aware and looking into it.

Related Objects

StatusSubtypeAssignedTask
Resolvedrook
Resolvedrook
ResolvedNone
Resolvedrook
Resolvedrook
Resolvedrook
Resolvedrook
Resolvedrook
Resolvedrook
Resolvedrook
ResolvedSDunlap
OpenNone
Resolvedrook
Resolvedrook
ResolvedBUG REPORTAndrew
Resolvedrook
Resolvedrook
Resolvedrook
Resolvedrook
Resolvedrook
Resolvedrook
ResolvedNone
ResolvedNone
Resolvedrook
Resolvedrook
Resolvedrook
ResolvedAndrew

Event Timeline

Thank you for working on this!

I am getting a new error when I try to build the db.

Error: Unable to retrieve details for database instance: c3858738-65c2-426e-a72a-711828486428 Details
Internal Server Error. Please keep this ID to help us figure out what went wrong: (18e032f3-d6e8-4067-ab63-3ff8d5022e8a). (HTTP 500)

More notes from the cli

root@cloudcontrol2001-dev:~# trove show paws-dev
+-------------------------+-------------------------------------------------------------------------------------------------------------------------+
| Property                | Value                                                                                                                   |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------+
| access                  | {}                                                                                                                      |
| created                 | 2023-01-05T22:15:18                                                                                                     |
| datastore               | mysql                                                                                                                   |
| datastore_version       | 5.7.29                                                                                                                  |
| encrypted_rpc_messaging | True                                                                                                                    |
| fault                   | Failed to get server bd0d4c04-b87e-478c-9d31-c1805dbf1169 for instance c3858738-65c2-426e-a72a-711828486428, error:     |
|                         |     Unexpected API Error.                                                                                               |
|                         | Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.                          |
|                         | <class 'sqlalch...                                                                                                      |
| fault_date              | 2023-01-05T22:33:27                                                                                                     |
| fault_details           | Traceback (most recent call last):                                                                                      |
|                         |   File "/usr/lib/python3/dist-packages/trove/taskmanager/models.py", line 795, in _service_is_active                    |
|                         |     server = self.nova_client.servers.get(c_id)                                                                         |
|                         |   File "/usr/lib/python3/dist-packages/novaclient/v2/servers.py", line 866, in get                                      |
|                         |     return self._get("/servers/%s" % base.getid(server), "server")                                                      |
|                         |   File "/usr/lib/python3/dist-packages/novaclient/base.py", line 352, in _get                                           |
|                         |     resp, body = self.api.client.get(url)                                                                               |
|                         |   File "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 395, in get                                      |
|                         |     return self.request(url, 'GET', **kwargs)                                                                           |
|                         |   File "/usr/lib/python3/dist-packages/novaclient/client.py", line 78, in request                                       |
|                         |     raise exceptions.from_response(resp, body, url, method)                                                             |
|                         | novaclient.exceptions.ClientException: Unexpected API Error.                                                            |
|                         | Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.                          |
|                         | <class 'sqlalchemy.exc.OperationalError'>                                                                               |
|                         | (HTTP 500) (Request-ID: req-cfa2a060-afc4-4e06-867b-54e9f68fc7d7)                                                       |
|                         |                                                                                                                         |
|                         | During handling of the above exception, another exception occurred:                                                     |
|                         |                                                                                                                         |
|                         | Traceback (most recent call last):                                                                                      |
|                         |   File "/usr/lib/python3/dist-packages/trove/taskmanager/models.py", line 434, in wait_for_instance                     |
|                         |     utils.poll_until(self._service_is_active,                                                                           |
|                         |   File "/usr/lib/python3/dist-packages/trove/common/utils.py", line 223, in poll_until                                  |
|                         |     return wait_for_task(task)                                                                                          |
|                         |   File "/usr/lib/python3/dist-packages/trove/common/utils.py", line 207, in wait_for_task                               |
|                         |     return polling_task.wait()                                                                                          |
|                         |   File "/usr/lib/python3/dist-packages/eventlet/event.py", line 125, in wait                                            |
|                         |     result = hub.switch()                                                                                               |
|                         |   File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 313, in switch                                       |
|                         |     return self.greenlet.switch()                                                                                       |
|                         |   File "/usr/lib/python3/dist-packages/oslo_service/loopingcall.py", line 150, in _run_loop                             |
|                         |     result = func(*self.args, **self.kw)                                                                                |
|                         |   File "/usr/lib/python3/dist-packages/trove/common/utils.py", line 194, in poll_and_check                              |
|                         |     obj = retriever()                                                                                                   |
|                         |   File "/usr/lib/python3/dist-packages/trove/taskmanager/models.py", line 801, in _service_is_active                    |
|                         |     raise TroveError(                                                                                                   |
|                         | trove.common.exception.TroveError:                                                                                      |
|                         |     Failed to get server bd0d4c04-b87e-478c-9d31-c1805dbf1169 for instance c3858738-65c2-426e-a72a-711828486428, error: |
|                         |     Unexpected API Error.                                                                                               |
|                         | Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.                          |
|                         | <class 'sqlalchemy.exc.OperationalError'>                                                                               |
|                         | (HTTP 500) (Request-ID: req-cfa2a060-afc4-4e06-867b-54e9f68fc7d7)                                                       |
|                         |                                                                                                                         |
| flavor                  | 5b2ca632-2ea0-4007-9b40-4f84f8e2428b                                                                                    |
| hostname                | 27ypfmkyjgt.svc.trove.codfw1dev.wikimedia.cloud                                                                         |
| id                      | c3858738-65c2-426e-a72a-711828486428                                                                                    |
| name                    | paws-dev                                                                                                                |
| operating_status        |                                                                                                                         |
| region                  | codfw1dev-r                                                                                                             |
| server_id               | bd0d4c04-b87e-478c-9d31-c1805dbf1169                                                                                    |
| service_status_updated  | 2023-01-05T22:33:27                                                                                                     |
| status                  | ERROR                                                                                                                   |
| tenant_id               | paws-dev                                                                                                                |
| updated                 | 2023-01-05T22:33:27                                                                                                     |
| volume                  | 1                                                                                                                       |
| volume_id               | f012c999-4db7-41c6-b30e-573120f1e521                                                                                    |
| volume_used             | 0.01                                                                                                                    |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------+

There are errors reported in the nova-api-wsgi log, perhaps primarily

pymysql.err.OperationalError: (1054, "Unknown column 'block_device_mapping.encrypted' in 'field list'")