Page MenuHomePhabricator

Trove configurations aren't working
Closed, ResolvedPublic

Description

There are plenty of reasons why you want a persistently customized database configuration. That's why that's a core feature of Trove. Strangely, we are unable to use the configuration feature (always needing to accept the default) because you cannot get a list of parameters from the trove API. I'm not sure if this is configuration, missing services, a bug or a policy/permissions issue.

This is observable in Horizon, but at the CLI level, to reproduce:
This is a valid datastore version ID:

[bstorm@cloudcontrol1003]:~ $ sudo wmcs-openstack datastore version show 379d8765-8503-4286-a046-0a4c8f8f745a
+-----------+--------------------------------------+
| Field     | Value                                |
+-----------+--------------------------------------+
| active    | True                                 |
| datastore | a5eb0629-1f06-4845-9ab4-b89f6eb622f4 |
| id        | 379d8765-8503-4286-a046-0a4c8f8f745a |
| image     | 6c715a45-ccd2-4e61-b23e-f8577ea1137d |
| name      | 10.5.10                              |
| packages  |                                      |
+-----------+--------------------------------------+

When I look for params:

[bstorm@cloudcontrol1003]:~ $ sudo wmcs-openstack database configuration parameter list --datastore mariadb 379d8765-8503-4286-a046-0a4c8f8f745a

[bstorm@cloudcontrol1003]:~ $

Without changing project, I get this from verbose mode:

http://openstack.eqiad1.wikimediacloud.org:8779 "GET /v1.0/admin/datastores/versions/379d8765-8503-4286-a046-0a4c8f8f745a/parameters HTTP/1.1" 200 32
RESP: [200] Connection: close Content-Length: 32 Content-Type: application/json Date: Fri, 03 Sep 2021 18:20:01 GMT
RESP BODY: {"configuration-parameters": []}

clean_up ListDatabaseConfigurationParameters:
END return value: 0

Changing to trove project changes nothing.

Also, it returns this only for valid parameters (here searching by name instead of ID):

[bstorm@cloudcontrol1003]:~ $ sudo wmcs-openstack --os-project-id trove database configuration parameter list --datastore mariadb 10.5.10

[bstorm@cloudcontrol1003]:~ $ sudo wmcs-openstack --os-project-id trove database configuration parameter list --datastore mariadb 10.5.1
Datastore version '10.5.1' cannot be found. (HTTP 400)

So it finds the datastore and either cannot or won't reveal any available parameters to configure.

Event Timeline

So, in that case, I propose adding these params to mariadb and mysql

innodb_buffer_pool_size, integer, max size of 57671680 (because it needs a max) and restart_required as well as wait_timeout set sanely (https://mariadb.com/kb/en/server-system-variables/#wait_timeout)

Mentioned in SAL (#wikimedia-cloud) [2021-09-03T19:14:54Z] <bstorm> adding config group validation rules for mariadb 10.5.10 T290349

Mentioned in SAL (#wikimedia-cloud) [2021-09-03T19:19:24Z] <bstorm> adding config group validation rules for postgresql and mysql T290349

Bstorm claimed this task.

These three commands made it all work (run individually):

trove-manage db_load_datastore_config_parameters mysql 5.7.29 /usr/lib/python3/dist-packages/trove/templates/mysql/validation-rules.json
trove-manage db_load_datastore_config_parameters mariadb 10.5.10 /usr/lib/python3/dist-packages/trove/templates/mariadb/validation-rules.json
trove-manage db_load_datastore_config_parameters postgresql 12.7 /usr/lib/python3/dist-packages/trove/templates/postgresql/validation-rules.json

The manual step is mentioned on step 5 here https://docs.openstack.org/trove/victoria/admin/datastore.html#create-datastore.
It is easily missed. It seems to be supported for all datastores now despite what many docs around the internet say about it being mysql only.