Page MenuHomePhabricator
Create Task
Maniphest T286720

trove: Unable to manage databases on mariadb 10.6.1
Closed, ResolvedPublic
Actions

Description

On my metricsinfra-db-1 trove instance (running mariadb 10.6.1) Trove fails to load the database list and instead fails with this error message:

Error: Unable to get databases data. Details
An error occurred communicating with the guest: 'utf8mb3_general_ci' not a valid collation.
Traceback (most recent call last):
File "/opt/guest-agent-venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming res = self.dispatcher.dispatch(message)
File "/opt/guest-agent-venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch return self._do_dispatch(endpoint, method, ctxt, args)
File "/opt/guest-agent-venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch result = func(ctxt, **new_args)
File "/opt/guest-agent-venv/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper result = f(*args, **kwargs)
File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/manager.py", line 798, in list_databases return self.adm.list_databases(limit, marker, include_marker)
File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/guestagent/datastore/mysql_common/service.py", line 333, in list_databases collate=database[2])
File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/common/db/mysql/models.py", line 46, in __init__ self.collate = collate
File "/opt/guest-agent-venv/lib/python3.6/site-packages/trove/common/db/mysql/models.py", line 85, in collate raise ValueError(_("'%s' not a valid collation.") % value)
ValueError: 'utf8mb3_general_ci' not a valid collation. . (HTTP 400)

The list of allowed collations seems to be coming from this list

Related Objects
Search...

Event Timeline

taavi created this task.Jul 15 2021, 1:32 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 15 2021, 1:32 PM
taavi added a parent task: T212595: [Feature request] Database as a Service (Trove) for Cloud VPS projects.Jul 15 2021, 1:56 PM
Andrew subscribed.Jul 15 2021, 3:17 PM

Thanks for the report! Trove support is spotty for different DB backends (I don't think anyone is keeping track of api/cli changes). In the short-term I'm inclined to just yank support for this version of mariadb, assuming that 10.5.1 works properly. (Either way I need to file an upstream bug for this before we close the phab issue).

Andrew added a comment.Jul 16 2021, 4:17 PM

I've confirmed that 10.5.1 works properly and was able to reproduce the issue with 10.6. I'm not sure if it's better to remove support or not; it seems like the other features work for 10.6 and many users are likely to manage their DBs via the CLI in any case.

Still needs an upstream bug though!

Andrew added a comment.Jul 16 2021, 4:28 PM

Upstream task: https://storyboard.openstack.org/#!/story/2009060

bd808 subscribed.Jul 16 2021, 4:30 PM

Mostly curious, where did the attempted use of 'utf8mb3_general_ci' come from? MySQL has a pretty horrible track record with UTF-8. The 'utf8mb3' encoding has historically been aliased as 'utf8' by MySQL, but it is not really what most people expect from a UTF-8 encoding. The 'utf8mb4' encoding is a UTF-8 encoding that can encode glyphs up to 4 bytes long. The 4th byte is needed to encode supplementary characters (think emoji).

taavi added a project: Upstream.Jul 16 2021, 4:46 PM
taavi moved this task from Backlog to Reported Upstream on the Upstream board.
Andrew closed this task as Resolved.Jul 16 2021, 4:48 PM
Andrew claimed this task.

I've removed the 10.6 option for now.

Andrew added a comment.Jul 16 2021, 6:41 PM
In T286720#7217869, @bd808 wrote:

Mostly curious, where did the attempted use of 'utf8mb3_general_ci' come from? MySQL has a pretty horrible track record with UTF-8. The 'utf8mb3' encoding has historically been aliased as 'utf8' by MySQL, but it is not really what most people expect from a UTF-8 encoding. The 'utf8mb4' encoding is a UTF-8 encoding that can encode glyphs up to 4 bytes long. The 4th byte is needed to encode supplementary characters (think emoji).

I don't know where it's coming from. That error showed up before @Majavah did anything at all -- it's from some interaction between upstream Trove and the mariadb container from Dockerhub.

• Bstorm subscribed.Jul 16 2021, 6:53 PM

You can set a global config either online with the shell or via a configuration set in horizon. If needed, we could provide recommended config sets. I am hoping to do that one way or another.

Andrew added a comment.Jul 16 2021, 9:13 PM
In T286720#7218329, @Bstorm wrote:

You can set a global config either online with the shell or via a configuration set in horizon. If needed, we could provide recommended config sets. I am hoping to do that one way or another.

I would have to read and think a lot before knowing what would be a good global config but I'm all for us setting different global configs.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits