Maniphest T192644

Errors during database initialization: Quibble does not support MySQL 5.7
Open, Needs TriagePublic
Actions

Assigned To

None

Authored By

	Lucas_Werkmeister_WMDE
	Apr 20 2018, 11:41 AM

Description

I tried to run quibble but ran into several errors during database initialization. Environment: Ubuntu 17.10 Artful, mysqld Ver 5.7.21-0ubuntu0.17.10.1, PHP 7.1.15-0ubuntu0.17.10.1

First, the mysql_install_db command complains that it is deprecated and we should “consider switching to mysqld --initialize”, and then it can’t find the mysqld binary. strace reveals that it only checks ./mysqld, ./bin/mysqld, /usr/bin/mysqld, /usr/local/bin/mysqld, /opt/mysql/bin/mysqld, /opt/mysql/server-5.7/sbin/mysqld, and /opt/mysql/server-5.7/bin/mysqld, but not /usr/sbin/mysqld where the binary actually resides, even though according to the manpage /usr/sbin/ is one of the “last resort” paths where it should look for mysqld.

But let’s take mysql_install_db’s advice and use mysqld --initialize instead:

diff --git a/quibble/backend.py b/quibble/backend.py
index 1f013b8..abb4345 100644
--- a/quibble/backend.py
+++ b/quibble/backend.py
@@ -109,7 +109,8 @@ class MySQL(BackendServer):
     def _install_db(self):
         self.log.info('Initializing MySQL data directory')
         p = subprocess.Popen([
-            'mysql_install_db',
+            'mysqld',
+            '--initialize',
             '--datadir=%s' % self.rootdir,
             '--user=%s' % pwd.getpwuid(os.getuid())[0],
             ],

But then the next problem happens – mysqld doesn’t have permission to open /var/log/mysql/error.log. And if I specify a different error log location in the command line, some other part will fail, because mysql reads the whole config in /etc/mysql/, which just isn’t tailored at all for running a different mysqld than the system service… and I’m not sure how to overcome that. (The same problem occurs if we add --mysqld-file=/usr/sbin/mysqld to mysql_install_db, by the way.)

@hashar I’m curious why you don’t get the same problem… what kind of system did you test this on? Do you have a better-suited /etc/mysql/ configuration? Is this distribution-dependent?

(And just in case I massively misunderstood something… running quibble outside of sudo and without a container is supposed to work, right? Otherwise this whole task is just inconsequential noise, sorry :) )

Related Objects

Mentioned In: T200693: Test failure in WikibaseLexeme selenium tests block merging patches to ContentTranslation

Event Timeline

Lucas_Werkmeister_WMDE created this task.Apr 20 2018, 11:41 AM

Lucas_Werkmeister_WMDE updated the task description. (Show Details)

running quibble outside of sudo and without a container is supposed to work, right?

Yes Quibble should be runnable on a local machine, a VM or whatever environment. But it does not have yet support for all variations that might exit on one installation :] Or in short: yes your bug report is absolutely valid!

the mysql_install_db command complains that it is deprecated and we should “consider switching to mysqld --initialize”

I have only tested Quibble with MariaDB from Debian and that is what the CI containers are using:

Jessie	10.0.32-0+deb8u1
Stretch	10.1.26-0+deb9u1

It is not deprecated there at least. If it doesn't look at /usr/sbin/, that might be due to it not being in $PATH or maybe they forgot to add it in. I have no clue really :]

I guess Quibble should detect the MySQL flavor and version, and tweak the command line appropriately to use --initialize. That should easy to handle.

But then the next problem happens – mysqld doesn’t have permission to open /var/log/mysql/error.log. And if I specify a different error log location in the command line, some other part will fail, because mysql reads the whole config in /etc/mysql/, which just isn’t tailored at all for running a different mysqld than the system service… and I’m not sure how to overcome that. (The same problem occurs if we add --mysqld-file=/usr/sbin/mysqld to mysql_install_db, by the way.)

Would have to test it. With MariaDB the MySQLBackend.start methods pass a few more extra flags, two of them being relative to a root directory which should be a temporary directory. In short:

self.errorlog = os.path.join(self.rootdir, 'error.log')
self.pidfile = os.path.join(self.rootdir, 'mysqld.pid')
self.socket = os.path.join(self.rootdir, 'socket')

self.server = subprocess.Popen([
    '/usr/sbin/mysqld',  # fixme drop path
    '--skip-networking',
    '--datadir=%s' % self.rootdir,
    '--log-error=%s' % self.errorlog,
    '--pid-file=%s' % self.pidfile,
    '--socket=%s' % self.socket,
    ],
    stdout=subprocess.DEVNULL,
    stderr=subprocess.DEVNULL,
)

So probably _install_db() needs to be passed similar parameters. I can try reproducing with an Ubuntu container and try to hack on it.

hashar moved this task from Backlog to Defect on the Quibble board.Apr 27 2018, 9:48 PM

hashar renamed this task from Errors during database initialization to Errors during database initialization: Quibble does not support MySQL 5.7.Jun 28 2018, 9:01 PM

Lucas_Werkmeister_WMDE mentioned this in T200693: Test failure in WikibaseLexeme selenium tests block merging patches to ContentTranslation.Jul 30 2018, 5:10 PM

Errors during database initialization: Quibble does not support MySQL 5.7Open, Needs TriagePublicActions

Description

Related Objects

Event Timeline

Errors during database initialization: Quibble does not support MySQL 5.7
Open, Needs TriagePublic
Actions