Page MenuHomePhabricator

Let quarry use the mariadb module
Closed, ResolvedPublic

Description

In production the module "mariadb" is used for DBs. We also have the module "mysql" which is older but not used by any prod DBs nowadays. Just few misc (cloud) things are still using the mysql module.

We would like to remove the mysql module entirely (--> T162070)

quarry is one of the few things using this last time we checked.

So this task is to make quarry switch to the other module. In the long-term we only want to support one module for DBs.

Another thing is that the mysql module shows up as one of the default classes for cloud VPS users for historic reasons but nowadays doesn't work anymore. (It tries to use precise/trusty APT sources etc). So that should also be replaced there. And we need to find out what exactly uses the module besides quarry.

Event Timeline

Ottomata triaged this task as Medium priority.Jan 16 2018, 8:09 PM

@Dzahn @jcrespo Any hint on what is the current equivalent of https://github.com/wikimedia/puppet/blob/production/modules/quarry/manifests/database.pp:

class quarry::database {
    $data_path = '/srv/mysql/data'

    class { '::mysql::server':
        package_name => 'mariadb-server',
        config_hash  => {
            'datadir'      => $data_path,
            'bind_address' => '0.0.0.0',
        },
    }
}

We are currently investigating on upgrading the whole quarry system and would be happy to test out changes. CC @Framawiki

Framawiki renamed this task from let quarry use the mariadb module to Let quarry use the mariadb module.Aug 20 2018, 7:03 PM
Framawiki updated the task description. (Show Details)

@zhuyifei1999 I think that would be a combination of:

class {'mariadb::packages_wmf':
class {'mariadb::service':

and

class { 'mariadb::config':

as used inside role/manifests/mariadb/core.pp

(but in a new role class)

Actually, instead of:

class {'mariadb::packages_wmf':
class {'mariadb::service':

class { 'mariadb::packages' should work to install the regular service (ping if it doesn't), or just install with puppet the mariadb-server package if the rest of the packages are not useful. We don't maintain regular installations very well, but I think class { 'mariadb': } should be doing the things you want by default, but maybe we should remove the monitoring.

We may need to update packages.pp to install mariadb-server metapackage (it currently hardcodes 5.5, but just because it was not being used).

Change 454217 had a related patch set uploaded (by Zhuyifei1999; owner: Zhuyifei1999):
[operations/puppet@production] mariadb::packages: Unversion server and client

https://gerrit.wikimedia.org/r/454217

Change 454217 merged by Jcrespo:
[operations/puppet@production] mariadb::packages: Unversion server and client

https://gerrit.wikimedia.org/r/454217

Change 454481 had a related patch set uploaded (by Zhuyifei1999; owner: Zhuyifei1999):
[operations/puppet@production] quarry::database: Use mariadb module instead of mysql module

https://gerrit.wikimedia.org/r/454481

Change 454481 merged by Bstorm:
[operations/puppet@production] quarry::database: Use mariadb instead of mysql module

https://gerrit.wikimedia.org/r/454481

This might be resolved now since the merge above. Much appreciated, thank you @zhuyifei1999 and @Bstorm

Did that already actively switch it?

The new instance quarry-db-01 is now on mariadb, but the old instance quarry-main-01 (with puppet disabled) is still on mysql. If everything goes well I think we can delete the old instance, hopefully by tomorrow.

why not modify mariadb::packages and use that instead? Was there a blocker for that? Packages is supposed to be mostly for non mw core db installations.

You may also want to include mariadb::service with managed => true so mariadb auto starts at the begining?

Both are not blockers, I am just asking and/or suggesting improvements.

why not modify mariadb::packages and use that instead? Was there a blocker for that? Packages is supposed to be mostly for non mw core db installations.

https://integration.wikimedia.org/ci/job/operations-puppet-tests-docker/27310/console happened:

19:18:43 wmf-style: total violations delta 1
19:18:43 NEW violations:
19:18:43 modules/profile/manifests/quarry/database.pp:6 wmf-style: profile 'profile::quarry::database' includes non-profile class mariadb::packages

You may also want to include mariadb::service with managed => true so mariadb auto starts at the begining?

Looking at mariadb::service it seems to want WMF packages for mariadb. Is it compatible with mariadb::packages? mariadb-server package seems to enable mariadb.service unit by default afaict.

profile 'profile::quarry::database' includes non-profile class mariadb::packages

That is probably pedantic, to use class { 'mariadb::packages': } instead, I guess.

Is it compatible with mariadb::packages

It should be, but I haven't cheched 100%.

Note all those are not requirements, just suggestions, feel free to use the classes in any way you want.

Change 460416 had a related patch set uploaded (by Zhuyifei1999; owner: Zhuyifei1999):
[operations/puppet@production] quarry::database: Use class declaration for mariadb::packages

https://gerrit.wikimedia.org/r/460416

Change 460416 merged by Dzahn:
[operations/puppet@production] quarry::database: Use class declaration for mariadb::packages

https://gerrit.wikimedia.org/r/460416

zhuyifei1999 claimed this task.