Page MenuHomePhabricator

Standardize/centralize mapping from section to mariadb port/socket and prom-mysql-exporter port
Open, MediumPublic

Description

For single-instance hosts, this is simple. The mariadb port and unix socket are :3306 and /run/mysqld/mysqld.sock, and the prom-mysql-exporter runs on :9104. For multi-instance hosts, this gets more complicated.

Each section has an 'index' that is not explicitly documented (that i could find, in any case). s1 has index 1, s8 has index 8, m5 has index 15, etc. The mariadb port for a multi-instance host is 3310 + the index of the section. So x1 (which has index 10) runs mariadb on port :3320. The prom-mysql-exporter port is 13310 + the index of the section (so staging with an index of 40 has a prom exporter port of :13350).

Mapping from a section to a mariadb unix socket is straight-forward, with one catch. The socket is /run/mysqld/mysqld.<section name>.sock. The problem, however, is that if the section name has a - in it this will break due to systemd's escaping rules. Currently we do not handle this special-case anywhere.

The above rules are implicitly encoded or relied upon in many places:

  • Puppet:
    • All the multiinstance mariadb puppet profiles. E.g. modules/profile/manifests/mariadb/dbstore_multiinstance.pp (link)
    • conftool-data/dbconfig-instance/instances.yaml (link)
    • modules/mariadb/files/check_mariadb.py (link)
    • modules/profile/files/prometheus/mysqld_exporter_config.py (link)
    • modules/profile/files/mariadb/backup_mariadb.py (link)
    • modules/profile/files/mariadb/remote_backup_mariadb.py (link)
    • Backup configs. E.g. modules/profile/templates/mariadb/backup_config/cumin1001.cnf.erb (link)
  • wmfmariadbpy:
    • wmfmariadbpy/WMFMariaDB.py (link)
    • wmfmariadbpy/WMFBackup.py (link)

Ideally, this mapping would be precisely defined in exactly one place, but even one place per language (puppet and python) would be a massive improvement. Centralising this would also have the benefit that we could fix things like the mapping of sections that contain - to unix sockets in a way that makes systemd happy.

Event Timeline

Kormat created this task.Jul 3 2020, 10:27 AM
Kormat moved this task from Triage to Backlog on the DBA board.
herron triaged this task as Medium priority.Mon, Jul 27, 1:44 PM

Change 620291 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/software/wmfmariadbpy@master] wmfmariadbpy: Load and provide a method for section to port assignment

https://gerrit.wikimedia.org/r/620291