Page MenuHomePhabricator

Bug on mariadb systemd unit on stretch for multi-instance hosts
Closed, ResolvedPublic

Description

On some version of the packages, the following options were setup:

RuntimeDirectory=mysqld
#RuntimeDirectoryPreserve=yes

This is a mistake, RunTimeDirectory shouldn't be enabled without RuntimeDirectoryPreserve=yes, and that is why this patch was enabled as a workaround until it works on buster:

746f56e57adf35a3 and followup 54969f1e7e6b70

However, the original patch for the unit file was merged, too. It should be non-problematic for single-instance hosts, but in the case of multi-instance ones, if one server is stopped or restarted, the sockets for the others will be lost in a limb, failing to be found on the visible filesystem, alerting, etc. Nothing catastrofic, but messy.

We need to patch the systemd unit on all multi-instance host so all references are removed until it is preserved on restart/stop, available on buster.

Event Timeline

jcrespo created this task.May 11 2018, 4:45 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 11 2018, 4:45 PM
jcrespo triaged this task as Medium priority.May 11 2018, 4:45 PM

Patching the unit manually doesn't work, because the state of the config at start is remembered, requiring a rolling restart.

This will require a 10.0.33-2, a systemctl daemon-reload and then a restart of all services.

Marostegui moved this task from Triage to In progress on the DBA board.May 13 2018, 1:57 PM

Change 432965 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/software@master] mariadb-package: Update systemd unit

https://gerrit.wikimedia.org/r/432965

Change 432965 merged by Jcrespo:
[operations/software@master] mariadb-package: Update systemd unit

https://gerrit.wikimedia.org/r/432965

Mentioned in SAL (#wikimedia-operations) [2018-05-16T13:17:31Z] <jynus> restarting mariadb processes on dbstore2001 T194516

jcrespo closed this task as Resolved.May 16 2018, 1:31 PM

This is now fixed on the latest package, and all affected (multiinstance) hosts, restarted. Heads up for m-i hosts in case it was missed on any CC @Marostegui

Vvjjkkii renamed this task from Bug on mariadb systemd unit on stretch for multi-instance hosts to 33caaaaaaa.Jul 1 2018, 1:10 AM
Vvjjkkii reopened this task as Open.
Vvjjkkii removed jcrespo as the assignee of this task.
Vvjjkkii raised the priority of this task from Medium to High.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed subscribers: gerritbot, Aklapper.
Marostegui renamed this task from 33caaaaaaa to Bug on mariadb systemd unit on stretch for multi-instance hosts.Jul 1 2018, 6:41 PM
Marostegui closed this task as Resolved.
Marostegui assigned this task to jcrespo.
Marostegui lowered the priority of this task from High to Medium.
Marostegui updated the task description. (Show Details)