Page MenuHomePhabricator

orchestrator: Select backend database solution
Closed, ResolvedPublic

Description

We could use m1, but the downside is that orchestrator doesn't support management of it's backend database cluster.

We've decided for now to use the tendril/zarcillo 'cluster' (db1115/db2093). As db1115 is running 10.1 (for tokudb support for tendril), we'll only create the orchestrator database on db2093 for now (which runs mariadb 10.4).

When we finally drop tendril we can upgrade db1115 to mariadb 10.4, and have replication. db2093 is already backed up, so we can always recover in case that node fails in the meantime.

Related Objects

StatusSubtypeAssignedTask
OpenNone
ResolvedKormat

Event Timeline

Kormat triaged this task as Medium priority.Oct 20 2020, 1:07 PM
Kormat created this task.
LSobanski moved this task from Triage to Refine on the DBA board.Oct 20 2020, 1:18 PM
LSobanski added a subscriber: LSobanski.

Mentioned in SAL (#wikimedia-operations) [2020-10-21T11:00:56Z] <marostegui> Upgrade db2093's mariadb version T266003

Marostegui added a subscriber: Marostegui.

Upgraded db2093 from 10.4.12 to 10.4.15
Rebooted it to pick the new kernels too.

db2093 now hosts the orchestrator database.
@Kormat the only pending thing is to decide what to do with monitoring and read_only right? As right now read_only alerts on db2093 as it is supposed to be read_only=true but that cannot longer happen as this host needs to be writable for orchestrator.

Change 636609 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] mariadb: Set both db_inventory nodes read-write

https://gerrit.wikimedia.org/r/636609

@Marostegui: correct. I've sent a CR to change the read_only status for now.

Change 636816 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db2093: Clarify it is active for orchestrator DB

https://gerrit.wikimedia.org/r/636816

Change 636816 merged by Marostegui:
[operations/puppet@production] db2093: Clarify it is active for orchestrator DB

https://gerrit.wikimedia.org/r/636816

Change 636609 merged by Kormat:
[operations/puppet@production] mariadb: Set both db_inventory nodes read-write

https://gerrit.wikimedia.org/r/636609

Marostegui closed this task as Resolved.Oct 30 2020, 11:21 AM
Marostegui assigned this task to Kormat.

This is all done!
Thanks Stevie!