orchestrator: Select backend database solution
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• Kormat
	Oct 20 2020, 1:07 PM

Description

We could use m1, but the downside is that orchestrator doesn't support management of it's backend database cluster.

We've decided for now to use the tendril/zarcillo 'cluster' (db1115/db2093). As db1115 is running 10.1 (for tokudb support for tendril), we'll only create the orchestrator database on db2093 for now (which runs mariadb 10.4).

When we finally drop tendril we can upgrade db1115 to mariadb 10.4, and have replication. db2093 is already backed up, so we can always recover in case that node fails in the meantime.

Details

	Subject	Repo	Branch	Lines +/-
	mariadb: Set both db_inventory nodes read-write	operations/puppet	production	+4 -1
	db2093: Clarify it is active for orchestrator DB	operations/puppet	production	+2 -0

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		None	T265990 orchestrator: Puppetize
		Resolved		• Kormat	T266003 orchestrator: Select backend database solution

Event Timeline

• Kormat triaged this task as Medium priority.Oct 20 2020, 1:07 PM

• Kormat created this task.

LSobanski moved this task from Triage to Refine on the DBA board.Oct 20 2020, 1:18 PM

LSobanski subscribed.

Mentioned in SAL (#wikimedia-operations) [2020-10-21T11:00:56Z] <marostegui> Upgrade db2093's mariadb version T266003

Upgraded db2093 from 10.4.12 to 10.4.15
Rebooted it to pick the new kernels too.

db2093 now hosts the orchestrator database.
@Kormat the only pending thing is to decide what to do with monitoring and read_only right? As right now read_only alerts on db2093 as it is supposed to be read_only=true but that cannot longer happen as this host needs to be writable for orchestrator.

Change 636609 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] mariadb: Set both db_inventory nodes read-write

https://gerrit.wikimedia.org/r/636609

gerritbot added a project: Patch-For-Review.Oct 27 2020, 8:56 AM

@Marostegui: correct. I've sent a CR to change the read_only status for now.

Change 636816 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db2093: Clarify it is active for orchestrator DB

https://gerrit.wikimedia.org/r/636816

Change 636816 merged by Marostegui:
[operations/puppet@production] db2093: Clarify it is active for orchestrator DB

https://gerrit.wikimedia.org/r/636816

Marostegui added a project: Orchestrator.Oct 28 2020, 9:17 AM

LSobanski removed a project: Data-Persistence.Oct 28 2020, 2:04 PM

Change 636609 merged by Kormat:
[operations/puppet@production] mariadb: Set both db_inventory nodes read-write

https://gerrit.wikimedia.org/r/636609

Maintenance_bot removed a project: Patch-For-Review.Oct 30 2020, 11:11 AM