CloudVPS: codfw1dev: database backup for clouddb2001-dev.codfw.wmnet
Closed, InvalidPublic
Actions

Assigned To

None

Authored By

	• aborrero
	Aug 1 2019, 9:14 AM

Description

As far as I know, the clouddb2001-dev.codfw.wmnet is not getting any backup or replication or anything that would allow us to recover from a data disaster.
This was not urgent at first since this is mostly the database for a non-customer-facing openstack deployment. But anyway, having backups of some sorts would save us a lot of work/time in the case we need to recover from a outage.
I suggest we evaluate the situation and decide what do we want to do here.

I may be wrong and there is some automagical mechanism for backups that I'm not aware of.
CC'ing @Marostegui and @jcrespo for advice.

Related Objects
Search...

Status	Assigned	Task
Resolved	• aborrero	T217891 CloudVPS: rework codfw deployments
Resolved	• aborrero	T219626 codfw1dev: bootstrap cloudcontrol servers in mitaka/stretch
Resolved	Andrew	T220096 codfw1dev: decide which DBs to reallocate to clouddb2001-dev
Invalid	None	T229559 CloudVPS: codfw1dev: database backup for clouddb2001-dev.codfw.wmnet
Resolved	Andrew	T224528 rack/setup codfw: cloudbackup2001.codfw.wmnet and cloudbackup2002.codfw.wmnet
Resolved	• aborrero	T234683 Build, package bdsync for Buster
Resolved	Andrew	T245808 Ferm rules for cloudbackup2001/2001
Resolved	Andrew	T336060 cloudbackup2001 lockup on 2023-05-05

Event Timeline

• aborrero created this task.Aug 1 2019, 9:14 AM

• aborrero triaged this task as Medium priority.

• aborrero moved this task from Inbox to Soon! on the cloud-services-team (Kanban) board.

We are not backuping anything related to that host, I didn't even know that host existed :-)
Given that the data is tiny:

root@clouddb2001-dev:/var/lib/mysql# du -sh .
5.9G	.

You can probably set up a cronjob running a mysqldump and placing it somewhere you trust to get those databases backed-up.

We do backup all the databases in db1073 (m5) which look similar to these, but I guess this is just development and the other one is production, right?

In T229559#5383746, @Marostegui wrote:

We do backup all the databases in db1073 (m5) which look similar to these, but I guess this is just development and the other one is production, right?

Correct. This is a "staging" cluster for OpenStack changes. The database here is one that we had hopes to put into T218570: DB planning: include a writeable (?) misc DB cluster in codfw for WMCS.

Backups could be sent to the new cloudbackup2xxx hosts once they are actually live (T224528: rack/setup codfw: cloudbackup2001.codfw.wmnet and cloudbackup2002.codfw.wmnet). There might be somewhere on the current labstore2xxx hosts to put the data in the meantime.

• bd808 added a subtask: T224528: rack/setup codfw: cloudbackup2001.codfw.wmnet and cloudbackup2002.codfw.wmnet.Sep 11 2019, 3:25 PM

Andrew closed subtask T224528: rack/setup codfw: cloudbackup2001.codfw.wmnet and cloudbackup2002.codfw.wmnet as Resolved.Nov 18 2019, 9:15 PM

fnegri edited projects, added cloud-services-team; removed cloud-services-team (Kanban).Jan 18 2023, 7:18 PM

fnegri moved this task from Kanban to Soon! on the cloud-services-team board.

this host no longer exists. We should devise a similar backup plan for the replacement, clouddb2002-dev.

fnegri mentioned this in T369308: Decommission clouddb2002-dev.codfw.wmnet.Jul 4 2024, 4:25 PM

As passed labtestwikitech, so passed these clouddb hosts.