Move PAWS to it's own database (and away from ToolsDB)
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• Bstorm
	Nov 10 2020, 8:53 PM

Description

PAWS hub uses ToolsDB as it's primary database. This isn't entirely an appropriate use of ToolsDB, which is ideally used for Toolforge tools, and ToolsDB has become an increasingly unstable service in need of replacing anyway. PAWS has relatively simple database needs and the DB could be maintained much more easily than ToolsDB overall.

The ideal would be a to use T212595: [Feature request] Database as a Service (Trove) for Cloud VPS projects. However, if we get around to it first, a couple little instances running mariadb in the PAWS cloud-vps project would be just fine.

Details

	Subject	Repo	Branch	Lines +/-
	paws trove: actually install mariadb-client	operations/puppet	production	+2 -0
	paws: add a very simple backup server	operations/puppet	production	+70 -0

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	• Bstorm	T267683 Move PAWS to it's own database (and away from ToolsDB)
Resolved	Andrew	T212595 [Feature request] Database as a Service (Trove) for Cloud VPS projects
Resolved	Andrew	T276956 Make a plan for Trove v. rabbit
Resolved	Andrew	T276960 Questions about Trove guest images
Resolved	taavi	T276961 Support Openstack Swift APIs via the radosgw
Resolved	Jhancock.wm	T289882 Q1:(Need By: TBD) rack/setup/install cloudswift100[12]
Resolved	Andrew	T347927 Better swift error messages for projects with invalid chars
Resolved	Andrew	T348885 Horizon Object Storage UI should not display for readers
Resolved	aborrero	T278505 Prepare/import debian packages for openstack trove
Resolved	aborrero	T280310 communication on port 5672 (amqp) between guest instances and cloudcontrols
Resolved	aborrero	T280835 Wrong version of Trove (13) in Victoria osbpo
Resolved	Andrew	T281654 Trove-dashboard: web app crashes when a db instance is deleted.
Resolved	Andrew	T281655 Investigate, adjust default access policies for Trove and trove-dashboard
Resolved	Andrew	T282798 Trove: don't permit hard affinity within DB clusters
Resolved	Andrew	T282809 Allow access to Trove API endpoints (port 8779) from cloud-vps instances
Resolved	Andrew	T283299 Add postgresql datastore to Trove
Resolved	Andrew	T286720 trove: Unable to manage databases on mariadb 10.6.1
Resolved	• Bstorm	T290349 Trove configurations aren't working
Resolved	Andrew	T349651 Support Trove + Swift integration
Open	Andrew	T355721 Fix 'openstack database instance rebuild'
Open	Andrew	T356291 Improve trove backup/restore

Event Timeline

• Bstorm triaged this task as High priority.Nov 10 2020, 8:53 PM

• Bstorm created this task.

Daniel_Mietchen subscribed.Nov 12 2020, 3:36 AM

Andrew moved this task from Inbox to Blocked on the cloud-services-team (Kanban) board.Apr 13 2021, 4:34 PM

We're going to do this with Trove at this point, marking that as a blocker.

The existing DB in toolsdb is < 10MB in size according to https://tool-db-usage.toolforge.org/. I've created a Trove instance of mysql that is 4 GB so it will have a lot of room to grow as well as enough quota to create a replica when we've enabled that feature in Trove. I won't move the paws system to that until we've got some operations docs and things set up.

Mentioned in SAL (#wikimedia-cloud) [2021-05-21T00:06:35Z] <bstorm> creating trove mysql instance pawsdb-1 T267683

@yuvipanda and @Chicocvenancio I have created a trove instance (mysql) on the paws project that has more than enough room for the database. We don't have replication ready in trove yet, so I plan to slap together a systemd timer to mysql dump the database on some kind of schedule until replication and trove backups are available in case we wanted to restore. I don't know how essential that is since we typically move to sqlite during database issues, but I think it makes sense to set up something like that.

I just wanted to make sure you were aware I was working on this. All that is needed is to change the hostname in secrets.yaml and it should Just Work. Decoupling this from toolsdb will mean less things that require non-volunteer support and, well, toolsdb has serious issues anyway. I may not have time to enable it this week, but that's the aim.

Change 699471 had a related patch set uploaded (by Bstorm; author: Bstorm):

[operations/puppet@production] paws: add a very simple backup server

https://gerrit.wikimedia.org/r/699471

gerritbot added a project: Patch-For-Review.Jun 11 2021, 9:25 PM

Change 699471 merged by Bstorm:

[operations/puppet@production] paws: add a very simple backup server

https://gerrit.wikimedia.org/r/699471

Maintenance_bot removed a project: Patch-For-Review.Jul 12 2021, 11:11 PM

Change 713672 had a related patch set uploaded (by Bstorm; author: Bstorm):

[operations/puppet@production] paws trove: actually install mariadb-client

https://gerrit.wikimedia.org/r/713672

gerritbot added a project: Patch-For-Review.Aug 18 2021, 6:26 PM