Find a way to remove non-replicated tables from ToolsDB
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• Bstorm
	Oct 21 2019, 8:03 PM

Description

There are four databases that are exempt from replication because they have historically caused problems with load and replication on ToolsDB (T127164: Some databases cannot be backed up/replicated on toolsdb). This is due to database size and level of change frequency and has been documented here https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#ToolsDB_Backups_and_Replication

The goal here is to find a solution to remove the restriction of non-replicated data from the ToolsDB service in order to improve service to all users by allowing easier failover and maintenance.

s51412__data (tools.persondata)
s51071__templatetiger_p (tools.templatetiger)
~~s52721__pagecount_stats_p (tools.pagecount)~~
s51290__dpl_p (tools.dplbot)

Related Objects
Search...

Status	Assigned	Task
Resolved	• Bstorm	T216208 ToolsDB overload and cleanup
Resolved	• Bstorm	T216441 Evaluate transferring the non-replicated tables to the new toolsdb server
Resolved	fnegri	T236101 Find a way to remove non-replicated tables from ToolsDB
Open	fnegri	T291782 Migrate largest ToolsDB users to Trove
Resolved	Andrew	T292546 cloud NFS: figure out backups for cinder volumes
Resolved	aborrero	T293752 cloud ceph: refactor rbd client puppet profiles
Duplicate	None	T294429 cinder-backups: figure out automation
Resolved	aborrero	T295584 eqiad: 2 VMs for cloudbackup-dev
Resolved	aborrero	T296413 cinder: get victoria point release in the bpo repo
Resolved	aborrero	T299708 network access to eqiad ceph cluster from cloudbackup2002
Resolved	Andrew	T339830 cinder-backup getting OOM-killed for large volumes
Resolved	Andrew	T344065 Replace cinder-backup process with backy2
Resolved	Andrew	T358855 Use cloudbackup100[12]-dev for cinder backup test/dev
Resolved	Andrew	T366071 'backy2 cleanup' not getting called properly on cloudbackup hosts
Resolved	Andrew	T323502 Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool)
Resolved	Andrew	T324984 Trove volume size limit of 31Gb
Resolved	fnegri	T326754 Clarify Trove and Toolsdb usage within WMCS
Resolved	TBurmeister	T326854 Publish revised Trove user guide
Resolved	fnegri	T328691 [toolsdb] Migrate linkwatcher db to Trove
Resolved	TheresNoTime	T334491 Update Linkwatcher tool to use new Trove database
Open	fnegri	T350862 [toolsdb] Migrate mixnmatch db to Trove
Open	fnegri	T369177 [toolsdb] Migrate quickstatements db to Trove
Resolved	fnegri	T301949 ToolsDB upgrade => Bullseye, MariaDB 10.4
Resolved	taavi	T301993 [toolsdb] Enable gtid to help replication recovery
Resolved	taavi	T311509 Request increased quota for tools Cloud VPS project
Declined	dcaro	T321828 [ceph] Tune performance for DB VMs
Resolved	Andrew	T324700 Attach a Cinder volume to clouddb1001/1002
Resolved	fnegri	T328273 apt error in clouddb1001
Resolved	fnegri	T328693 [toolsdb] Migrate "s54518__mw" db to Trove
Resolved	fnegri	T329521 [toolsdb] Set up tools-db-2 to replicate tools-db-1
Resolved	fnegri	T329970 [toolsdb] set up tools-db-1 to replicate from clouddb1001
Resolved	fnegri	T330655 monitoring: Disk space check can fail to read fuse mounts
Resolved	fnegri	T333330 Disk space low in cloudvirt10[19-20]
Resolved	fnegri	T333471 Move all tools from clouddb1001 to tools-db-1
Resolved	fnegri	T333567 Move ToolsDB Grafana dashboard to grafana.wmcloud.org
Invalid	fnegri	T334627 Unable to archive tools databases
Resolved	fnegri	T334924 ToolsDB: rename Cinder volumes

Event Timeline

• Bstorm triaged this task as Medium priority.Oct 21 2019, 8:03 PM

• Bstorm created this task.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 21 2019, 8:03 PM

bd808 updated the task description. (Show Details)Oct 21 2019, 8:10 PM

• Bstorm added a parent task: T216441: Evaluate transferring the non-replicated tables to the new toolsdb server.Oct 21 2019, 8:11 PM

• Bstorm moved this task from Backlog to ToolsDB on the Data-Services board.Oct 21 2019, 9:44 PM

• Bstorm added a parent task: T212595: [Feature request] Database as a Service (Trove) for Cloud VPS projects.Oct 9 2020, 4:08 PM

• Bstorm removed a parent task: T212595: [Feature request] Database as a Service (Trove) for Cloud VPS projects.Oct 9 2020, 4:11 PM

• Bstorm added a subtask: T212595: [Feature request] Database as a Service (Trove) for Cloud VPS projects.

• Bstorm mentioned this in T263679: Plan and manage the required downtime for the clouddb-services VMs on cloudvirt1019 and cloudvirt1020.Oct 15 2020, 9:29 PM

Andrew closed subtask T212595: [Feature request] Database as a Service (Trove) for Cloud VPS projects as Resolved.Aug 24 2021, 9:30 PM

taavi edited subtasks, added: T291782: Migrate largest ToolsDB users to Trove; removed: T212595: [Feature request] Database as a Service (Trove) for Cloud VPS projects.Nov 7 2021, 5:51 PM

Mentioned in SAL (#wikimedia-cloud) [2022-02-23T17:40:27Z] <andrewbogott> restarting lots of openstack services to try to clear up the mess that is T236101

bd808 updated the task description. (Show Details)Feb 23 2022, 5:54 PM

fnegri edited projects, added cloud-services-team; removed cloud-services-team (Kanban).Jan 18 2023, 7:33 PM

fnegri moved this task from Kanban to Inbox on the cloud-services-team board.

This should no longer be a problem after we complete the migration in T301949.

fnegri added a subtask: T301949: ToolsDB upgrade => Bullseye, MariaDB 10.4.Apr 3 2023, 11:12 AM

fnegri changed the status of subtask T301949: ToolsDB upgrade => Bullseye, MariaDB 10.4 from Open to In Progress.Apr 7 2023, 5:42 PM

fnegri closed this task as Resolved.Apr 14 2023, 1:51 PM

fnegri claimed this task.

fnegri closed subtask T301949: ToolsDB upgrade => Bullseye, MariaDB 10.4 as Resolved.May 9 2023, 12:35 PM