ToolsDB is a heavily loaded shared database. Combining many small users to a shared database makes some sense, but we should move some of its larger/heavier users to separate instances. Doing it via OpenStack Trove seems like the best solution.
Still to sort out: backups, replication?, how to enable self-provisioning of a new db by a Toolforge maintainer
ToolsDB databases ordered by size: https://tool-db-usage.toolforge.org/
Ideally we would migrate all users off of toolsdb onto Trove or something else. Anyone who isn't using any esoteric "features" of ToolsDB and who can migrate should do so.
I would like to Trove to have replication and/or backup solutions available before we move much user data there. The existing toolsdb has both backups AND replication so it's a step-backwards in terms of resiliency until we have Trove spruced up.
I opened T326754: Clarify Trove and Toolsdb usage within WMCS, as per the description, I think what was suggested here has largely come to pass. Thoughts?
I think this task can still be useful as a parent task for any migration we want to do from ToolsDB to Trove.
I propose the scope for this task becomes:
As of today, data from https://tool-db-usage.toolforge.org/
Total number of tools is 475.
| Size under | 10 MB | 100 MB | 500 MB | 1 GB | 5 GB | 10 GB | 25 GB | 50 GB |
| Number of Tools | 235 | 337 | 391 | 407 | 433 | 450 | 464 | 468 |
| Percentage of Tools | 49.47% | 70.95% | 82.32% | 85.68% | 91.16% | 94.74% | 97.68% | 98.53% |
As the table suggest, more than 90% of tools fall under the 5GB limit, and almost 95% fall under the 10GB Limit.
Choosing a 10GB cutoff would mean a migration of 25 tools to trove. A 25GB cutoff would mean a migration of 11 tools to trove.
Another early candidate for this should be https://tool-db-usage.toolforge.org/owner/s51412