We've talked about this in the past, but never actually made a task. The upcoming Stretch deprecation seems like a good opportunity to do it.
ToolsDB is a heavily loaded shared database. Combining many small users to a shared database makes some sense, but we should move some of its larger/heavier users to separate instances. Doing it via OpenStack Trove seems like the best solution.
Still to sort out: backups, replication?, how to enable self-provisioning of a new db by a Toolforge maintainer
Ideally we would migrate all users off of toolsdb onto Trove or something else. Anyone who isn't using any esoteric "features" of ToolsDB and who can migrate should do so.
I would like to Trove to have replication and/or backup solutions available before we move much user data there. The existing toolsdb has both backups AND replication so it's a step-backwards in terms of resiliency until we have Trove spruced up.
I opened T326754: Clarify Trove and Toolsdb usage within WMCS, as per the description, I think what was suggested here has largely come to pass. Thoughts?
I think this task can still be useful as a parent task for any migration we want to do from ToolsDB to Trove.
I propose the scope for this task becomes:
As of today, data from https://tool-db-usage.toolforge.org/
Total number of tools is 475.
| Size under | 10 MB | 100 MB | 500 MB | 1 GB | 5 GB | 10 GB | 25 GB | 50 GB |
| Number of Tools | 235 | 337 | 391 | 407 | 433 | 450 | 464 | 468 |
| Percentage of Tools | 49.47% | 70.95% | 82.32% | 85.68% | 91.16% | 94.74% | 97.68% | 98.53% |
As the table suggest, more than 90% of tools fall under the 5GB limit, and almost 95% fall under the 10GB Limit.
Choosing a 10GB cutoff would mean a migration of 25 tools to trove. A 25GB cutoff would mean a migration of 11 tools to trove.
Another early candidate for this should be https://tool-db-usage.toolforge.org/owner/s51412