Page MenuHomePhabricator

[X1][Epic] Move Wikibase's entity usage database tables to the extension database cluster
Open, Needs TriagePublic

Description

In order to improve the database space usage, wbc_entity_usage tables of all wikis are to be moved to the "extension" database cluster (x1).

Steps involved (rough list first, serving as a base for breakdown into subtasks)

  • Adjust Wikibase logic to have a separate database configuration for the entity usage storage: T424185
  • wbc_entity_usage tables of all wikis are moved to x1 cluster: T424440
  • switch the Wikibase config to point to x1 for entity usage storage
    • Details of rollout strategy to be defined, might want some gradual rollout etc

Event Timeline

Regarding your question: It depends on how it is being implemented. The tables are too big to be moved manually. So you have to either 1- just create them empty and let them repopulate organically 2- Write a maint script to populate it even if it's slightly outdated. Either way, the only thing SRE will do is just creating the tables.

Thanks for confirming @Ladsgroup. I have been (naively) hopeful we could get around "backfilling" the tables given as we got lucky with that when moving Wikidata's term-related tables in T351820. But I see that even if those tables were potentially bigger, there was just a dozen of tables from one cluster. Here we have several hundred tables, many of them fairly big, and from a half a dozen of clusters.
We'll need to plan for that then.

The relevant part is probably this:

We acknowledge that third party tools on Cloud VPS and Toolforge may rely on the Wikibase database schema. Because of this, changes to the available tables and fields are subject to the above notification policy. However, note that the database schema is not designed to be a public API, and less consideration is given to backwards compatibility.

OTOH, I assume the usage of this table to be rather low (specially compared to other tables). Is there a way to poll for usage via email to mailing lists or similar? Maybe noone really use it and we can get away with not exposing it to the wikireplicas at all.

The relevant part is probably this:

We acknowledge that third party tools on Cloud VPS and Toolforge may rely on the Wikibase database schema. Because of this, changes to the available tables and fields are subject to the above notification policy. However, note that the database schema is not designed to be a public API, and less consideration is given to backwards compatibility.

Agreed

OTOH, I assume the usage of this table to be rather low (specially compared to other tables). Is there a way to poll for usage via email to mailing lists or similar? Maybe noone really use it and we can get away with not exposing it to the wikireplicas at all.

I assume it is low too but I am not certain enough. Asking around is probably up to @Danny_Benjafield_WMDE. (I can help.)

SuzanneWood-WMDE renamed this task from Move Wikibase's entity usage database tables to the extension database cluster to [X1][Epic] Move Wikibase's entity usage database tables to the extension database cluster.Wed, May 20, 12:01 PM