Page MenuHomePhabricator

Configure External Store on Vagrant with blobs tables in databases named after wiki DB and proper isolation
Open, MediumPublic

Description

I hit this with Wikidata, but later (T185114) it stopped working entirely: https://phabricator.wikimedia.org/diffusion/EWBA/browse/master/lib/includes/Store/Sql/WikiPageEntityRevisionLookup.php;f37e244cd624ead83bc9552ff0b5d4c23880948e$199 .

Originally, all the Vagrant wikis shared the same external store: https://phabricator.wikimedia.org/source/mediawiki-vagrant/browse/master/puppet/modules/role/templates/externalstore/conf.php.erb , unlike production.


Background: In production, each wiki (e.g. enwiki) has corresponding enwiki External Store databases, but they are not the same database. There may be more than one blob table in this other enwiki DB (e.g. blobs_cluster1, blobs_cluster2) (I'm not sure if having multiple blobs tables in the same DB happens in practice in production), but there are no non-blobs tables.


As a temporary solution on Vagrant (T185114), we will put the blobs tables in the main DB (where the other tables are) on Vagrant without changing users. But this makes it potentially possible for code to falsely work locally (by accessing a table that should be elsewhere).

To solve that, we should either restrict the tables by user, or setup a whole separate MySQL (AFAIK this is the only way to have two databases with the same name on one computer).

Event Timeline

Change 344295 had a related patch set uploaded (by Mattflaschen):
[mediawiki/core@master] WIP: First attempt at handing 'dbname' for foreign connections

https://gerrit.wikimedia.org/r/344295

You mean that they all use the same dbName regardless of what wiki content is for?

In any case, the dbName field in the server config map is only meant as a default if one is not specified in openForeignConnection(). If support for forcing a dbname is added, then it should probably go in the ExternalStoreDB level and be configurable.

You mean that they all use the same dbName regardless of what wiki content is for?

Correct.

Change 369586 had a related patch set uploaded (by Mattflaschen; owner: Mattflaschen):
[mediawiki/vagrant@master] Remove externalstore role; instead use only for wikimediaflow

https://gerrit.wikimedia.org/r/369586

Mattflaschen-WMF renamed this task from External Store on Vagrant does not properly work with foreign wiki references to Configure External Store on Vagrant to match production (DB per wiki, only blobs_clustern table in that DB).Jan 17 2018, 7:29 PM
Mattflaschen-WMF renamed this task from Configure External Store on Vagrant to match production (DB per wiki, only blobs_clustern table in that DB) to Configure External Store on Vagrant with blobs tables in databases named after DB..Jan 17 2018, 7:54 PM
Mattflaschen-WMF renamed this task from Configure External Store on Vagrant with blobs tables in databases named after DB. to Configure External Store on Vagrant with blobs tables in databases named after DB.
Mattflaschen-WMF updated the task description. (Show Details)

@aaron Am I right that having two databases with the same name (i.e. "enwiki where revision, page, etc. are" and "enwiki where blobs is") on the same machine requires two different MySQL data directories and ports/sockets?

@SBisson et al, as mentioned in the description, I'm thinking it is enough to partition by user permissions (one enwiki database, but different users can access different tables). But I'm open to other ideas.

Mattflaschen-WMF renamed this task from Configure External Store on Vagrant with blobs tables in databases named after DB to Configure External Store on Vagrant with blobs tables in databases named after DB and proper isolation.Jan 17 2018, 8:06 PM

@aaron Am I right that having two databases with the same name (i.e. "enwiki where revision, page, etc. are" and "enwiki where blobs is") on the same machine requires two different MySQL data directories and ports/sockets?

Yes.

Mattflaschen-WMF renamed this task from Configure External Store on Vagrant with blobs tables in databases named after DB and proper isolation to Configure External Store on Vagrant with blobs tables in databases named after wiki DB and proper isolation.Jan 17 2018, 10:51 PM
Krinkle triaged this task as Medium priority.Jul 23 2019, 4:31 PM
Krinkle moved this task from Untriaged to Revision on the MediaWiki-Core-Revision-backend board.