Page MenuHomePhabricator

E10. Have Flow use ExternalStore on MediaWiki-Vagrant
Closed, ResolvedPublic2 Story Points

Description

Flow has two paths for content storage, and this is the one used in production.

This can benefit both local and MediaWiki-Vagrant testing.

See T95580: Flow data missing on Wikimedia production wikis.

For QA'ing this, you can check that rev_content in flow_revision has a URL starting with DB://, and optionally check the 'external' database to match that up.

Event Timeline

Mattflaschen-WMF updated the task description. (Show Details)
Mattflaschen-WMF raised the priority of this task from to Needs Triage.
Restricted Application added a project: Collaboration-Team-Triage. · View Herald TranscriptApr 13 2015, 3:22 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
EBernhardson triaged this task as High priority.Apr 13 2015, 5:53 PM
EBernhardson added a subscriber: EBernhardson.
DannyH set Security to None.Apr 13 2015, 6:11 PM
DannyH edited a custom field.

All we should need is:

$wgExternalServers['external'] = array(
    array(
        'host' => '...',
        'user' => '...',
        'password' => '...',
        'dbname' => '...',
        'type' => 'mysql',
        'load' => 1,
        'flags' => DBO_DEBUG | DBO_DEFAULT
    ),
);
$wgExternalStores = array( 'DB' );
$wgDefaultExternalStore = array( 'DB://external' ); // = $wgExternalServers['external']
$wgFlowExternalStore = $wgDefaultExternalStore;

However, tests didn't seem to work for me with ES.
Can't remember why exactly - probably not properly cloning blobs table. Maybe because we don't add them to $tablesUsed, maybe not supported?.
So we may need a defined( 'MW_PHPUNIT_TEST' ) conditional.

DannyH renamed this task from Have Flow use ExternalStore on MediaWiki-Vagrant to E10. Have Flow use ExternalStore on MediaWiki-Vagrant.Jun 30 2015, 9:05 PM

Change 223724 had a related patch set uploaded (by Mattflaschen):
WIP: Add optional externalstore role and use it for Flow

https://gerrit.wikimedia.org/r/223724

DannyH added a subscriber: DannyH.Jul 22 2015, 9:32 PM

What does this ticket need? It's been in the "In Development" column for a while, with no development that I know of.

Change 274643 had a related patch set uploaded (by Mattflaschen):
WIP: Add support for External Store for unit tests

https://gerrit.wikimedia.org/r/274643

Change 274643 merged by jenkins-bot:
Add support for External Store for unit tests

https://gerrit.wikimedia.org/r/274643

Change 277454 had a related patch set uploaded (by Mattflaschen):
Move dbSetup=true; follow-up d920237

https://gerrit.wikimedia.org/r/277454

Change 277454 merged by jenkins-bot:
Move dbSetup=true; follow-up d920237

https://gerrit.wikimedia.org/r/277454

Mattflaschen-WMF added a comment.EditedMar 15 2016, 9:54 PM

Vagrant patch itself (https://gerrit.wikimedia.org/r/#/c/223724/) needs to be updated then reviewed.

DannyH removed a subscriber: DannyH.Mar 16 2016, 11:45 PM

Change 223724 merged by jenkins-bot:
Add optional externalstore role and use it for Flow

https://gerrit.wikimedia.org/r/223724

Checked on vagrant - all newly created pages refer to external.
e.g.

(20:22) root@localhost:[wiki]>  SELECT page_id, rev_id, old_id, old_text FROM page JOIN revision ON page_latest = rev_id JOIN text ON rev_text_id = old_id WHERE page_namespace = 1 AND page_title  = 'ET5';
+---------+--------+--------+-------------------+
| page_id | rev_id | old_id | old_text          |
+---------+--------+--------+-------------------+
|     170 |    400 |    397 | DB://external/396 |
+---------+--------+--------+-------------------+
1 row in set (0.00 sec)
(20:23) root@localhost:[wiki]> SELECT rev_id, old_id, old_text FROM revision JOIN text ON old_id = rev_text_id ORDER by rev_id DESC LIMIT 5;
+--------+--------+-------------------+
| rev_id | old_id | old_text          |
+--------+--------+-------------------+
|    400 |    397 | DB://external/396 |
|    399 |    396 | DB://external/394 |
|    398 |    395 | DB://external/393 |
|    397 |    394 | DB://external/385 |
|    396 |    393 | DB://external/384 |
+--------+--------+-------------------+
5 rows in set (0.00 sec)
jmatazzoni closed this task as Resolved.Apr 29 2016, 4:37 PM