There needs to be a second External Store "cluster" so we can test cutting over.
I think we can just use the main wiki databases, but with a different 'blobs table' parameter.
Use blobs_flow1 (naming from https://phabricator.wikimedia.org/T107610#1506876)
We do now (though it's not being used yet, pending https://gerrit.wikimedia.org/r/#/c/282440/ ). However, there is a missing blocker here.
Created second one (I used a less clear name the first time so I had to fix and re-run):
export MEDIAWIKI_STAGING_DIR; ../../extensions/WikimediaMaintenance/storage/make-all-blobs deployment-db1.deployment-prep.eqiad.wmflabs blobs_flow1
Patch coming soon to cut over (needs to be enabled before we can run the dry-run script).
Mentioned in SAL [2016-05-11T01:44:18Z] <matt_flaschen> Created Flow-specific External Store tables (blobs_flow1) on all wiki databases on Beta Cluster: T128417
Change 288323 had a related patch set uploaded (by Mattflaschen):
Add Flow-specific External Store on Beta Cluster
Change 288323 merged by jenkins-bot:
Add Flow-specific External Store on Beta Cluster
There was a minor problem with that patch, so I had to do rOMWC1317ae266772: Fix Flow external store: Protocol was missing. However, it's done and works now. From tests on http://en.wikipedia.beta.wmflabs.org/wiki/Talk:Flow :
mysql> SELECT HEX(rev_id), HEX(rev_type_id), rev_content, rev_flags FROM flow_revision ORDER BY rev_id DESC LIMIT 7;
+------------------------+------------------------+----------------------+------------------------------------------+
| HEX(rev_id) | HEX(rev_type_id) | rev_content | rev_flags |
+------------------------+------------------------+----------------------+------------------------------------------+
| 0552975C385265DDB4696B | 0551CE4140A1E14F40AF18 | DB://flow_cluster1/6 | utf-8,gzip,html,external |
| 05529758A039E14F40AF18 | 05529758A039E14F40AF18 | DB://flow_cluster1/5 | utf-8,gzip,html,external |
| 055297579259E14F40AF18 | 055297579259E14F40AF18 | DB://flow_cluster1/4 | utf-8,gzip,html,external |
| 055297579245E24F40AF18 | 055297579245E14F40AF18 | DB://flow_cluster1/3 | utf-8,gzip,topic-title-wikitext,external |
| 055297550E49E14F40AF18 | 055297550E49E14F40AF18 | DB://flow_cluster1/2 | utf-8,gzip,html,external |
| 055297550D5DE14F40AF18 | 055297550BF1E14F40AF18 | DB://flow_cluster1/1 | utf-8,gzip,topic-title-wikitext,external |
| 0552897AA28A65DDB4696B | 0539E8BC9D23F1DDB4696B | DB://cluster1/11309 | utf-8,gzip,html,external |
+------------------------+------------------------+----------------------+------------------------------------------+
7 rows in set (0.00 sec)
mysql> SELECT * FROM blobs_flow1 LIMIT 10;
+---------+---------------------------------------------------------------------------------------------+
| blob_id | blob_text |
+---------+---------------------------------------------------------------------------------------------+
| 1 | �K-W(�/�L |
| 2 | �)PHI,I�-H,*��L�U�VJ).R��6�16�1�1��U�+�,�Pp��/�-.HM�L�LVp�(I-�K�.�/J��/� |
| 3 |
J-I-.Q�K-W(�/�L |
| 4 | �)PHI,I�-H,*��L�U�VJ).R��6�16�1�1��U�+�,�Pp��/�-.HM�L�LVH�(I-�K�Q(.�/J��/� |
| 5 | ��@0 �_�t� ,�/���ѓ6i�rWAĿ{�'p6ے,K
n(^��U�ݴ�h�~�8� �H��SP {�)���-�a�3�a#HN�U_�� |
| 6 | �)PHI,I�-H,*��L�U�VJ).R��6�14�1�1��U�su�
�Rp�O-V�H-J��/� |
+---------+---------------------------------------------------------------------------------------------+
6 rows in set (0.00 sec)I also tested on one other Beta Cluster wiki (Catalan Wikipedia) just for completeness.
MariaDB [enwiki]> select HEX(rev_id), HEX(rev_type_id), rev_content, rev_flags from flow_revision order by rev_mod_timestamp desc limit 7; +------------------------+------------------------+--------------------------+---------------------------------------------------------------------+ | HEX(rev_id) | HEX(rev_type_id) | rev_content | rev_flags | +------------------------+------------------------+--------------------------+---------------------------------------------------------------------+ | 055581A78F3DE14F40AF18 | 0555819D8C29E14F40AF18 | DB://flow_cluster1/12125 | utf-8,gzip,topic-title-wikitext,external | | 055581A44B6A65DDB4696B | 0555819E881665DDB4696B | DB://flow_cluster1/12124 | utf-8,gzip,topic-title-wikitext,external | | 0555694F9B2DE14F40AF18 | 0550BEC83CE1E14F40AF18 | DB://flow_cluster1/7792 | utf-8,gzip,topic-title-wikitext,external,external,external,external | | 0555694AEF1DE14F40AF18 | 05518E23E2A9E14F40AF18 | DB://flow_cluster1/7790 | utf-8,gzip,topic-title-wikitext,external,external | | 0555693D406DE14F40AF18 | 0550BEC83CE1E14F40AF18 | DB://flow_cluster1/7789 | utf-8,gzip,topic-title-wikitext,external,external | | 05556869E8C5E14F40AF18 | 0543CB7BDE46D2DDB4696B | DB://flow_cluster1/7782 | utf-8,gzip,wikitext,external | | 055568460D5265DDB4696B | 054B38A67DD354DDB4696B | DB://flow_cluster1/7774 | utf-8,gzip,topic-title-wikitext,external | +------------------------+------------------------+--------------------------+---------------------------------------------------------------------+ 7 rows in set (0.28 sec)
Just FYI, rev_mod_timestamp is the timestamp of a moderation action, and is not always set (it can be null). If you just want to order by timestamp, use rev_id.
Change 321810 had a related patch set uploaded (by Mattflaschen):
Add Flow External Store to addWiki.php