Using the script from T173774: Create script to dump recently changed categories, generate daily dumps of categories that were changed. This will allow to load only daily updates instead of reloading the whole category set (which with commonswiki can take significant time and stalls the updates for up to an hour now).
- Mentioned In
- T260232: BatchRowIterator slow query on commonswiki
T200202: WDQS disk usage increase is correlated with reloading of categories
- Mentioned Here
- T198629: Small number of sql errors on BatchRowIterator::next from BatchRowIterator::next
T173774: Create script to dump recently changed categories
On testwiki and test2wiki I get the following, running for the last seven day interval (all other wikis run ok):
Wikimedia\Rdbms\DBQueryError from line 1443 of /srv/mediawiki/php-1.32.0-wmf.10/includes/libs/rdbms/database/Database.php: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading? Query: SELECT rc_timestamp,page_title,page_namespace,rc_title,rc_cur_id,pp_propname,cat_pages,cat_subcats,cat_files FROM `recentchanges` FORCE INDEX (new_name_timestamp) LEFT JOIN `page_props` ON (pp_propname = 'hiddencat' AND (pp_page = rc_cur_id)) LEFT JOIN `category` ON ((cat_title = rc_title)) WHERE (rc_timestamp >= '20180625095244') AND (rc_timestamp < '20180702095244') AND rc_namespace = '14' AND rc_new = '0' AND rc_log_type = 'move' AND rc_type = '3' ORDER BY rc_timestamp ASC LIMIT 200 Function: BatchRowIterator::next Error: 1054 Unknown column 'page_title' in 'field list'
Can we take care of that before this goes live? Even just removing those two wikis from the categoriedrdf db list would be ok.
@ArielGlen found the bug:
$tables += $extra_tables;
But += does not work as expected on indexed arrays in PHP. using array_merge instead fixes the issue.
$tables = array_merge( $tables, $extra_tables );
This never worked. A local test run immediately failed with an exception.