Page MenuHomePhabricator

WikiExporter does not respect groupLoadsByDB[$wiki]['dump']
Closed, ResolvedPublic

Description

Author: afeldman

Description:
I was just investigating some performance issues on one of the main enwiki slaves (db60) and see several > hour long dump queries running from storage4:

9526634472wikiadmin10.0.10.4:35311enwikiQuery3801Writing to netSELECT /* WikiExporter::dumpFrom */ * FROM page INNER JOIN revision ON ((page_id=rev_page)) WHERE page_id >= 465001 AND page_id < 665001 ORDER BY page_id ASC

db.php contains:

'groupLoadsByDB' => array(

'enwiki' => array(
        'watchlist' => array(
                'db36' => 1,
        ),
        'recentchangeslinked' => array(
                'db36' => 1,
        ),
        'contributions' => array(
                'db36' => 1,
        ),
        'dump' => array(
                'db36' => 1,
        ),

)

Which was intended to direct all enwiki dump queries to db36, but it looks like this is no longer respected. Please fix.


Version: 1.21.x
Severity: normal

Details

Reference
bz41668

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 1:04 AM
bzimport set Reference to bz41668.
bzimport added a subscriber: Unknown Object (MLST).
TTO subscribed.

This is an old task. Is it still relevant in the current context?

BackupDumper->backupDb() in maintenance/backup.inc gets the right db based on the dumps group, and passes this to WikiExporter. This has been working properly for the dumps scripts for a couple of years now at least.

WikiExporter is intended not to use LB or respect db groups; that should be done by the caller. If there are other callers (MW extensions) that use WikiExporter inappropriately, then they should be fixed.

jcrespo claimed this task.
jcrespo subscribed.

Not relevant anymore.