Page MenuHomePhabricator

SpecialContributions Revision::getParentLengths wrong load balancing db connection
Closed, ResolvedPublic

Description

SpecialContributions::doBatchLookups() calls Revision::getParentLengths() like this:

Revision::getParentLengths( $this->getDatabase(), $revIds );

getDatabase returns $this->mDb which is set in the SpecialContributions constructor using 'contributions' to identify it to the database load balancer:

$this->mDb = wfGetDB( DB_SLAVE, 'contributions' );

Passing mDb to getParentLengths is not ideal as the query type is quite different and limits what can be done in the backend. Consider passing a normal unnamed slave connection instead.

On WMF production shards mDb is usually a connection to a slave set aside for special queries on a shard. One of the possible optimizations that can be applied to such slaves is table partitioning based on range or hash of rev_user (this approach is proving effective for the similar LogPager queries).

Unfortunately partitioning increases the cost of some other types of queries including the huge batched lookups generated by getParentLengths. It would be nice to have the best of both worlds.


Version: 1.23.0
Severity: normal

Details

Reference
bz59987

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 2:42 AM
bzimport set Reference to bz59987.

Change 107319 had a related patch set uploaded by Legoktm:
SpecialContributions: Avoid using 'contributions' slave for getParentLengths

https://gerrit.wikimedia.org/r/107319

Change 107319 merged by jenkins-bot:
SpecialContributions: Avoid using 'contributions' slave for getParentLengths

https://gerrit.wikimedia.org/r/107319

Change 107545 had a related patch set uploaded by Aaron Schulz:
Use regular slaves for user contribs parent revision queries

https://gerrit.wikimedia.org/r/107545

Change 107545 had a related patch set uploaded by Krinkle:
Use regular slaves for user contribs parent revision queries

https://gerrit.wikimedia.org/r/107545

Change 107545 merged by jenkins-bot:
Use regular slaves for user contribs parent revision queries

https://gerrit.wikimedia.org/r/107545