Page MenuHomePhabricator

Refactor LoadBalancer connection pooling to be more efficient
Open, LowPublic

Description

Currently, foreign and local domain connections always use different handles. Also, the foreign connection logic does not support reuse of handles while one is already in use, e.g.:

$cMain = $lb->getConnection( DB_MASTER );
// some queries
$conn1 = $lb->getConnection( DB_MASTER, [], 'x' );
// some queries
$conn2 = $lb->getConnection( DB_MASTER, [], 'y' );
// some queries
$lb->reuseConnection( $conn1 );
// some queries
$lb->reuseConnection( $conn2 );

...will acquire three connections. If everything used connection references, and those DBConnRef instance checked and corrected the domain in each proxy method (implementing IDatabase), then the above scenario could just use a single connection and one atomic transaction. E.g.:

$cMain = $lb->getConnectionRef( DB_MASTER );
// some queries
$conn1 = $lb->getConnectionRef( DB_MASTER, [], 'x' );
// some queries
$conn2 = $lb->getConnectionRef( DB_MASTER, [], 'y' );
// some queries
// $conn2 falls out of scope
// some queries
// $conn1 falls out of scope
// $cMain falls out of scope

Connections acquired by raw getConnection() calls would still use a separate pool for backwards compatibility (no sudden domain changes from DBConnRef instances proxying them).

Event Timeline

aaron created this task.Jun 26 2019, 2:37 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 26 2019, 2:37 AM

Change 519305 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/core@master] [WIP] rdbms: improve LoadBalancer connection pool reuse

https://gerrit.wikimedia.org/r/519305

kchapman moved this task from Inbox to Doing on the Performance-Team board.Jul 1 2019, 7:53 PM
Krinkle moved this task from Untriaged to Rdbms library on the Wikimedia-Rdbms board.
Krinkle triaged this task as Medium priority.Jul 23 2019, 5:30 PM
Restricted Application added a project: Platform Engineering. · View Herald TranscriptJul 23 2019, 5:30 PM
Krinkle lowered the priority of this task from Medium to Low.Apr 21 2020, 5:53 PM
TK-999 added a subscriber: TK-999.May 23 2020, 9:52 PM

Change 643563 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/core@master] Migrate LoadBalancer::getConnection() calls to LoadBalancer::getConnectionRef()

https://gerrit.wikimedia.org/r/643563