updateSpecialPages.php will try to re-connect to a DB indefinitely
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	taavi
	Oct 9 2023, 7:08 AM

Description

As seen in here: https://gerrit.wikimedia.org/g/mediawiki/core/+/4bc5734399d240534af95139cefced7df760c5e2/maintenance/updateSpecialPages.php#138

	private function reopenAndWaitForReplicas() {
		$lbFactory = $this->getServiceContainer()->getDBLoadBalancerFactory();
		$lb = $lbFactory->getMainLB();
		if ( !$lb->pingAll() ) {
			$this->output( "\n" );
			do {
				$this->error( "Connection failed, reconnecting in 10 seconds..." );
				sleep( 10 );
			} while ( !$lb->pingAll() );
			$this->output( "Reconnected\n\n" );
		}
		// Wait for the replica DB to catch up
		$this->waitForReplication();
	}

This will get stuck if the DB it was previously using gets taken down for maintenance or something similar.

Details

	Subject	Repo	Branch	Lines +/-
	updateSpecialPages: Call ::waitForReplication() if a host isn't reachable	mediawiki/core	master	+1 -1

Customize query in gerrit

Related Objects

Mentioned In: T348433: On Wikispecies and other wikis, the data in Special:LonelyPages has not been updated since 25 September
T348419: Error connecting to db2109 as user wikiadmin2023: :real_connect(): (HY000/2002): Connection refused

Event Timeline

taavi created this task.Oct 9 2023, 7:08 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 9 2023, 7:08 AM

ABran-WMF mentioned this in T348419: Error connecting to db2109 as user wikiadmin2023: :real_connect(): (HY000/2002): Connection refused.Oct 9 2023, 8:41 AM

Clement_Goubert mentioned this in T348433: On Wikispecies and other wikis, the data in Special:LonelyPages has not been updated since 25 September.Oct 9 2023, 9:16 AM

Change 964455 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] updateSpecialPages: Call ::waitForReplication() if a host isn't reachable

https://gerrit.wikimedia.org/r/964455

gerritbot added a project: Patch-For-Review.Oct 9 2023, 10:31 AM

I would just delete the whole thing, it breaks many assumptions (caller shouldn't know about replicas and their health), it pings every replica (why?) but it's not easily doable. For now I make a tiny change to make sure this doesn't happen again.

Change 964455 merged by jenkins-bot:

[mediawiki/core@master] updateSpecialPages: Call ::waitForReplication() if a host isn't reachable

https://gerrit.wikimedia.org/r/964455

Maintenance_bot removed a project: Patch-For-Review.Oct 9 2023, 1:30 PM

So currently if a replica is dead, it'll try to connect indefinitely until someone depools the host. Is that a good enough solution for this ticket? If so, please close it.

That sounds fine.

updateSpecialPages.php will try to re-connect to a DB indefinitelyClosed, ResolvedPublicActions

Description

Details

Related Objects

Event Timeline

updateSpecialPages.php will try to re-connect to a DB indefinitely
Closed, ResolvedPublic
Actions