Page MenuHomePhabricator

Table 'metawiki.globaluser' doesn't exist on query
Closed, DuplicatePublic

Description

For the last few days I have been seen errors from time to time like this (44 errors in the last 24h):

error	   	Table 'metawiki.globaluser' doesn't exist (10.192.0.146)
t exception.class	   	RuntimeException
t exception.code	   	0
t exception.file	   	/srv/mediawiki/php-1.36.0-wmf.9/includes/libs/rdbms/database/Database.php:1680
t exception.message	   	
t exception.trace	   	#0 /srv/mediawiki/php-1.36.0-wmf.9/includes/libs/rdbms/database/Database.php(1658): Wikimedia\Rdbms\Database->getQueryExceptionAndLog(string, integer, string, string)
#1 /srv/mediawiki/php-1.36.0-wmf.9/includes/libs/rdbms/database/Database.php(1227): Wikimedia\Rdbms\Database->reportQueryError(string, integer, string, string, boolean)
#2 /srv/mediawiki/php-1.36.0-wmf.9/includes/libs/rdbms/database/Database.php(1907): Wikimedia\Rdbms\Database->query(string, string, integer)
#3 /srv/mediawiki/php-1.36.0-wmf.9/includes/libs/rdbms/database/Database.php(2007): Wikimedia\Rdbms\Database->select(array, array, array, string, array, array)
#4 /srv/mediawiki/php-1.36.0-wmf.9/includes/libs/rdbms/database/DBConnRef.php(68): Wikimedia\Rdbms\Database->selectRow(array, array, array, string, array, array)
#5 /srv/mediawiki/php-1.36.0-wmf.9/includes/libs/rdbms/database/DBConnRef.php(331): Wikimedia\Rdbms\DBConnRef->__call(string, array)
#6 /srv/mediawiki/php-1.36.0-wmf.9/extensions/CentralAuth/includes/CentralAuthUser.php(470): Wikimedia\Rdbms\DBConnRef->selectRow(array, array, array, string, array, array)
#7 /srv/mediawiki/php-1.36.0-wmf.9/extensions/CentralAuth/includes/CentralAuthUser.php(534): CentralAuthUser->loadFromDatabase()
#8 /srv/mediawiki/php-1.36.0-wmf.9/includes/libs/objectcache/wancache/WANObjectCache.php(1533): CentralAuthUser->{closure}(array, integer, array, double, array)
#9 /srv/mediawiki/php-1.36.0-wmf.9/includes/libs/objectcache/wancache/WANObjectCache.php(2496): WANObjectCache->fetchOrRegenerate(string, integer, Closure, array, array)
#10 /srv/mediawiki/php-1.36.0-wmf.9/includes/deferred/MWCallableUpdate.php(38): WANObjectCache->{closure}()
#11 /srv/mediawiki/php-1.36.0-wmf.9/includes/deferred/DeferredUpdates.php(467): MWCallableUpdate->doUpdate()
#12 /srv/mediawiki/php-1.36.0-wmf.9/includes/deferred/DeferredUpdates.php(344): DeferredUpdates::attemptUpdate(MWCallableUpdate, Wikimedia\Rdbms\LBFactoryMulti)
#13 /srv/mediawiki/php-1.36.0-wmf.9/includes/deferred/DeferredUpdates.php(278): DeferredUpdates::run(MWCallableUpdate, Wikimedia\Rdbms\LBFactoryMulti, Monolog\Logger, BufferingStatsdDataFactory, string)
#14 /srv/mediawiki/php-1.36.0-wmf.9/includes/deferred/DeferredUpdates.php(194): DeferredUpdates::handleUpdateQueue(array, string, integer)
#15 /srv/mediawiki/php-1.36.0-wmf.9/includes/MediaWiki.php(1113): DeferredUpdates::doUpdates(string)
#16 /srv/mediawiki/php-1.36.0-wmf.9/includes/MediaWiki.php(849): MediaWiki->restInPeace()
#17 /srv/mediawiki/php-1.36.0-wmf.9/includes/MediaWiki.php(861): MediaWiki->{closure}()
#18 /srv/mediawiki/php-1.36.0-wmf.9/includes/MediaWiki.php(582): MediaWiki->doPostOutputShutdown()
#19 /srv/mediawiki/php-1.36.0-wmf.9/index.php(53): MediaWiki->run()
#20 /srv/mediawiki/php-1.36.0-wmf.9/index.php(46): wfIndexMain()
#21 /srv/mediawiki/w/index.php(3): require(string)
#22 {main}
t facility	   	user
t fname	   	CentralAuthUser::loadFromDatabase
t host	   	mw2272
t http_method	   	GET
t ip	   	10.192.48.94
t level	   	ERROR
t logsource	   	mw2272
t message	   	Error 1146 from CentralAuthUser::loadFromDatabase, Table 'metawiki.globaluser' doesn't exist (10.192.0.146) SELECT  gu_id,gu_name,lu_wiki,gu_salt,gu_password,gu_auth_token,gu_locked,gu_hidden,gu_registration,gu_email,gu_email_authenticated,gu_home_db,gu_cas_token  FROM `globaluser` LEFT OUTER JOIN `localuser` ON ((gu_name=lu_name) AND lu_wiki = 'enwiki')   WHERE gu_name = 'Goldentzofficial'  LIMIT 1   10.192.0.146

For some reason something is expecting to have globaluser table on metawiki, and that table only lives in centralauth.

Event Timeline

Not really sure which tags to use here

Do we know which wiki it's coming from? Or is it metawiki itself?

MarcoAurelio renamed this task from Table 'metawiki.globaluser' doesn't exist on query. to Table 'metawiki.globaluser' doesn't exist on query.Sep 22 2020, 7:54 PM

Do we know which wiki it's coming from? Or is it metawiki itself?

Not that we vary $wgCentralAuthDatabase anyway in code or config...

So I'm presuming this code is hiccuping?

		$lbFactory = MediaWikiServices::getInstance()->getDBLoadBalancerFactory();
		return $lbFactory->getMainLB( $wgCentralAuthDatabase )
			->getConnectionRef( DB_REPLICA, 'centralauth', $wgCentralAuthDatabase );

Noting 'centralauth' is a group... Has anything changed with that recently? I did see T263127: Remove groups from db configs. but seemingly nothing actually actioned on that yet?

It is indeed exactly the same. CA code and db config are all fine. The issue is rdbms mixing up connection objects.

Do we know which wiki it's coming from? Or is it metawiki itself?

enwiki from what I am seeing on logstash