DBError: Access denied for user 'wikiadmin'@'172.16.%' to database 'mainstash' on Beta Cluster
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Assigned To

Authored By

	TheresNoTime
	Nov 5 2022, 8:39 AM

Description

Error

mwversion: 1.40.0-alpha
reqId: ae07a48d44bd6cdcca6e4cd2

normalized_message

DBError: Cannot access the database: Access denied for user 'wikiadmin'@'172.16.%' to database 'mainstash' (172.16.0.238:3306)

exception.trace

from /srv/mediawiki/php-master/includes/libs/rdbms/loadbalancer/LoadBalancer.php(1321)
#0 /srv/mediawiki/php-master/includes/libs/rdbms/loadbalancer/LoadBalancer.php(949): Wikimedia\Rdbms\LoadBalancer->reportConnectionError()
#1 /srv/mediawiki/php-master/includes/libs/rdbms/loadbalancer/LoadBalancer.php(922): Wikimedia\Rdbms\LoadBalancer->getServerConnection(integer, string, integer)
#2 /srv/mediawiki/php-master/includes/libs/rdbms/database/DBConnRef.php(103): Wikimedia\Rdbms\LoadBalancer->getConnectionInternal(integer, array, string, integer)
#3 /srv/mediawiki/php-master/includes/objectcache/SqlBagOStuff.php(1688): Wikimedia\Rdbms\DBConnRef->ensureConnection()
#4 /srv/mediawiki/php-master/includes/objectcache/SqlBagOStuff.php(403): SqlBagOStuff->getConnectionViaLoadBalancer()
#5 /srv/mediawiki/php-master/includes/objectcache/SqlBagOStuff.php(490): SqlBagOStuff->getConnection(integer)
#6 /srv/mediawiki/php-master/includes/objectcache/SqlBagOStuff.php(334): SqlBagOStuff->fetchBlobs(array)
#7 /srv/mediawiki/php-master/includes/libs/objectcache/MediumSpecificBagOStuff.php(667): SqlBagOStuff->doGetMulti(array, integer)
#8 /srv/mediawiki/php-master/includes/ResourceLoader/dependencystore/KeyValueDependencyStore.php(50): MediumSpecificBagOStuff->getMulti(array)
#9 /srv/mediawiki/php-master/includes/ResourceLoader/ResourceLoader.php(462): Wikimedia\DependencyStore\KeyValueDependencyStore->retrieveMulti(string, array)
#10 /srv/mediawiki/php-master/extensions/WikimediaMaintenance/blameStartupRegistry.php(79): MediaWiki\ResourceLoader\ResourceLoader->preloadModuleInfo(array, MediaWiki\ResourceLoader\Context)
#11 /srv/mediawiki/php-master/maintenance/includes/MaintenanceRunner.php(309): BlameStartupRegistry->execute()
#12 /srv/mediawiki/php-master/maintenance/doMaintenance.php(85): MediaWiki\Maintenance\MaintenanceRunner->run()
#13 /srv/mediawiki/php-master/extensions/WikimediaMaintenance/blameStartupRegistry.php(336): require_once(string)
#14 /srv/mediawiki/multiversion/MWScript.php(120): require_once(string)
#15 {main}

Impact

Notes

Given there's been 1000s of these, I'm sure it's a known problem (though I can't find a task...)
https://beta-logs.wmcloud.org/app/dashboards#/doc/5f0c9be0-0b6f-11ec-9cde-3f4490e09a26/logstash-mediawiki-1-7.0.0-1-2022.11.05?id=xm7xRoQBjmLLAkJCTYkY

Event Timeline

TheresNoTime created this task.Nov 5 2022, 8:39 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 5 2022, 8:39 AM

Krinkle added a project: Performance-Team (Radar).Dec 22 2022, 7:45 PM

Mentioned in SAL (#wikimedia-releng) [2023-03-19T22:10:20Z] <TheresNoTime> (beta) GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, REFERENCES, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, EVENT, TRIGGER ON mainstash.* TO 'wikiadmin'@'172.16.%'; T322469

Restricted Application added a project: Wikimedia-production-error. · View Herald TranscriptMar 19 2023, 10:10 PM

In T322469#8708351, @Stashbot wrote:

Mentioned in SAL (#wikimedia-releng) [2023-03-19T22:10:20Z] <TheresNoTime> (beta) GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, REFERENCES, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, EVENT, TRIGGER ON mainstash.* TO 'wikiadmin'@'172.16.%'; T322469

This is one less GRANT than has been applied for centralauth.* and %wik%.*, but I based it off of production-core.sql.erb — errors seem to have stopped in /srv/mw-log/rdbms.log :-)

thcipriani removed a project: Wikimedia-production-error.Apr 13 2023, 3:50 PM

Thanks @TheresNoTime

DBError: Access denied for user 'wikiadmin'@'172.16.%' to database 'mainstash' on Beta ClusterClosed, ResolvedPublicPRODUCTION ERRORActions