Page MenuHomePhabricator

terbium maintenance cron "processEchoEmailBatch.php" is getting "access denied" from database
Closed, DuplicatePublic

Description

Cron Daemon root@terbium.eqiad.wmnet via wikimedia.org 
	
5:06 PM (1 hour ago)
	
to www-data
[c4e9a60847a4d7d9b18d4873] [no req]   Wikimedia\Rdbms\DBConnectionError from line 788 of /srv/mediawiki/php-1.30.0-wmf.4/includes/libs/rdbms/database/Database.php: Cannot access the database: Access denied for user 'wikiadmin'@'10.64.32.13' (using password: YES) (208.80.153.14)
Backtrace:
#0 /srv/mediawiki/php-1.30.0-wmf.4/includes/libs/rdbms/loadbalancer/LoadBalancer.php(995): Wikimedia\Rdbms\Database->reportConnectionError(string)
#1 /srv/mediawiki/php-1.30.0-wmf.4/includes/libs/rdbms/loadbalancer/LoadBalancer.php(666): Wikimedia\Rdbms\LoadBalancer->reportConnectionError()
#2 /srv/mediawiki/php-1.30.0-wmf.4/extensions/Echo/includes/EchoDbFactory.php(123): Wikimedia\Rdbms\LoadBalancer->getConnection(integer, array, boolean)
#3 /srv/mediawiki/php-1.30.0-wmf.4/extensions/Echo/includes/EmailBatch.php(340): MWEchoDbFactory::getDB(integer)
#4 /srv/mediawiki/php-1.30.0-wmf.4/extensions/Echo/maintenance/processEchoEmailBatch.php(49): MWEchoEmailBatch::getUsersToNotify(integer, integer)
#5 /srv/mediawiki/php-1.30.0-wmf.4/maintenance/doMaintenance.php(111): ProcessEchoEmailBatch->execute()
#6 /srv/mediawiki/php-1.30.0-wmf.4/extensions/Echo/maintenance/processEchoEmailBatch.php(80): require_once(string)
#7 /srv/mediawiki/multiversion/MWScript.php(99): require_once(string)
#8 {main}

Event Timeline

Dzahn created this task.Jun 8 2017, 1:34 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 8 2017, 1:34 AM

'wikiadmin'@'10.64.32.13' (using password: YES) (208.80.153.14)

10.64.32.13 = terbium
208.80.153.14 = labtestweb2001.wikimedia.org

^ labtestweb? oh really, is that from testing and wasn't switched to the right server?

Mentioned in SAL (#wikimedia-operations) [2017-06-08T01:45:25Z] <mutante> manually running mediawiki maintenance job "echo_mail_batch" (on terbium as www-data, just like cron). did _NOT_ get denied by DB (T167373)

Dzahn added a comment.EditedJun 8 2017, 1:50 AM

^ can't reproduce it when manually running it (as the same user from the same host), but ... the emails have been arriving (almost) every day since May 25


[terbium:~] $ sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/dblists/echo.dblist extensions/Echo/maintenance/processEchoEmailBatch.php`
...
zuwiktionary
-----------------------------------------------------------------
zuwiktionary:  Started processing... 
zuwiktionary:  Completed
Dzahn triaged this task as Medium priority.Jun 9 2017, 9:27 PM