Beta Cluster does not have php7.0-redis available
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Etonkovidova
	Mar 9 2019, 12:34 AM

Description

I cannot login to beta cluster db. I checked the instructions on https://www.mediawiki.org/wiki/Beta_Cluster - they refer to the way that I was doing successfully before. Today I got the same error as was reported in T217323:

 $ ssh  deployment-deploy01.eqiad.wmflabs 
Linux deployment-deploy01 4.9.0-8-amd64 #1 SMP Debian 4.9.110-3+deb9u6 (2018-10-08) x86_64
Debian GNU/Linux 9.5 (stretch)
The last Puppet run was at Fri Mar  8 23:59:42 UTC 2019 (24 minutes ago). 
Last login: Sat Mar  9 00:22:46 2019 from 172.16.1.136
groups: cannot find name for group ID 50120
groups: cannot find name for group ID 50120
groups: cannot find name for group ID 50120

etonkovidova@deployment-deploy01:~$ sql enwiki
Fatal error: Uncaught RuntimeException: RedisConnectionPool requires a Redis client library. See https://www.mediawiki.org/wiki/Redis#Setup in /srv/mediawiki-staging/php-master/includes/libs/redis/RedisConnectionPool.php:80
Stack trace:
#0 /srv/mediawiki-staging/php-master/includes/libs/redis/RedisConnectionPool.php(152): RedisConnectionPool->__construct(Array, '8d9cd2955414c7d...')
#1 /srv/mediawiki-staging/php-master/includes/libs/objectcache/RedisBagOStuff.php(77): RedisConnectionPool::singleton(Array)
#2 /srv/mediawiki-staging/php-master/includes/objectcache/ObjectCache.php(216): RedisBagOStuff->__construct(Array)
#3 /srv/mediawiki-staging/php-master/includes/ServiceWiring.php(242): ObjectCache::newFromParams(Array)
#4 /srv/mediawiki-staging/php-master/includes/libs/services/ServiceContainer.php(430): Wikimedia\Services\ServiceContainer->{closure}(Object(MediaWiki\MediaWikiServices))
#5 /srv/mediawiki-staging/php-master/includes/libs/services/ServiceContainer.php(414): Wikimedia\Services\ServiceContainer->createServic in /srv/mediawiki-staging/php-master/includes/libs/redis/RedisConnectionPool.php on line 80

With mysql enwiki the output gives other details:

etonkovidova@deployment-deploy01:~$ mysql enwiki
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2 "No such file or directory")

I do not have /var/run/mysqld directory.
Checking for mysql presence:

etonkovidova@deployment-deploy01:~$ ps ax | grep mysql
18622 pts/3    R+     0:00 grep mysql

etonkovidova@deployment-deploy01:~$ dpkg --get-selections | grep mysql
default-mysql-client				install
libdbd-mysql-perl				install
mysql-client					install
mysql-common					install
php-mysql					install
php7.0-mysql					deinstall
php7.2-mysql					install

Any advice? Is there a different way to access betalabs db?

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	scap: use php7.2 in the "sql" script	operations/puppet	production	+2 -2

Customize query in gerrit

Related Objects

Mentioned Here: T216712: Switch PHP 7.2 packages to an internal component
P8200 (An Untitled Masterwork)
T217323: Beta site api.phps gets - RuntimeException: RedisConnectionPool requires a Redis client library

Event Timeline

Etonkovidova created this task.Mar 9 2019, 12:34 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 9 2019, 12:34 AM

Etonkovidova updated the task description. (Show Details)Mar 9 2019, 12:35 AM

You won't find MySQLd itself running on a deployment host. MySQL hosts are named beginning with deployment-db
You might find mwscript sql.php enwiki works on deployment hosts.

Also something is broken on -db05:

root@deployment-db05:~# mysql
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2 "No such file or directory")
root@deployment-db05:~# service mariadb status | grep PID
 Main PID: 20647 (mysqld)
root@deployment-db05:~# lsof -p 20647 | grep sock
mysqld  20647 mysql  mem    REG              254,3       22520  131519 /opt/wmf-mariadb101/lib/plugin/auth_socket.so
mysqld  20647 mysql   23u  unix 0xffff89d3678e5800         0t0 3525014 /tmp/mysql.sock type=STREAM
root@deployment-db05:~# ls -lh /tmp/mysql.sock
ls: cannot access '/tmp/mysql.sock': No such file or directory

@Krenair: mariadb no longer puts its socket in /tmp/mysql.sock, it seems that the socket is in a 'systemd-private-xxx' directory within /tmp. See:


twentyafterfour@deployment-db05:~$ ls /tmp
pymp-DurpyP                                                            systemd-private-d6c71da3465641b3aa68e8390a2cc75c-mariadb.service-zWseWR
pymp-SZ81bV                                                            systemd-private-d6c71da3465641b3aa68e8390a2cc75c-systemd-timesyncd.service-BPWcb0
systemd-private-d6c71da3465641b3aa68e8390a2cc75c-lldpd.service-rB1ha9
twentyafterfour@deployment-db05:~$ sudo ls /tmp/systemd-private-d6c71da3465641b3aa68e8390a2cc75c-mariadb.service-zWseWR/tmp
mysql.sock

I am having the same issue:

P8200 (An Untitled Masterwork)

1	maurelio@deployment-deploy01:~$ sql deploymentwiki;
2
3	Fatal error: Uncaught RuntimeException: RedisConnectionPool requires a Redis client library. See https://www.mediawiki.org/wiki/Redis#Setup in /srv/mediawiki-staging/php-master/includes/libs/redis/RedisConnectionPool.php:80
4
5	Stack trace:
6	#0 /srv/mediawiki-staging/php-master/includes/libs/redis/RedisConnectionPool.php(152): RedisConnectionPool->__construct(Array, '8d9cd2955414c7d...')
7	#1 /srv/mediawiki-staging/php-master/includes/libs/objectcache/RedisBagOStuff.php(77): RedisConnectionPool::singleton(Array)
8	#2 /srv/mediawiki-staging/php-master/includes/objectcache/ObjectCache.php(216): RedisBagOStuff->__construct(Array)
9	#3 /srv/mediawiki-staging/php-master/includes/ServiceWiring.php(242): ObjectCache::newFromParams(Array)
10	#4 /srv/mediawiki-staging/php-master/includes/libs/services/ServiceContainer.php(430): Wikimedia\Services\ServiceContainer->{closure}(Object(MediaWiki\MediaWikiServices))
11	#5 /srv/mediawiki-staging/php-master/includes/libs/services/ServiceContainer.php(414): Wikimedia\Services\ServiceContainer->createServic in /srv/mediawiki-staging/php-master/includes/libs/redis/RedisConnectionPool.php on line 80

However I note that in the past I could do sql eswiki; on deployment-deploy01 just fine.

on deployment-deploy01 in /usr/local/bin/sql we have php=php7.0 but apt doesn't have a php7.0-redis package

• mmodell added a project: PHP 7.0 support.Mar 14 2019, 4:12 PM

In T217938#5024516, @mmodell wrote:

on deployment-deploy01 in /usr/local/bin/sql we have php=php7.0 but apt doesn't have a php7.0-redis package

Erik B just tried modifying the sql script to call hhvm instead of php and it works, so there's a workaround.

But, @Joe is this something that is expected with the php7 migration?

greg renamed this task from Cannot access beta cluster db to Beta Cluster does not have php7.0-redis available.Mar 15 2019, 5:01 PM

Sorry for coming late to the party, I was AFK last week.

I should really eradicate php7.0 from both production and beta, as it's unsupported at this point.

Only php7.2 should be used.

Digging further:

php-redis was rebuilt as part of the work for T216712 without php 7.0 support
it has been installed in deployment-prep but not in production

So we just need to convert all scripts to php 7.2 before installing it further.

Joe claimed this task.Mar 18 2019, 12:27 PM

Change 497297 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/puppet@production] scap: use php7.2 in the "sql" script

https://gerrit.wikimedia.org/r/497297

gerritbot added a project: Patch-For-Review.Mar 18 2019, 1:22 PM

Change 497297 merged by Giuseppe Lavagetto:
[operations/puppet@production] scap: use php7.2 in the "sql" script

https://gerrit.wikimedia.org/r/497297

• mmodell awarded a token.Mar 18 2019, 2:12 PM

Thanks! I can access betalabs db now. Resolving the ticket.

Etonkovidova awarded a token.Mar 19 2019, 5:13 PM

Change 529921 had a related patch set uploaded (by Effie Mouzeli; owner: Effie Mouzeli):
[operations/puppet@production] mw-maintenance and scap: Revert changes for PHP7 transition

https://gerrit.wikimedia.org/r/529921