Page MenuHomePhabricator

mwscript dies on mwmaint with PHP=php7.2 due to php-redis missing
Closed, ResolvedPublic


tgr@mwmaint1002:~$ mwscript eval.php commonswiki
> ^D
tgr@mwmaint1002:~$ PHP=php7.0 mwscript eval.php commonswiki
> ^D
tgr@mwmaint1002:~$ PHP=php7.2 mwscript eval.php commonswiki
Fatal error: Uncaught RuntimeException: RedisConnectionPool requires a Redis client library. See in /srv/mediawiki/php-1.33.0-wmf.14/includes/libs/redis/RedisConnectionPool.php:80
Stack trace:
#0 /srv/mediawiki/php-1.33.0-wmf.14/includes/libs/redis/RedisConnectionPool.php(152): RedisConnectionPool->__construct(Array, '4b1773007d25335...')
#1 /srv/mediawiki/php-1.33.0-wmf.14/includes/libs/objectcache/RedisBagOStuff.php(77): RedisConnectionPool::singleton(Array)
#2 /srv/mediawiki/php-1.33.0-wmf.14/includes/objectcache/ObjectCache.php(216): RedisBagOStuff->__construct(Array)
#3 /srv/mediawiki/php-1.33.0-wmf.14/includes/ServiceWiring.php(242): ObjectCache::newFromParams(Array)
#4 /srv/mediawiki/php-1.33.0-wmf.14/includes/libs/services/ServiceContainer.php(430): Wikimedia\Services\ServiceContainer->{closure}(Object(MediaWiki\MediaWikiServices))
#5 /srv/mediawiki/php-1.33.0-wmf.14/includes/libs/services/ServiceContainer.php(414): Wikimedia\Services\ServiceContainer->createService('Main in /srv/mediawiki/php-1.33.0-wmf.14/includes/libs/redis/RedisConnectionPool.php on line 80

mwdebug1002 works fine.

Event Timeline

Tgr created this task.Feb 6 2019, 3:36 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 6 2019, 3:36 AM
Reedy added a subscriber: Reedy.Feb 6 2019, 4:57 AM

Looks like we have 3.1.1-1 installed, not 4.1.1-1+0~20180819152012.3+stretch~1.gbpd95942 (which is available)

jijiki claimed this task.Feb 6 2019, 9:04 AM
jijiki triaged this task as Medium priority.
jijiki added projects: serviceops, User-jijiki.
jijiki moved this task from Backlog to Doing on the serviceops board.Feb 6 2019, 12:08 PM
jijiki moved this task from Backlog/Radar to In Progress on the User-jijiki board.Feb 6 2019, 4:50 PM
Tgr added a comment.Feb 6 2019, 6:47 PM

IIRC (we ran into similar issues on Vagrant in T213016: Sometimes Redis does not work on MediaWiki-Vagrant with PHP 7.2) there is no php7.2-redis, just a single php-redis which works with all versions of PHP. The Wikimedia apt repo used to have the wrong version of it pinned, but I believe that has been fixed since.

Reedy added a comment.Feb 6 2019, 6:56 PM

In modules/contint/manifests/packages/php.pp we're doing ensure => latest

reedy@deploy1001:~$ apt-get install -s php-redis
NOTE: This is only a simulation!
      apt-get needs root privileges for real execution.
      Keep also in mind that locking is deactivated,
      so don't depend on the relevance to the real current situation!
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following package was automatically installed and is no longer required:
Use 'apt autoremove' to remove it.
Suggested packages:
The following packages will be upgraded:
1 upgraded, 0 newly installed, 0 to remove and 61 not upgraded.
Inst php-redis [3.1.1-1] (4.1.1-1+0~20180819152012.3+stretch~1.gbpd95942 Wikimedia:9/stretch-wikimedia [amd64])
Conf php-redis (4.1.1-1+0~20180819152012.3+stretch~1.gbpd95942 Wikimedia:9/stretch-wikimedia [amd64])
Dzahn added a comment.Feb 6 2019, 7:18 PM

In modules/contint/manifests/packages/php.pp we're doing ensure => latest

We should probably stop doing that. Surprise upgrades from "latest" have bitten us too much in the past.

Krenair added a subscriber: Krenair.Feb 7 2019, 1:24 PM

In modules/contint/manifests/packages/php.pp we're doing ensure => latest

We should probably stop doing that. Surprise upgrades from "latest" have bitten us too much in the past.

If we don't use latest, won't we instead be bitten quietly and gradually as new servers are brought online (or existing ones reimaged for whatever reason)? Is this sort of stuff tested for and caught more quickly when ops do manual package updates?

Dzahn added a comment.Feb 7 2019, 3:17 PM

If we use "present" (and not a specific version or "latest" either) we would get whatever the latest version is at the moment of installing a new server. A change in version at the time of installing a new server would be more expected than one on a random Sunday?

Mentioned in SAL (#wikimedia-operations) [2019-02-08T15:10:55Z] <jijiki> Upgrading php-redis 4.1.1 to mwmaint1002 - T215376

Mentioned in SAL (#wikimedia-operations) [2019-02-08T15:33:36Z] <_joe_> apt-get upgrade on mwmaint2001 to fix the php installation T215376

Joe added a subscriber: Joe.Feb 8 2019, 3:48 PM

`All the extensions were not upgraded at the time we did the 7.0 => 7.2 transition - my bad! Updating them manually was the only thing that was needed - we needed to update all of them and not just php-redis.

As for using ensure => latest, that would've broken our installation badly and is generally considered bad practice, and something we actively avoid doing in production, where we have centralized means to check and upgrade our debian packages.

Joe closed this task as Resolved.Feb 11 2019, 10:45 AM