Page MenuHomePhabricator

"Memcached::touch(): touch is only supported with binary protocol" from wikitech as it's not running HHVM
Closed, ResolvedPublic

Description

Warning: Memcached::touch(): touch is only supported with binary protocol in /vagrant/mediawiki/includes/objectcache/MemcachedPeclBagOStuff.php on line 232

Call Stack:
    0.0005     240320   1. {main}() /var/www/w/MWScript.php:0
    0.0122     322928   2. require_once('/vagrant/mediawiki/extensions/Flow/maintenance/convertNamespaceFromWikitext.php') /var/www/w/MWScript.php:95
    0.0166     698560   3. require_once('/vagrant/mediawiki/maintenance/doMaintenance.php') /vagrant/mediawiki/extensions/Flow/maintenance/convertNamespaceFromWikitext.php:100
    0.6389   22430512   4. ConvertNamespaceFromWikitext->execute() /vagrant/mediawiki/maintenance/doMaintenance.php:106
    0.7097   23167264   5. wfGetDB() /vagrant/mediawiki/extensions/Flow/maintenance/convertNamespaceFromWikitext.php:70
    0.7178   23737872   6. LoadBalancer->getConnection() /vagrant/mediawiki/includes/GlobalFunctions.php:3149
    0.7377   25508488   7. LoadBalancer->getReadOnlyReason() /vagrant/mediawiki/includes/db/loadbalancer/LoadBalancer.php:595
    0.7377   25508808   8. LoadBalancer->masterRunningReadOnly() /vagrant/mediawiki/includes/db/loadbalancer/LoadBalancer.php:1315
    0.7380   25510848   9. WANObjectCache->getWithSetCallback() /vagrant/mediawiki/includes/db/loadbalancer/LoadBalancer.php:1346
    0.7380   25511784  10. WANObjectCache->doGetWithSetCallback() /vagrant/mediawiki/includes/libs/objectcache/WANObjectCache.php:847
    0.7421   25518752  11. MemcachedPeclBagOStuff->changeTTL() /vagrant/mediawiki/includes/libs/objectcache/WANObjectCache.php:966
    0.7421   25518800  12. Memcached->touch() /vagrant/mediawiki/includes/objectcache/MemcachedPeclBagOStuff.php:232

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 20 2016, 12:29 AM
aaron added a subscriber: aaron.Aug 20 2016, 4:00 AM

Looks like the memcached client devs were too lazy to add ASCII support to this on older versions. Seems to be fine in newer ones (http://php.net/manual/en/memcached.touch.php).

We can't just use BINARY because we use twemproxy, which is ASCII only.

@SBisson it is present in production if you use Zend. I tested with HHVM (which doesn't have the issue) earlier without considering this.

On Vagrant, we probably can just use BINARY though.

Change 306488 had a related patch set uploaded (by Mattflaschen):
Memcached PECL: Allow using binary protocol

https://gerrit.wikimedia.org/r/306488

Change 306490 had a related patch set uploaded (by Mattflaschen):
Use binary protocol with memcached

https://gerrit.wikimedia.org/r/306490

Change 306488 merged by jenkins-bot:
Memcached PECL: Allow using binary protocol

https://gerrit.wikimedia.org/r/306488

Change 306490 merged by jenkins-bot:
Use binary protocol with memcached

https://gerrit.wikimedia.org/r/306490

I've got the same issue running /srv/mediawiki/php-1.28.0-wmf.20/extensions/WikimediaMaintenance/createExtensionTables.php, see T127792#2690863.

Reedy added a subscriber: Reedy.Feb 16 2017, 12:43 PM

There's a load of these on wikitech

mwdeploy@silver:/var/log/apache2$ grep -c "touch is only supported with binary protocol" error.log
1404
mwdeploy@silver:/var/log/apache2$ zgrep -c "touch is only supported with binary protocol" error.log.*
error.log.1:5161
error.log.10.gz:5166
error.log.11.gz:5450
error.log.12.gz:5526
error.log.13.gz:5356
error.log.14.gz:5271
error.log.15.gz:5106
error.log.16.gz:5141
error.log.17.gz:4902
error.log.18.gz:5408
error.log.19.gz:5346
error.log.20.gz:5157
error.log.21.gz:4761
error.log.22.gz:4883
error.log.23.gz:4990
error.log.24.gz:4711
error.log.25.gz:5623
error.log.26.gz:5538
error.log.27.gz:5114
error.log.28.gz:4655
error.log.29.gz:4758
error.log.2.gz:5211
error.log.30.gz:4753
error.log.3.gz:5234
error.log.4.gz:5401
error.log.5.gz:5202
error.log.6.gz:5190
error.log.7.gz:4968
error.log.8.gz:5024
error.log.9.gz:5048
Krinkle triaged this task as Normal priority.Apr 7 2017, 11:00 PM
Krinkle moved this task from Untriaged to libs/objectcache on the MediaWiki-Cache board.
Jdforrester-WMF renamed this task from "Memcached::touch(): touch is only supported with binary protocol" to "Memcached::touch(): touch is only supported with binary protocol" from wikitech as it's not running HHVM.May 4 2017, 7:18 PM
Jdforrester-WMF added subscribers: Andrew, chasemp, bd808, mmodell.
bd808 added a comment.May 4 2017, 7:51 PM

Just so my last analysis from T158613#3218329 doesn't get lost, I'm reposting it here:

So the reason this doesn't make MediaWiki hhvm-only is because normal non-WMF deploys of mediawiki don't include twemproxy?

So, any php-memcached released since 2014-03-12 should work fine with the current MediaWiki core code. We just happen to live in the dark ages due to silver being a Trusty host. Jessie ships php-memcached v2.2.0 which should support touch() via the ASCII protocol that we use.

$ php -v
PHP 5.6.30-0+deb8u1 (cli) (built: Feb  8 2017 08:50:21)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
    with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies
    with Xdebug v2.2.5, Copyright (c) 2002-2014, by Derick Rethans
$ php --re memcached | head
Extension [ <persistent> extension #53 memcached version 2.2.0 ] {

  - INI {
    Entry [ memcached.sess_locking <ALL> ]
      Current = '1'
    }
    Entry [ memcached.sess_consistent_hash <ALL> ]
      Current = '0'
    }
    Entry [ memcached.sess_binary <ALL> ]
$ dpkg-installed | grep memcached
libmemcached11:amd64 1.0.18-4
libmemcachedutil2:amd64 1.0.18-4
memcached            1.4.21-1.1+deb8u1
php5-memcached       2.2.0-2
demon added a subscriber: demon.Jun 13 2017, 7:43 PM

I thought we shut this error up at least. It's back on wmf.5

bd808 removed Mattflaschen-WMF as the assignee of this task.Jun 13 2017, 7:54 PM

I thought we shut this error up at least. It's back on wmf.5

We tried but switching the protocol didn't work because twemproxy doesn't support the binary protocol. We had to revert almost immediately because edits and logins were broken.

Restricted Application added a project: Cloud-Services. · View Herald TranscriptJun 13 2017, 7:55 PM

As I said on IRC: why do we even attempt this codepath when we're not in binary mode?

As I said on IRC: why do we even attempt this codepath when we're not in binary mode?

We would have to check the php-memcached version too because in v2.2.0RC1 the behavior flips and binary mode raises the warning and text mode works. A better question if we are worried about compatibility might be why are we using touch() at all.

demon added a comment.Jun 14 2017, 5:12 AM

I mean the performance gain over a get/set is pretty obvious. But if we can support it reliably we might need some sort of fallback.

bd808 moved this task from Triage to Backlog on the Cloud-Services board.Jun 17 2017, 8:01 PM

Change 379677 had a related patch set uploaded (by Chad; owner: Chad):
[mediawiki/core@master] Possibly re-attempt memcached touch when we've failed

https://gerrit.wikimedia.org/r/379677

Change 379677 abandoned by Chad:
Possibly re-attempt memcached touch when we've failed

Reason:
Meh

https://gerrit.wikimedia.org/r/379677

Looks like T186802 is another instance of this. I only noticed it recently but can't swear that it's new.

bd808 changed the status of subtask T98813: Move wikitech (silver) to HHVM from Stalled to Open.Feb 22 2018, 6:41 PM
Krinkle closed this task as Resolved.Aug 8 2018, 9:14 PM
Krinkle claimed this task.
Krinkle added a subscriber: Krinkle.

I believe this issue was specific to PHP5. However, as of about two months ago, WMF no longer runs MediaWiki on PHP5 anywhere. Per T98813, Wikitech was switched to HHVM.

mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:11 PM