PHP 5.6 and 5.5 does not work with Mediawiki 1.26 on Centos 6.6 with default APCu settings
Closed, ResolvedPublic

Description

"PHP Fatal error: Unsupported operand types in /var/www/html/wiki/includes/registration/ExtensionRegistry.php on line 272"
It appears that this is the problem line in ExtensionsRegistry.php

$this->loaded += $info['credits'];
This error occurs on PHP 5.5 and PHP 5.6

It does not occur on PHP 5.4

Maniphest changed the visibility from "Public (No Login Required)" to "Custom Policy".Dec 3 2015, 7:59 PM
Maniphest changed the edit policy from "All Users" to "Custom Policy".
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 3 2015, 7:59 PM
Minddragon changed the title from "" to "PHP 5.6 and 5.5 does not work with Mediawiki 1.26 on Centos 6.6 ".Dec 3 2015, 7:59 PM
Minddragon edited the task description. (Show Details)
Minddragon changed Security from None to Software security bug.
Minddragon edited subscribers, added: Minddragon; removed: Aklapper.
Krenair added a subscriber: Krenair.Dec 3 2015, 8:02 PM

How is this a software security bug?

MaxSem changed the visibility from "Custom Policy" to "Public (No Login Required)".
MaxSem changed Security from Software security bug to None.
MaxSem removed a subscriber: Krenair.
Restricted Application added a subscriber: StudiesWorld. · View Herald TranscriptDec 3 2015, 8:02 PM
Legoktm added a subscriber: Legoktm.

Can you please indicate what extensions and skins you have installed and their versions? One of them likely has an invalid extension.json or skin.json file.

Steps to replicate:

  1. Have a CentOS machine, apache, Mysql, PHP 5.6w (I did yum install php56w php56w-common php56w-pecl* and a few others
  2. Download Mediawiki 1.26 tarball
  3. Ran the initial Mediawiki setup with no extensions enabled
  4. Observed the crash. I couldn't get skins to work as well.
  5. Tried Yum remove php56* and installed PHP55* and got the same thing
  6. Did Yum remove php55* and installed 54* and everything worked including default Mediawiki extensions.

Please note several other people on support forums are reporting similar conditions.

Anomie added a subscriber: Anomie.EditedDec 3 2015, 9:41 PM

I managed to reproduce this with a virtual machine:

Then I found that a package "php56w" does not exist. I'm guessing you're using https://mirror.webtatic.com/yum/el6/latest.rpm as an additional repo? Let's try that.

After poking at it a bit, it looks like ExtensionRegistry decided to use APC for caching and the APC cache is somehow or other corrupting the keys in the stored arrays. Uninstalling php56w-pecl-apcu makes it work by not using APC. Probably this didn't happen in 1.25 because the code removed in https://gerrit.wikimedia.org/r/#/c/226441/ avoided the corruption by only storing strings in APC.

ori added a subscriber: ori.Dec 13 2015, 12:01 AM
ori added a comment.Dec 13 2015, 12:46 AM

Following @Anomie's instructions, I end up with these warnings & notices:

Can you stick a var_dump($data) right after ExtensionRegistry reads out of the APC cache at line 133?

ori added a comment.Dec 13 2015, 1:17 AM

Can you stick a var_dump($data) right after ExtensionRegistry reads out of the APC cache at line 133?

ori added a comment.Dec 13 2015, 1:19 AM

Can you stick a var_dump($data) right after ExtensionRegistry reads out of the APC cache at line 133?

It does appear to be corrupt somehow. Replacing $data = $this->cache->get( $key ); with $data = false; (and thus forcing the data to be computed rather than loaded from the cache) works.

ori added a comment.Dec 13 2015, 1:25 AM
This comment was removed by ori.
ori added a comment.Dec 13 2015, 1:30 AM

Changing apc.serializer = 'default' to apc.serializer = 'php' in /etc/php.d/apcu.ini and then restarting httpd makes the problem go away.

ori added a comment.Dec 13 2015, 1:38 AM

It'd be great to have a minimal reproduction case (e.g. some cooked-up array that reliably triggers this corruption). Then, when APCBagOStuff is first initialized, try to set/get it, and if it is corrupt, default to the old behavior (serialize and deserialize in user code).

ori added a comment.Dec 13 2015, 5:53 AM

I'm pretty certain the bug is the one described here:

@Minddragon, you should edit /etc/php.d/apcu.ini and ensure apc.serializer is set to php rather than default.

Since this is a possible use-after-free in apcu's serialization code, I don't think that we should try to trigger it, as I suggested above. We should just fall back to the old behavior of serializing everything in PHP "user space" when APCu is used with the default serializer.

Change 258734 had a related patch set uploaded (by Ori.livneh):
Work around APCu memory corruption bug

https://gerrit.wikimedia.org/r/258734

@ori thank you guys for figuring it out :)

Change 258734 merged by jenkins-bot:
Work around APCu memory corruption bug

https://gerrit.wikimedia.org/r/258734

Change 258930 had a related patch set uploaded (by Legoktm):
Work around APCu memory corruption bug

https://gerrit.wikimedia.org/r/258930

Change 258930 merged by jenkins-bot:
Work around APCu memory corruption bug

https://gerrit.wikimedia.org/r/258930

Legoktm assigned this task to ori.Dec 14 2015, 7:04 AM
Legoktm closed this task as "Resolved".

Fixed in master and backported to REL1_26. Will be included as part of the 1.26.1 release. Thanks everyone for investigating and fixing this :)

Legoktm changed the title from "PHP 5.6 and 5.5 does not work with Mediawiki 1.26 on Centos 6.6 " to "PHP 5.6 and 5.5 does not work with Mediawiki 1.26 on Centos 6.6 with default APCu settings".Dec 14 2015, 7:05 AM
Legoktm added a project: MediaWiki-Cache.