Diagnosing opcache corruption
Please see https://wikitech.wikimedia.org/wiki/User:CDanis/Diagnosing_opcache_corruption
An old error report for the case that began this tracking task
Error: Call to undefined method Monolog\Logger::get() from /srv/mediawiki/php-1.35.0-wmf.19/includes/libs/objectcache/wancache/WANObjectCache.php:1303
#0 /srv/mediawiki/php-1.35.0-wmf.19/extensions/CentralAuth/includes/CentralAuthUser.php(558): WANObjectCache->getWithSetCallback(string, integer, Closure, array) #1 /srv/mediawiki/php-1.35.0-wmf.19/extensions/CentralAuth/includes/CentralAuthUser.php(414): CentralAuthUser->loadFromCache() #2 /srv/mediawiki/php-1.35.0-wmf.19/extensions/CentralAuth/includes/CentralAuthUser.php(589): CentralAuthUser->loadState() #3 /srv/mediawiki/php-1.35.0-wmf.19/extensions/CentralAuth/includes/CentralAuthUser.php(686): CentralAuthUser->getId() #4 /srv/mediawiki/php-1.35.0-wmf.19/extensions/CentralAuth/includes/CentralAuthHooks.php(917): CentralAuthUser->exists() #5 /srv/mediawiki/php-1.35.0-wmf.19/includes/Hooks.php(174): CentralAuthHooks::onGetUserBlock(User, string, NULL) #6 /srv/mediawiki/php-1.35.0-wmf.19/includes/Hooks.php(202): Hooks::callHook(string, array, array, NULL) #7 /srv/mediawiki/php-1.35.0-wmf.19/includes/block/BlockManager.php(167): Hooks::run(string, array) #8 /srv/mediawiki/php-1.35.0-wmf.19/includes/user/User.php(1800): MediaWiki\Block\BlockManager->getUserBlock(User, WebRequest, boolean) #9 /srv/mediawiki/php-1.35.0-wmf.19/includes/user/User.php(2119): User->getBlockedStatus(boolean) #10 /srv/mediawiki/php-1.35.0-wmf.19/includes/block/BlockManager.php(470): User->getBlock() #11 /srv/mediawiki/php-1.35.0-wmf.19/includes/MediaWiki.php(755): MediaWiki\Block\BlockManager->trackBlockWithCookie(User, WebResponse) #12 /srv/mediawiki/php-1.35.0-wmf.19/includes/api/ApiMain.php(548): MediaWiki::preOutputCommit(DerivativeContext) #13 /srv/mediawiki/php-1.35.0-wmf.19/includes/api/ApiMain.php(493): ApiMain->executeActionWithErrorHandling() #14 /srv/mediawiki/php-1.35.0-wmf.19/api.php(84): ApiMain->execute() #15 /srv/mediawiki/w/api.php(3): require(string) #16 {main}
- Request ID: XkWMYwpAIDkAALfuQmcAAACH
- Host: mw1345
- Request Method: GET
- Request URL: commons.wikimedia.org/w/api.php?action=query&generator=allimages&prop=imageinfo&…
- phpversion: 7.2.26-1+0~20191218.33+debian9~1.gbpb5a340+wmf1
- timestamp: 2020-02-13T17:51:28+00:00
Logstash record:
https://logstash.wikimedia.org/app/kibana#/doc/logstash-*/logstash-deploy-2020.02.13/mediawiki?id=AXA_rVssh3Uj6x1zToLY
Source code
CentralAuthUser.php@1.35.0-wmf.19
$cache = MediaWikiServices::getInstance()->getMainWANObjectCache(); $data = $cache->getWithSetCallback( # line 540 $this->getCacheKey( $cache ), $cache::TTL_DAY, function ( $oldValue, &$ttl, array &$setOpts ) { $dbr = CentralAuthUtils::getCentralReplicaDB(); $setOpts += Database::getCacheSetOptions( $dbr ); $this->loadFromDatabase(); $this->loadAttached(); $this->loadGroups(); $data = []; foreach ( self::$mCacheVars as $var ) { $data[$var] = $this->$var; } return $data; }, [ 'pcTTL' => $cache::TTL_PROC_LONG, 'version' => self::VERSION ] # line 558 ); # line 559
WANObjectCache.php@1.35.0-wmf.19
final public function getWithSetCallback( $key, $ttl, $callback, array $opts = [], array $cbParams = [] ) { $version = $opts['version'] ?? null; $pcTTL = $opts['pcTTL'] ?? self::TTL_UNCACHEABLE; $pCache = ( $pcTTL >= 0 ) ? $this->getProcessCache( $opts['pcGroup'] ?? self::PC_PRIMARY ) # line 1296 : null; if ( $pCache && $this->callbackDepth == 0 ) { $cached = $pCache->get( $this->getProcessCacheKey( $key, $version ), $pcTTL, false ); # line 1303 if ( $cached !== false ) { $this->logger->debug( "getWithSetCallback($key): process cache hit" ); # line 1305 return $cached; } }
Notes
Stuff is very messed up here:
- The error Call to undefined method Monolog\Logger::get() is attribued to line WANObjectCache.php:L1303, which is calling $pCache->get() where $pCache is an instance of MapCacheLRU. It is meant to be calling MapCacheLRU->get().
- For some reason the engine is thinking about Monolog\Logger, which is strange. For what it's worth, there is an instance of that object "near by" – two lines down on line 1305. Could it be confusing the two??
- The stack trace says WANObjectCache->getWithSetCallback was called from CentralAuthUser.php:L558. This is also odd, because that line is neither the start nor the end of the function call. See snippet above. Looks like this may be off by a few as well.
Might be opcache corruption.