Page MenuHomePhabricator
Create Task
Maniphest T316611

No localisation cache found for English. Please run maintenance/rebuildLocalisationCache.php
Closed, InvalidPublicBUG REPORT
Actions

Description

A few hundred errors per hour: https://logstash.wikimedia.org/goto/ca11f5e07ffa274656d7dbf5548e34d4
Seems somehow related to GrowthExperiments community configuration:

from /srv/mediawiki/php-1.39.0-wmf.26/includes/language/LocalisationCache.php(515)
#0 /srv/mediawiki/php-1.39.0-wmf.26/includes/language/LocalisationCache.php(373): LocalisationCache->initLanguage(string)
#1 /srv/mediawiki/php-1.39.0-wmf.26/includes/language/LocalisationCache.php(314): LocalisationCache->loadItem(string, string)
#2 /srv/mediawiki/php-1.39.0-wmf.26/includes/language/Language.php(664): LocalisationCache->getItem(string, string)
#3 /srv/mediawiki/php-1.39.0-wmf.26/includes/title/MediaWikiTitleCodec.php(119): Language->needsGenderDistinction()
#4 /srv/mediawiki/php-1.39.0-wmf.26/includes/title/MediaWikiTitleCodec.php(156): MediaWikiTitleCodec->getNamespaceName(integer, string)
#5 /srv/mediawiki/php-1.39.0-wmf.26/includes/cache/LinkCache.php(114): MediaWikiTitleCodec->formatTitle(integer, string)
#6 /srv/mediawiki/php-1.39.0-wmf.26/includes/cache/LinkCache.php(234): LinkCache->getCacheKey(array, boolean)
#7 /srv/mediawiki/php-1.39.0-wmf.26/includes/page/PageStore.php(177): LinkCache->isBadLink(array)
#8 /srv/mediawiki/php-1.39.0-wmf.26/includes/page/PageStore.php(154): MediaWiki\Page\PageStore->getPageByNameViaLinkCache(integer, string, integer)
#9 /srv/mediawiki/php-1.39.0-wmf.26/includes/page/PageStore.php(326): MediaWiki\Page\PageStore->getPageByName(integer, string, integer)
#10 /srv/mediawiki/php-1.39.0-wmf.26/includes/Title.php(4103): MediaWiki\Page\PageStore->getPageByReference(Title, integer)
#11 /srv/mediawiki/php-1.39.0-wmf.26/includes/Title.php(1098): Title->getFieldFromPageStore(string, integer)
#12 /srv/mediawiki/php-1.39.0-wmf.26/includes/Title.php(1115): Title->getContentModel()
#13 /srv/mediawiki/php-1.39.0-wmf.26/includes/Title.php(1573): Title->hasContentModel(string)
#14 /srv/mediawiki/php-1.39.0-wmf.26/extensions/GrowthExperiments/includes/Config/WikiPageConfig.php(63): Title->isSiteJsonConfigPage()
#15 /srv/mediawiki/php-1.39.0-wmf.26/extensions/GrowthExperiments/includes/Config/WikiPageConfig.php(86): GrowthExperiments\Config\WikiPageConfig->getConfigTitle()
#16 /srv/mediawiki/php-1.39.0-wmf.26/extensions/GrowthExperiments/includes/Config/WikiPageConfig.php(153): GrowthExperiments\Config\WikiPageConfig->getConfigData(integer)
#17 /srv/mediawiki/php-1.39.0-wmf.26/extensions/GrowthExperiments/includes/Config/GrowthExperimentsMultiConfig.php(113): GrowthExperiments\Config\WikiPageConfig->hasWithFlags(string, integer)
#18 /srv/mediawiki/php-1.39.0-wmf.26/extensions/GrowthExperiments/includes/Config/GrowthExperimentsMultiConfig.php(96): GrowthExperiments\Config\GrowthExperimentsMultiConfig->getWithFlags(string)
#19 /srv/mediawiki/php-1.39.0-wmf.26/extensions/GrowthExperiments/ServiceWiring.php(468): GrowthExperiments\Config\GrowthExperimentsMultiConfig->get(string)
#20 /srv/mediawiki/php-1.39.0-wmf.26/vendor/wikimedia/services/src/ServiceContainer.php(447): Wikimedia\Services\ServiceContainer::{closure}(MediaWiki\MediaWikiServices)
#21 /srv/mediawiki/php-1.39.0-wmf.26/vendor/wikimedia/services/src/ServiceContainer.php(411): Wikimedia\Services\ServiceContainer->createService(string)
#22 /srv/mediawiki/php-1.39.0-wmf.26/includes/MediaWikiServices.php(299): Wikimedia\Services\ServiceContainer->getService(string)
#23 /srv/mediawiki/php-1.39.0-wmf.26/vendor/wikimedia/services/src/ServiceContainer.php(419): MediaWiki\MediaWikiServices->getService(string)
#24 /srv/mediawiki/php-1.39.0-wmf.26/extensions/GrowthExperiments/includes/GrowthExperimentsServices.php(175): Wikimedia\Services\ServiceContainer->get(string)
#25 /srv/mediawiki/php-1.39.0-wmf.26/extensions/GrowthExperiments/ServiceWiring.php(444): GrowthExperiments\GrowthExperimentsServices->getMentorProviderWikitext()
#26 /srv/mediawiki/php-1.39.0-wmf.26/vendor/wikimedia/services/src/ServiceContainer.php(447): Wikimedia\Services\ServiceContainer::{closure}(MediaWiki\MediaWikiServices)
#27 /srv/mediawiki/php-1.39.0-wmf.26/vendor/wikimedia/services/src/ServiceContainer.php(411): Wikimedia\Services\ServiceContainer->createService(string)
#28 /srv/mediawiki/php-1.39.0-wmf.26/includes/MediaWikiServices.php(299): Wikimedia\Services\ServiceContainer->getService(string)
#29 /srv/mediawiki/php-1.39.0-wmf.26/vendor/wikimedia/services/src/ServiceContainer.php(419): MediaWiki\MediaWikiServices->getService(string)
#30 /srv/mediawiki/php-1.39.0-wmf.26/extensions/GrowthExperiments/includes/GrowthExperimentsServices.php(167): Wikimedia\Services\ServiceContainer->get(string)
#31 /srv/mediawiki/php-1.39.0-wmf.26/extensions/GrowthExperiments/ServiceWiring.php(426): GrowthExperiments\GrowthExperimentsServices->getMentorProvider()
#32 /srv/mediawiki/php-1.39.0-wmf.26/vendor/wikimedia/services/src/ServiceContainer.php(447): Wikimedia\Services\ServiceContainer::{closure}(MediaWiki\MediaWikiServices)
#33 /srv/mediawiki/php-1.39.0-wmf.26/vendor/wikimedia/services/src/ServiceContainer.php(411): Wikimedia\Services\ServiceContainer->createService(string)
#34 /srv/mediawiki/php-1.39.0-wmf.26/includes/MediaWikiServices.php(299): Wikimedia\Services\ServiceContainer->getService(string)
#35 /srv/mediawiki/php-1.39.0-wmf.26/vendor/wikimedia/services/src/ServiceContainer.php(419): MediaWiki\MediaWikiServices->getService(string)
#36 /srv/mediawiki/php-1.39.0-wmf.26/vendor/wikimedia/object-factory/src/ObjectFactory.php(211): Wikimedia\Services\ServiceContainer->get(string)
#37 /srv/mediawiki/php-1.39.0-wmf.26/vendor/wikimedia/object-factory/src/ObjectFactory.php(152): Wikimedia\ObjectFactory\ObjectFactory::getObjectFromSpec(array, array)
#38 /srv/mediawiki/php-1.39.0-wmf.26/includes/HookContainer/HookContainer.php(473): Wikimedia\ObjectFactory\ObjectFactory->createObject(array)
#39 /srv/mediawiki/php-1.39.0-wmf.26/includes/HookContainer/HookContainer.php(156): MediaWiki\HookContainer\HookContainer->getHandlers(string, array)
#40 /srv/mediawiki/php-1.39.0-wmf.26/includes/HookContainer/HookRunner.php(4133): MediaWiki\HookContainer\HookContainer->run(string, array)
#41 /srv/mediawiki/php-1.39.0-wmf.26/includes/user/DefaultOptionsLookup.php(110): MediaWiki\HookContainer\HookRunner->onUserGetDefaultOptions(array)
#42 /srv/mediawiki/php-1.39.0-wmf.26/includes/user/UserOptionsManager.php(596): MediaWiki\User\DefaultOptionsLookup->getDefaultOptions()
#43 /srv/mediawiki/php-1.39.0-wmf.26/includes/user/UserOptionsManager.php(498): MediaWiki\User\UserOptionsManager->loadOriginalOptions(User, integer, NULL)
#44 /srv/mediawiki/php-1.39.0-wmf.26/includes/user/UserOptionsManager.php(148): MediaWiki\User\UserOptionsManager->loadUserOptions(User, integer)
#45 /srv/mediawiki/php-1.39.0-wmf.26/includes/context/RequestContext.php(460): MediaWiki\User\UserOptionsManager->getOption(User, string)
#46 /srv/mediawiki/php-1.39.0-wmf.26/includes/StubUserLang.php(34): RequestContext->getLanguage()
#47 /srv/mediawiki/php-1.39.0-wmf.26/includes/StubObject.php(223): StubUserLang->_newObject()
#48 /srv/mediawiki/php-1.39.0-wmf.26/includes/StubObject.php(103): StubObject->_unstub(string, integer)
#49 /srv/mediawiki/php-1.39.0-wmf.26/includes/content/ContentHandler.php(727): StubObject::unstub(StubUserLang)
#50 /srv/mediawiki/php-1.39.0-wmf.26/includes/Title.php(3944): ContentHandler->getPageLanguage(Title)
#51 /srv/mediawiki/php-1.39.0-wmf.26/includes/OutputPage.php(2538): Title->getPageLanguage()
#52 /srv/mediawiki/php-1.39.0-wmf.26/includes/OutputPage.php(2670): OutputPage->addAcceptLanguage()
#53 /srv/mediawiki/php-1.39.0-wmf.26/includes/OutputPage.php(2781): OutputPage->sendCacheControl()
#54 /srv/mediawiki/php-1.39.0-wmf.26/includes/MediaWiki.php(1008): OutputPage->output()
#55 /srv/mediawiki/php-1.39.0-wmf.26/includes/MediaWiki.php(875): MediaWiki->maybeDoHttpsRedirect()
#56 /srv/mediawiki/php-1.39.0-wmf.26/includes/MediaWiki.php(562): MediaWiki->main()
#57 /srv/mediawiki/php-1.39.0-wmf.26/index.php(50): MediaWiki->run()
#58 /srv/mediawiki/php-1.39.0-wmf.26/index.php(46): wfIndexMain()
#59 /srv/mediawiki/w/index.php(3): require(string)
#60 {main}

Started happening at UTC 14h:

Screenshot Capture - 2022-08-29 - 16-30-17.png (228×671 px, 12 KB)

Related Objects

Event Timeline

Tgr created this task.Aug 29 2022, 11:31 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 29 2022, 11:31 PM
Tgr added a comment.Aug 30 2022, 12:46 AM

LCStore uses PHP serialization so possibly connected to T316601: PHP Warning: Erroneous data format for unserializing 'Wikimedia\Rdbms\MySQLPrimaryPos', except that this error seems to be PHP 7.4 only, and 7.4 should have no trouble reading the 7.2 serialization format...

Tgr added a comment.EditedAug 30 2022, 1:25 AM

The high-level control flow is: after generating the response, MediaWiki needs to figure out the language, for the HTTP headers -> unstub user language -> load user settings -> call UserGetDefaultOptions hook -> initialize HelpPanelHooks -> load GrowthExperimentsMentorManager as dependency -> load GrowthExperimentsMentorProvider[Wikitext] as dependency -> check GEHomepageMentorsList in community configuration -> call WikiPageConfig to load the configuration page -> check with Title::isSiteJsonConfigPage() that it really is a confiugration page -> pull page data from LinkCache -> LinkCache calls MediaWikiTitleCodec::formatTitle to convert namespace + title into a prefixed title string -> needs to load namespaceGenderAliases from localisation cache.

That's a scary looking dependency chain (also it could start anywhere, not just when sending the output - the user language is unstubbed on demand, whenever it's first needed, which is typically whenever something is done with Message objects), but I'm not sure there's anything wrong with it, so maybe this is not related to GrowthExperiments.

The immediate reason for the error is probably LCStoreCDB::get() returning null, but that's not captured in the stack trace, and there are no other errors during the same request.

Tgr added a project: Wikimedia-production-error.Aug 30 2022, 4:27 AM
Joe closed this task as Invalid.Aug 30 2022, 5:23 AM
Joe subscribed.

These errors come from servers we didn't finish installing yesterday, but are still in the process of getting a complete mediawiki push because we ran out of time, apologies for the confusion.

These errors derive from monitoring requests to these newly installed hosts, none has to do with serving live traffic nor has any real value.

Stashbot added a comment.Aug 30 2022, 6:53 AM

Mentioned in SAL (#wikimedia-operations) [2022-08-30T06:53:41Z] <_joe_> running scap pull on parse1* T316611

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL