Page MenuHomePhabricator
Create Task
Maniphest T373668

MediaWiki\Revision\RevisionAccessException: Failed to load data blob from {address} for revision {revision}. If this problem persist, use the findBadBlobs maintenance script to investigate the issue and mark bad blobs.
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

Error
normalized_message
[{reqId}] {exception_url}   MediaWiki\Revision\RevisionAccessException: Failed to load data blob from {address} for revision {revision}. If this problem persist, use the findBadBlobs maintenance script to investigate the issue and mark bad blobs.
exception.trace
from /srv/mediawiki/php-1.43.0-wmf.20/includes/Revision/RevisionStore.php(1212)
#0 /srv/mediawiki/php-1.43.0-wmf.20/includes/Revision/RevisionStore.php(1519): MediaWiki\Revision\RevisionStore->loadSlotContent(MediaWiki\Revision\SlotRecord, null, null, null, int)
#1 [internal function]: MediaWiki\Revision\RevisionStore->MediaWiki\Revision\{closure}(MediaWiki\Revision\SlotRecord)
#2 /srv/mediawiki/php-1.43.0-wmf.20/includes/Revision/SlotRecord.php(324): call_user_func(Closure, MediaWiki\Revision\SlotRecord)
#3 /srv/mediawiki/php-1.43.0-wmf.20/includes/Revision/RevisionRecord.php(158): MediaWiki\Revision\SlotRecord->getContent()
#4 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3764): MediaWiki\Revision\RevisionRecord->getContent(string)
#5 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3629): MediaWiki\Parser\Parser::statelessFetchTemplate(MediaWiki\Title\Title, MediaWiki\Parser\Parser)
#6 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3523): MediaWiki\Parser\Parser->fetchTemplateAndTitle(MediaWiki\Title\Title)
#7 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3262): MediaWiki\Parser\Parser->getTemplateDom(MediaWiki\Title\Title)
#8 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPFrame_Hash.php(276): MediaWiki\Parser\Parser->braceSubstitution(array, PPTemplateFrame_Hash)
#9 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPTemplateFrame_Hash.php(98): PPFrame_Hash->expand(PPNode_Hash_Tree, int)
#10 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3334): PPTemplateFrame_Hash->cachedExpand(string, PPNode_Hash_Tree)
#11 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPFrame_Hash.php(276): MediaWiki\Parser\Parser->braceSubstitution(array, PPFrame_Hash)
#12 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPTemplateFrame_Hash.php(161): PPFrame_Hash->expand(PPNode_Hash_Tree, int)
#13 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPTemplateFrame_Hash.php(188): PPTemplateFrame_Hash->getNumberedArgument(string)
#14 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3944): PPTemplateFrame_Hash->getArgument(string)
#15 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPFrame_Hash.php(293): MediaWiki\Parser\Parser->argSubstitution(array, PPTemplateFrame_Hash)
#16 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3059): PPFrame_Hash->expand(PPNode_Hash_Tree)
#17 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPFrame_Hash.php(276): MediaWiki\Parser\Parser->braceSubstitution(array, PPTemplateFrame_Hash)
#18 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPTemplateFrame_Hash.php(178): PPFrame_Hash->expand(PPNode_Hash_Tree, int)
#19 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPTemplateFrame_Hash.php(190): PPTemplateFrame_Hash->getNamedArgument(string)
#20 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPTemplateFrame_Hash.php(122): PPTemplateFrame_Hash->getArgument(string)
#21 /srv/mediawiki/php-1.43.0-wmf.20/extensions/Scribunto/includes/Engines/LuaCommon/LuaEngine.php(714): PPTemplateFrame_Hash->getArguments()
#22 /srv/mediawiki/php-1.43.0-wmf.20/extensions/Scribunto/includes/Engines/LuaSandbox/LuaSandboxCallback.php(31): MediaWiki\Extension\Scribunto\Engines\LuaCommon\LuaEngine->getAllExpandedArguments(string)
#23 [internal function]: MediaWiki\Extension\Scribunto\Engines\LuaSandbox\LuaSandboxCallback->__call(string, array)
#24 /srv/mediawiki/php-1.43.0-wmf.20/extensions/Scribunto/includes/Engines/LuaSandbox/LuaSandboxInterpreter.php(136): LuaSandboxFunction->call(LuaSandboxFunction)
#25 /srv/mediawiki/php-1.43.0-wmf.20/extensions/Scribunto/includes/Engines/LuaCommon/LuaEngine.php(312): MediaWiki\Extension\Scribunto\Engines\LuaSandbox\LuaSandboxInterpreter->callFunction(LuaSandboxFunction, LuaSandboxFunction)
#26 /srv/mediawiki/php-1.43.0-wmf.20/extensions/Scribunto/includes/Engines/LuaCommon/LuaModule.php(75): MediaWiki\Extension\Scribunto\Engines\LuaCommon\LuaEngine->executeFunctionChunk(LuaSandboxFunction, PPTemplateFrame_Hash)
#27 /srv/mediawiki/php-1.43.0-wmf.20/extensions/Scribunto/includes/Hooks.php(175): MediaWiki\Extension\Scribunto\Engines\LuaCommon\LuaModule->invoke(string, PPTemplateFrame_Hash)
#28 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3463): MediaWiki\Extension\Scribunto\Hooks->invokeHook(MediaWiki\Parser\Parser, PPTemplateFrame_Hash, array)
#29 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3146): MediaWiki\Parser\Parser->callParserFunction(PPTemplateFrame_Hash, string, array)
#30 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPFrame_Hash.php(276): MediaWiki\Parser\Parser->braceSubstitution(array, PPTemplateFrame_Hash)
#31 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3337): PPFrame_Hash->expand(PPNode_Hash_Tree)
#32 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPFrame_Hash.php(276): MediaWiki\Parser\Parser->braceSubstitution(array, PPTemplateFrame_Hash)
#33 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3337): PPFrame_Hash->expand(PPNode_Hash_Tree)
#34 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPFrame_Hash.php(276): MediaWiki\Parser\Parser->braceSubstitution(array, PPFrame_Hash)
#35 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(2980): PPFrame_Hash->expand(PPNode_Hash_Tree, int)
#36 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(1626): MediaWiki\Parser\Parser->replaceVariables(string)
#37 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(728): MediaWiki\Parser\Parser->internalParse(string)
#38 /srv/mediawiki/php-1.43.0-wmf.20/includes/content/WikitextContentHandler.php(384): MediaWiki\Parser\Parser->parse(string, MediaWiki\Title\Title, ParserOptions, bool, bool, null)
#39 /srv/mediawiki/php-1.43.0-wmf.20/includes/content/ContentHandler.php(1693): MediaWiki\Content\WikitextContentHandler->fillParserOutput(MediaWiki\Content\WikitextContent, MediaWiki\Content\Renderer\ContentParseParams, MediaWiki\Parser\ParserOutput)
#40 /srv/mediawiki/php-1.43.0-wmf.20/includes/content/Renderer/ContentRenderer.php(79): MediaWiki\Content\ContentHandler->getParserOutput(MediaWiki\Content\WikitextContent, MediaWiki\Content\Renderer\ContentParseParams)
#41 /srv/mediawiki/php-1.43.0-wmf.20/includes/Revision/RenderedRevision.php(262): MediaWiki\Content\Renderer\ContentRenderer->getParserOutput(MediaWiki\Content\WikitextContent, MediaWiki\Title\Title, MediaWiki\Revision\MutableRevisionRecord, ParserOptions, array)
#42 /srv/mediawiki/php-1.43.0-wmf.20/includes/Revision/RenderedRevision.php(234): MediaWiki\Revision\RenderedRevision->getSlotParserOutputUncached(MediaWiki\Content\WikitextContent, array)
#43 /srv/mediawiki/php-1.43.0-wmf.20/includes/Revision/RevisionRenderer.php(234): MediaWiki\Revision\RenderedRevision->getSlotParserOutput(string, array)
#44 /srv/mediawiki/php-1.43.0-wmf.20/includes/Revision/RevisionRenderer.php(170): MediaWiki\Revision\RevisionRenderer->combineSlotOutput(MediaWiki\Revision\RenderedRevision, ParserOptions, array)
#45 [internal function]: MediaWiki\Revision\RevisionRenderer->MediaWiki\Revision\{closure}(MediaWiki\Revision\RenderedRevision, array)
#46 /srv/mediawiki/php-1.43.0-wmf.20/includes/Revision/RenderedRevision.php(199): call_user_func(Closure, MediaWiki\Revision\RenderedRevision, array)
#47 /srv/mediawiki/php-1.43.0-wmf.20/extensions/TemplateData/includes/Hooks.php(116): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput(array)
#48 /srv/mediawiki/php-1.43.0-wmf.20/includes/HookContainer/HookContainer.php(159): MediaWiki\Extension\TemplateData\Hooks->onMultiContentSave(MediaWiki\Revision\RenderedRevision, MediaWiki\User\User, MediaWiki\CommentStore\CommentStoreComment, int, MediaWiki\Storage\PageUpdateStatus)
#49 /srv/mediawiki/php-1.43.0-wmf.20/includes/HookContainer/HookRunner.php(2662): MediaWiki\HookContainer\HookContainer->run(string, array)
#50 /srv/mediawiki/php-1.43.0-wmf.20/includes/Storage/PageUpdater.php(894): MediaWiki\HookContainer\HookRunner->onMultiContentSave(MediaWiki\Revision\RenderedRevision, MediaWiki\User\User, MediaWiki\CommentStore\CommentStoreComment, int, MediaWiki\Storage\PageUpdateStatus)
#51 /srv/mediawiki/php-1.43.0-wmf.20/includes/page/MovePage.php(977): MediaWiki\Storage\PageUpdater->saveRevision(MediaWiki\CommentStore\CommentStoreComment)
#52 /srv/mediawiki/php-1.43.0-wmf.20/includes/page/MovePage.php(644): MediaWiki\Page\MovePage->moveToInternal(MediaWiki\User\User, MediaWiki\Title\Title, string, bool, array)
#53 /srv/mediawiki/php-1.43.0-wmf.20/includes/page/MovePage.php(492): MediaWiki\Page\MovePage->moveUnsafe(MediaWiki\User\User, string, bool, array)
#54 /srv/mediawiki/php-1.43.0-wmf.20/includes/specials/SpecialMovePage.php(789): MediaWiki\Page\MovePage->moveIfAllowed(MediaWiki\User\User, string, bool)
#55 /srv/mediawiki/php-1.43.0-wmf.20/includes/specials/SpecialMovePage.php(238): MediaWiki\Specials\SpecialMovePage->doSubmit()
#56 /srv/mediawiki/php-1.43.0-wmf.20/extensions/Translate/src/PageTranslation/MoveTranslatableBundleSpecialPage.php(160): MediaWiki\Specials\SpecialMovePage->execute(null)
#57 /srv/mediawiki/php-1.43.0-wmf.20/includes/specialpage/SpecialPage.php(719): MediaWiki\Extension\Translate\PageTranslation\MoveTranslatableBundleSpecialPage->execute(null)
#58 /srv/mediawiki/php-1.43.0-wmf.20/includes/specialpage/SpecialPageFactory.php(1705): MediaWiki\SpecialPage\SpecialPage->run(null)
#59 /srv/mediawiki/php-1.43.0-wmf.20/includes/actions/ActionEntryPoint.php(502): MediaWiki\SpecialPage\SpecialPageFactory->executePath(string, MediaWiki\Context\RequestContext)
#60 /srv/mediawiki/php-1.43.0-wmf.20/includes/actions/ActionEntryPoint.php(145): MediaWiki\Actions\ActionEntryPoint->performRequest()
#61 /srv/mediawiki/php-1.43.0-wmf.20/includes/MediaWikiEntryPoint.php(200): MediaWiki\Actions\ActionEntryPoint->execute()
#62 /srv/mediawiki/php-1.43.0-wmf.20/index.php(58): MediaWiki\MediaWikiEntryPoint->run()
#63 /srv/mediawiki/w/index.php(3): require(string)
#64 {main}
exception.previous.trace
from /srv/mediawiki/php-1.43.0-wmf.20/includes/Storage/SqlBlobStore.php(279)
#0 /srv/mediawiki/php-1.43.0-wmf.20/includes/Revision/RevisionStore.php(1208): MediaWiki\Storage\SqlBlobStore->getBlob(string, int)
#1 /srv/mediawiki/php-1.43.0-wmf.20/includes/Revision/RevisionStore.php(1519): MediaWiki\Revision\RevisionStore->loadSlotContent(MediaWiki\Revision\SlotRecord, null, null, null, int)
#2 [internal function]: MediaWiki\Revision\RevisionStore->MediaWiki\Revision\{closure}(MediaWiki\Revision\SlotRecord)
#3 /srv/mediawiki/php-1.43.0-wmf.20/includes/Revision/SlotRecord.php(324): call_user_func(Closure, MediaWiki\Revision\SlotRecord)
#4 /srv/mediawiki/php-1.43.0-wmf.20/includes/Revision/RevisionRecord.php(158): MediaWiki\Revision\SlotRecord->getContent()
#5 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3764): MediaWiki\Revision\RevisionRecord->getContent(string)
#6 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3629): MediaWiki\Parser\Parser::statelessFetchTemplate(MediaWiki\Title\Title, MediaWiki\Parser\Parser)
#7 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3523): MediaWiki\Parser\Parser->fetchTemplateAndTitle(MediaWiki\Title\Title)
#8 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3262): MediaWiki\Parser\Parser->getTemplateDom(MediaWiki\Title\Title)
#9 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPFrame_Hash.php(276): MediaWiki\Parser\Parser->braceSubstitution(array, PPTemplateFrame_Hash)
#10 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPTemplateFrame_Hash.php(98): PPFrame_Hash->expand(PPNode_Hash_Tree, int)
#11 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3334): PPTemplateFrame_Hash->cachedExpand(string, PPNode_Hash_Tree)
#12 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPFrame_Hash.php(276): MediaWiki\Parser\Parser->braceSubstitution(array, PPFrame_Hash)
#13 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPTemplateFrame_Hash.php(161): PPFrame_Hash->expand(PPNode_Hash_Tree, int)
#14 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPTemplateFrame_Hash.php(188): PPTemplateFrame_Hash->getNumberedArgument(string)
#15 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3944): PPTemplateFrame_Hash->getArgument(string)
#16 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPFrame_Hash.php(293): MediaWiki\Parser\Parser->argSubstitution(array, PPTemplateFrame_Hash)
#17 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3059): PPFrame_Hash->expand(PPNode_Hash_Tree)
#18 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPFrame_Hash.php(276): MediaWiki\Parser\Parser->braceSubstitution(array, PPTemplateFrame_Hash)
#19 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPTemplateFrame_Hash.php(178): PPFrame_Hash->expand(PPNode_Hash_Tree, int)
#20 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPTemplateFrame_Hash.php(190): PPTemplateFrame_Hash->getNamedArgument(string)
#21 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPTemplateFrame_Hash.php(122): PPTemplateFrame_Hash->getArgument(string)
#22 /srv/mediawiki/php-1.43.0-wmf.20/extensions/Scribunto/includes/Engines/LuaCommon/LuaEngine.php(714): PPTemplateFrame_Hash->getArguments()
#23 /srv/mediawiki/php-1.43.0-wmf.20/extensions/Scribunto/includes/Engines/LuaSandbox/LuaSandboxCallback.php(31): MediaWiki\Extension\Scribunto\Engines\LuaCommon\LuaEngine->getAllExpandedArguments(string)
#24 [internal function]: MediaWiki\Extension\Scribunto\Engines\LuaSandbox\LuaSandboxCallback->__call(string, array)
#25 /srv/mediawiki/php-1.43.0-wmf.20/extensions/Scribunto/includes/Engines/LuaSandbox/LuaSandboxInterpreter.php(136): LuaSandboxFunction->call(LuaSandboxFunction)
#26 /srv/mediawiki/php-1.43.0-wmf.20/extensions/Scribunto/includes/Engines/LuaCommon/LuaEngine.php(312): MediaWiki\Extension\Scribunto\Engines\LuaSandbox\LuaSandboxInterpreter->callFunction(LuaSandboxFunction, LuaSandboxFunction)
#27 /srv/mediawiki/php-1.43.0-wmf.20/extensions/Scribunto/includes/Engines/LuaCommon/LuaModule.php(75): MediaWiki\Extension\Scribunto\Engines\LuaCommon\LuaEngine->executeFunctionChunk(LuaSandboxFunction, PPTemplateFrame_Hash)
#28 /srv/mediawiki/php-1.43.0-wmf.20/extensions/Scribunto/includes/Hooks.php(175): MediaWiki\Extension\Scribunto\Engines\LuaCommon\LuaModule->invoke(string, PPTemplateFrame_Hash)
#29 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3463): MediaWiki\Extension\Scribunto\Hooks->invokeHook(MediaWiki\Parser\Parser, PPTemplateFrame_Hash, array)
#30 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3146): MediaWiki\Parser\Parser->callParserFunction(PPTemplateFrame_Hash, string, array)
#31 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPFrame_Hash.php(276): MediaWiki\Parser\Parser->braceSubstitution(array, PPTemplateFrame_Hash)
#32 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3337): PPFrame_Hash->expand(PPNode_Hash_Tree)
#33 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPFrame_Hash.php(276): MediaWiki\Parser\Parser->braceSubstitution(array, PPTemplateFrame_Hash)
#34 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(3337): PPFrame_Hash->expand(PPNode_Hash_Tree)
#35 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/PPFrame_Hash.php(276): MediaWiki\Parser\Parser->braceSubstitution(array, PPFrame_Hash)
#36 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(2980): PPFrame_Hash->expand(PPNode_Hash_Tree, int)
#37 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(1626): MediaWiki\Parser\Parser->replaceVariables(string)
#38 /srv/mediawiki/php-1.43.0-wmf.20/includes/parser/Parser.php(728): MediaWiki\Parser\Parser->internalParse(string)
#39 /srv/mediawiki/php-1.43.0-wmf.20/includes/content/WikitextContentHandler.php(384): MediaWiki\Parser\Parser->parse(string, MediaWiki\Title\Title, ParserOptions, bool, bool, null)
#40 /srv/mediawiki/php-1.43.0-wmf.20/includes/content/ContentHandler.php(1693): MediaWiki\Content\WikitextContentHandler->fillParserOutput(MediaWiki\Content\WikitextContent, MediaWiki\Content\Renderer\ContentParseParams, MediaWiki\Parser\ParserOutput)
#41 /srv/mediawiki/php-1.43.0-wmf.20/includes/content/Renderer/ContentRenderer.php(79): MediaWiki\Content\ContentHandler->getParserOutput(MediaWiki\Content\WikitextContent, MediaWiki\Content\Renderer\ContentParseParams)
#42 /srv/mediawiki/php-1.43.0-wmf.20/includes/Revision/RenderedRevision.php(262): MediaWiki\Content\Renderer\ContentRenderer->getParserOutput(MediaWiki\Content\WikitextContent, MediaWiki\Title\Title, MediaWiki\Revision\MutableRevisionRecord, ParserOptions, array)
#43 /srv/mediawiki/php-1.43.0-wmf.20/includes/Revision/RenderedRevision.php(234): MediaWiki\Revision\RenderedRevision->getSlotParserOutputUncached(MediaWiki\Content\WikitextContent, array)
#44 /srv/mediawiki/php-1.43.0-wmf.20/includes/Revision/RevisionRenderer.php(234): MediaWiki\Revision\RenderedRevision->getSlotParserOutput(string, array)
#45 /srv/mediawiki/php-1.43.0-wmf.20/includes/Revision/RevisionRenderer.php(170): MediaWiki\Revision\RevisionRenderer->combineSlotOutput(MediaWiki\Revision\RenderedRevision, ParserOptions, array)
#46 [internal function]: MediaWiki\Revision\RevisionRenderer->MediaWiki\Revision\{closure}(MediaWiki\Revision\RenderedRevision, array)
#47 /srv/mediawiki/php-1.43.0-wmf.20/includes/Revision/RenderedRevision.php(199): call_user_func(Closure, MediaWiki\Revision\RenderedRevision, array)
#48 /srv/mediawiki/php-1.43.0-wmf.20/extensions/TemplateData/includes/Hooks.php(116): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput(array)
#49 /srv/mediawiki/php-1.43.0-wmf.20/includes/HookContainer/HookContainer.php(159): MediaWiki\Extension\TemplateData\Hooks->onMultiContentSave(MediaWiki\Revision\RenderedRevision, MediaWiki\User\User, MediaWiki\CommentStore\CommentStoreComment, int, MediaWiki\Storage\PageUpdateStatus)
#50 /srv/mediawiki/php-1.43.0-wmf.20/includes/HookContainer/HookRunner.php(2662): MediaWiki\HookContainer\HookContainer->run(string, array)
#51 /srv/mediawiki/php-1.43.0-wmf.20/includes/Storage/PageUpdater.php(894): MediaWiki\HookContainer\HookRunner->onMultiContentSave(MediaWiki\Revision\RenderedRevision, MediaWiki\User\User, MediaWiki\CommentStore\CommentStoreComment, int, MediaWiki\Storage\PageUpdateStatus)
#52 /srv/mediawiki/php-1.43.0-wmf.20/includes/page/MovePage.php(977): MediaWiki\Storage\PageUpdater->saveRevision(MediaWiki\CommentStore\CommentStoreComment)
#53 /srv/mediawiki/php-1.43.0-wmf.20/includes/page/MovePage.php(644): MediaWiki\Page\MovePage->moveToInternal(MediaWiki\User\User, MediaWiki\Title\Title, string, bool, array)
#54 /srv/mediawiki/php-1.43.0-wmf.20/includes/page/MovePage.php(492): MediaWiki\Page\MovePage->moveUnsafe(MediaWiki\User\User, string, bool, array)
#55 /srv/mediawiki/php-1.43.0-wmf.20/includes/specials/SpecialMovePage.php(789): MediaWiki\Page\MovePage->moveIfAllowed(MediaWiki\User\User, string, bool)
#56 /srv/mediawiki/php-1.43.0-wmf.20/includes/specials/SpecialMovePage.php(238): MediaWiki\Specials\SpecialMovePage->doSubmit()
#57 /srv/mediawiki/php-1.43.0-wmf.20/extensions/Translate/src/PageTranslation/MoveTranslatableBundleSpecialPage.php(160): MediaWiki\Specials\SpecialMovePage->execute(null)
#58 /srv/mediawiki/php-1.43.0-wmf.20/includes/specialpage/SpecialPage.php(719): MediaWiki\Extension\Translate\PageTranslation\MoveTranslatableBundleSpecialPage->execute(null)
#59 /srv/mediawiki/php-1.43.0-wmf.20/includes/specialpage/SpecialPageFactory.php(1705): MediaWiki\SpecialPage\SpecialPage->run(null)
#60 /srv/mediawiki/php-1.43.0-wmf.20/includes/actions/ActionEntryPoint.php(502): MediaWiki\SpecialPage\SpecialPageFactory->executePath(string, MediaWiki\Context\RequestContext)
#61 /srv/mediawiki/php-1.43.0-wmf.20/includes/actions/ActionEntryPoint.php(145): MediaWiki\Actions\ActionEntryPoint->performRequest()
#62 /srv/mediawiki/php-1.43.0-wmf.20/includes/MediaWikiEntryPoint.php(200): MediaWiki\Actions\ActionEntryPoint->execute()
#63 /srv/mediawiki/php-1.43.0-wmf.20/index.php(58): MediaWiki\MediaWikiEntryPoint->run()
#64 /srv/mediawiki/w/index.php(3): require(string)
#65 {main}
Impact

Error on moving pages (and perhaps on saving pages).

Notes

Seen on Special:MovePage but also in some requests to api.php. This is occurring for at least the last 30 days but there is a spike (~300) today (August 30).

To reproduce, try https://test.wikipedia.org/wiki/Special:MovePage/TestNotTemp and renaming the page.

Details

Request URL
https://test.wikipedia.org/w/index.php?action=submit&title=*
Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Write old_id values to file instead of deleting rowsmediawiki/extensions/WikimediaMaintenancemaster+8 -5
Customize query in gerrit

Related Objects

Event Timeline

kostajh created this task.Aug 30 2024, 10:25 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 30 2024, 10:25 AM
kostajh updated the task description. (Show Details)Aug 30 2024, 10:26 AM
kostajh added a project: DBA.
kostajh updated the task description. (Show Details)
Michael subscribed.Aug 30 2024, 10:29 AM
Ladsgroup added subscribers: Zabe, Ladsgroup.Aug 30 2024, 10:34 AM

@Zabe might be caused by your maint script run?

Ladsgroup added a comment.Aug 30 2024, 10:39 AM

Matches T183490#10104662

Zabe added a comment.Aug 30 2024, 11:50 AM

Ok. There are now some revisions which are broken. Sadly its quite a few, but on the other hand it at least is less than 1% or so. The reasons moves are broken is because the latest revision of https://test.wikipedia.org/wiki/Template:Redirect_template is broken, which is it tries to load. But it is only a random symptom and not interesting in itself.

After some digging I at least realised what happened.

My script was written under the assumption that to rows in the content table never share the same content_address. But apparently this is not the case.

wikiadmin2023@10.64.32.82(testwiki)> select c1.content_id, c2.content_id, c1.content_address from content c1, content c2 where c1.content_address = c2.content_address and c1.content_id != c2.content_id limit 1 \G
*************************** 1. row ***************************
     content_id: 1669
     content_id: 167537
content_address: tt:3870
1 row in set (1.153 sec)

wikiadmin2023@10.64.32.82(testwiki)>

So what happened was, it went through the rows of the content table until it reaches the first of multiple rows referencing a certain row in the text table. It then updates the row in the content table to directly reference external storage and the deleting the corresponding row from the text table. But now when it reaches the second content row which references this row in the text table, that no longer exists. The script does nothing in that case, but more importantly, this results in that revision being no longer accessible.

Ladsgroup added a comment.Aug 30 2024, 12:05 PM

I can get the binlogs for the deletions and updates and use that to fix the 3K cases that got broken because of this. Can you work on fixing the script itself?

Zabe added a comment.Aug 30 2024, 1:07 PM

So yeah. There are apparently fully duped rows in the content table (besides the autoincrement id) which is not supposed to happen in MCR but still is the case.

wikiadmin2023@10.64.0.173(enwiki)> select * from content where content_id in (44, 693777302) \G
*************************** 1. row ***************************
     content_id: 44
   content_size: 143
   content_sha1: t1lll5musb1qh58a5p7fr5tbshxx4rt
  content_model: 1
content_address: tt:761401374
*************************** 2. row ***************************
     content_id: 693777302
   content_size: 143
   content_sha1: t1lll5musb1qh58a5p7fr5tbshxx4rt
  content_model: 1
content_address: tt:761401374
2 rows in set (0.002 sec)

wikiadmin2023@10.64.0.173(enwiki)>

So in order to make the script work with this, the only really practical approach which comes to my mind is to do the update and delete in two steps. In the first step we go through the content table and update it to directly reference external storage, but instead of deleting the referenced rows from the text table. We are going to just store the ids of them somewhere and only after updating all rows in the content table we delete the corresponding text rows. By doing that, the text row still exists when reaching the second row in the content table referencing it.

There would be other approaches like doing the migration page by page or so (and then going through all revisions of that page and only then doing the deletions from the text table, since the duped rows apparently always belong to the same page). But that feels very clunky.

gerritbot added a comment.Aug 30 2024, 1:43 PM

Change #1069196 had a related patch set uploaded (by Zabe; author: Zabe):

[mediawiki/extensions/WikimediaMaintenance@master] Write old_id values to file instead of deleting rows

https://gerrit.wikimedia.org/r/1069196

gerritbot added a project: Patch-For-Review.Aug 30 2024, 1:43 PM
gerritbot added a comment.Aug 30 2024, 5:34 PM

Change #1069196 abandoned by Zabe:

[mediawiki/extensions/WikimediaMaintenance@master] Write old_id values to file instead of deleting rows

Reason:

lets use a bloom filter instead

https://gerrit.wikimedia.org/r/1069196

Maintenance_bot removed a project: Patch-For-Review.Aug 30 2024, 6:30 PM
Umherirrender merged a task: T373718: [testwiki-wmf.20] MediaWiki pages cannot be edited - Failed to load data blob from Unable to fetch blob error .Aug 30 2024, 9:17 PM
Umherirrender added subscribers: Etonkovidova, • dancy, bd808 and 2 others.
Zabe mentioned this in T373761: access to zh.wikipedia.beta.wmflabs.org fails with ServerError 500.Sep 1 2024, 2:00 PM
ABran-WMF triaged this task as Medium priority.Sep 2 2024, 6:13 AM
ABran-WMF moved this task from Triage to In progress on the DBA board.
Bugreporter merged a task: T373774: Village Pump at test.wikipedia is broken.Sep 2 2024, 8:13 AM
Bugreporter added a subscriber: Koavf.
Johannnes89 subscribed.Sep 2 2024, 8:49 AM
Etonkovidova mentioned this in T373831: [testwiki-wmf.20] Structured discussion (Flow) pages display Internal error.Sep 3 2024, 12:12 AM
Aklapper removed a project: Release-Engineering-Team.Sep 3 2024, 7:42 AM
Ladsgroup added a comment.Sep 3 2024, 12:41 PM

I'm fixing the existing issues in testwiki now

Ladsgroup added a comment.Sep 3 2024, 1:59 PM

So I first got the list of missing text ids:

select content_id,  content_address from content where content_address like 'tt:%';

Put it in a file.

Then I went through binlog of a replica (on RBR) and splitting any UPDATEs to content table of testwiki

Then with this python script (P68607) I produced the UPDATE statements to run in production. The result was P68608

Ran it on master of s3 and now there is no rows in content table starting with tt:

wikiadmin2023@10.64.48.160(testwiki)> select * from content where content_address like 'tt:%';
Empty set (0.694 sec)

The bug still is being reproduced but I assume it's probably because of caching (I think we cache for a week?) will see.

Zabe added a comment.Sep 3 2024, 2:12 PM

Its not caching. You missed to add the es: prefix.

wikiadmin2023@10.64.132.14(testwiki)> select count(*) from content where content_address like 'DB%';
+----------+
| count(*) |
+----------+
|     3809 |
+----------+
1 row in set (0.169 sec)

wikiadmin2023@10.64.132.14(testwiki)>
Zabe added a comment.Sep 3 2024, 2:14 PM

I think running

UPDATE content SET content_address = CONCAT('es:', content_address) WHERE content_address LIKE 'DB%';

should fix the issue.

Ladsgroup added a comment.Sep 3 2024, 2:48 PM

oh thanks. Done.

Zabe merged a task: T373831: [testwiki-wmf.20] Structured discussion (Flow) pages display Internal error.Sep 3 2024, 2:57 PM
Ladsgroup added a comment.Sep 3 2024, 3:21 PM

I deleted the cache entry for https://test.wikipedia.org/wiki/Template:Redirect_template to allow moves again. It will all expire in a day. But if there is any specific one that needs invalidation right now, here is how:

$cache = \MediaWiki\MediaWikiServices::getInstance()->getMainWANObjectCache();
$key = $cache->makeKey('revision-slots', '', 77562, 406941);
$cache->delete( $key );
Ladsgroup closed this task as Resolved.Sep 4 2024, 10:43 AM
Ladsgroup claimed this task.
Ladsgroup moved this task from In progress to Done on the DBA board.

Back to zero now: https://logstash.wikimedia.org/goto/a3640e8aacb4197ea9fa08bc645002b8

grafik.png (1,035×333 px, 19 KB)

Dreamy_Jazz subscribed.Sep 5 2024, 9:21 AM

This appears to also be broken on the beta wikis. See https://en.wikipedia.beta.wmflabs.org/w/index.php?title=Fox&diff=prev&oldid=629015

Zabe added a comment.Sep 5 2024, 9:48 AM

tbh I would just mark them as bad on beta cluster.

Ladsgroup added a comment.Sep 5 2024, 10:19 AM
In T373668#10120652, @Zabe wrote:

tbh I would just mark them as bad on beta cluster.

Agreed.

Zabe added a comment.Sep 11 2024, 3:34 PM
In T373668#10120652, @Zabe wrote:

tbh I would just mark them as bad on beta cluster.

Did that.

Dreamy_Jazz added a comment.Sep 11 2024, 3:35 PM
In T373668#10137854, @Zabe wrote:
In T373668#10120652, @Zabe wrote:

tbh I would just mark them as bad on beta cluster.

Did that.

:D

Bugreporter merged a task: T374756: Fatal exception of type "MediaWiki\Revision\RevisionAccessException".Sep 14 2024, 8:04 AM
Bugreporter added a subscriber: Toadette.
AntiCompositeNumber mentioned this in T374760: Unable to edit any page on www.mediawiki.org with MediaWiki\Revision\RevisionAccessException.Sep 14 2024, 1:14 PM
Xqt merged a task: T373761: access to zh.wikipedia.beta.wmflabs.org fails with ServerError 500.Sep 19 2024, 8:27 AM
Xqt added subscribers: Xqt, Stang.
SunAfterRain subscribed.Sep 27 2024, 6:27 AM

https://zh.wikipedia.beta.wmflabs.org/w/index.php?title=Template:Navbox&oldid=8047 has an error message: "The text of this revision is missing or corrupted." Although the error content is somewhat different, could it possibly be related to this task?

Stang unsubscribed.Oct 24 2024, 3:22 AM
Zabe mentioned this in T393237: Some en.wikipedia pageviews fatal "RevisionAccessException: Failed to load data blob from {address} for revision {revision}.".May 10 2025, 12:30 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits