Request timeout in LuaError::getScriptTraceHtml() leads to shared memory corruption, segfault flood
Open, MediumPublicPRODUCTION ERROR
Actions

Assigned To

Authored By

	ssastry
	Jul 19 2023, 9:56 PM

Description

A timeout led to a flood of PHP errors and segfaults on parse1007, which ended when php-fpm automatically restarted after 800ms.

Error

mwversion: 1.41.0-wmf.17
reqId: 08715471-a750-4647-aad4-8ab04f62bbd6
Find reqId in Logstash

normalized_message

[{reqId}] {exception_url}   Wikimedia\RequestTimeout\RequestTimeoutException: The maximum execution time of {limit} seconds was exceeded

exception.trace

from /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/request-timeout/src/Detail/ExcimerTimerWrapper.php(97)
#0 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/request-timeout/src/Detail/ExcimerTimerWrapper.php(72): Wikimedia\RequestTimeout\Detail\ExcimerTimerWrapper->onTimeout(integer)
#1 /srv/mediawiki/php-1.41.0-wmf.17/includes/parser/Preprocessor_Hash.php(495): Wikimedia\RequestTimeout\Detail\ExcimerTimerWrapper->Wikimedia\RequestTimeout\Detail\{closure}(integer)
#2 /srv/mediawiki/php-1.41.0-wmf.17/includes/parser/Preprocessor_Hash.php(137): Preprocessor_Hash->buildDomTreeArrayFromText(string, integer)
#3 /srv/mediawiki/php-1.41.0-wmf.17/includes/parser/Parser.php(2889): Preprocessor_Hash->preprocessToObj(string, integer)
#4 /srv/mediawiki/php-1.41.0-wmf.17/includes/parser/Parser.php(4220): Parser->preprocessToDom(string)
#5 /srv/mediawiki/php-1.41.0-wmf.17/includes/parser/Parser.php(1625): Parser->finalizeHeadings(string, string, boolean)
#6 /srv/mediawiki/php-1.41.0-wmf.17/includes/parser/Parser.php(700): Parser->internalParse(string)
#7 /srv/mediawiki/php-1.41.0-wmf.17/includes/language/MessageCache.php(1484): Parser->parse(string, MediaWiki\Title\Title, ParserOptions, boolean)
#8 /srv/mediawiki/php-1.41.0-wmf.17/includes/language/Message.php(1428): MessageCache->parse(string, MediaWiki\Title\Title, boolean, boolean, LanguageEn)
#9 /srv/mediawiki/php-1.41.0-wmf.17/includes/language/Message.php(1005): Message->parseText(string)
#10 /srv/mediawiki/php-1.41.0-wmf.17/includes/language/Message.php(1043): Message->format(string)
#11 /srv/mediawiki/php-1.41.0-wmf.17/extensions/Scribunto/includes/Engines/LuaCommon/LuaError.php(85): Message->parse()
#12 /srv/mediawiki/php-1.41.0-wmf.17/extensions/Scribunto/includes/Hooks.php(160): MediaWiki\Extension\Scribunto\Engines\LuaCommon\LuaError->getScriptTraceHtml(array)
#13 /srv/mediawiki/php-1.41.0-wmf.17/includes/parser/Parser.php(3414): MediaWiki\Extension\Scribunto\Hooks::invokeHook(Parser, PPTemplateFrame_Hash, array)
#14 /srv/mediawiki/php-1.41.0-wmf.17/includes/parser/Parser.php(3099): Parser->callParserFunction(PPTemplateFrame_Hash, string, array)
#15 /srv/mediawiki/php-1.41.0-wmf.17/includes/parser/PPFrame_Hash.php(274): Parser->braceSubstitution(array, PPTemplateFrame_Hash)
#16 /srv/mediawiki/php-1.41.0-wmf.17/includes/parser/Parser.php(3290): PPFrame_Hash->expand(PPNode_Hash_Tree)
#17 /srv/mediawiki/php-1.41.0-wmf.17/includes/parser/PPFrame_Hash.php(274): Parser->braceSubstitution(array, PPTemplateFrame_Hash)
#18 /srv/mediawiki/php-1.41.0-wmf.17/includes/parser/Parser.php(3290): PPFrame_Hash->expand(PPNode_Hash_Tree)
#19 /srv/mediawiki/php-1.41.0-wmf.17/includes/parser/PPFrame_Hash.php(274): Parser->braceSubstitution(array, PPFrame_Hash)
#20 /srv/mediawiki/php-1.41.0-wmf.17/includes/parser/Parser.php(2932): PPFrame_Hash->expand(PPNode_Hash_Tree, integer)
#21 /srv/mediawiki/php-1.41.0-wmf.17/includes/parser/Parsoid/Config/DataAccess.php(388): Parser->replaceVariables(string, PPFrame_Hash)
#22 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wikitext/Wikitext.php(43): MediaWiki\Parser\Parsoid\Config\DataAccess->preprocessWikitext(MediaWiki\Parser\Parsoid\Config\PageConfig, ParserOutput, string)
#23 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/TT/TemplateHandler.php(1035): Wikimedia\Parsoid\Wikitext\Wikitext::preprocess(Wikimedia\Parsoid\Config\Env, string)
#24 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/TT/TemplateHandler.php(1078): Wikimedia\Parsoid\Wt2Html\TT\TemplateHandler->expandTemplate(Wikimedia\Parsoid\Wt2Html\TT\TemplateEncapsulator)
#25 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/TT/TemplateHandler.php(1123): Wikimedia\Parsoid\Wt2Html\TT\TemplateHandler->onTemplate(Wikimedia\Parsoid\Tokens\SelfclosingTagTk)
#26 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/TT/TokenHandler.php(150): Wikimedia\Parsoid\Wt2Html\TT\TemplateHandler->onTag(Wikimedia\Parsoid\Tokens\SelfclosingTagTk)
#27 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/TokenTransformManager.php(132): Wikimedia\Parsoid\Wt2Html\TT\TokenHandler->process(array)
#28 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/TokenTransformManager.php(185): Wikimedia\Parsoid\Wt2Html\TokenTransformManager->processChunk(array)
#29 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/ParserPipeline.php(140): Wikimedia\Parsoid\Wt2Html\TokenTransformManager->process(array, array)
#30 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Utils/PipelineUtils.php(106): Wikimedia\Parsoid\Wt2Html\ParserPipeline->parse(array, array)
#31 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/Frame.php(142): Wikimedia\Parsoid\Utils\PipelineUtils::processContentInPipeline(Wikimedia\Parsoid\Config\Env, Wikimedia\Parsoid\Wt2Html\Frame, array, array)
#32 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/TT/AttributeTransformManager.php(71): Wikimedia\Parsoid\Wt2Html\Frame->expand(array, array)
#33 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/TT/AttributeExpander.php(643): Wikimedia\Parsoid\Wt2Html\TT\AttributeTransformManager->process(array)
#34 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/TT/AttributeExpander.php(698): Wikimedia\Parsoid\Wt2Html\TT\AttributeExpander->processComplexAttributes(Wikimedia\Parsoid\Tokens\SelfclosingTagTk)
#35 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/TT/TokenHandler.php(159): Wikimedia\Parsoid\Wt2Html\TT\AttributeExpander->onAny(Wikimedia\Parsoid\Tokens\SelfclosingTagTk)
#36 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/TokenTransformManager.php(132): Wikimedia\Parsoid\Wt2Html\TT\TokenHandler->process(array)
#37 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/TokenTransformManager.php(185): Wikimedia\Parsoid\Wt2Html\TokenTransformManager->processChunk(array)
#38 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/ParserPipeline.php(140): Wikimedia\Parsoid\Wt2Html\TokenTransformManager->process(array, array)
#39 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Utils/PipelineUtils.php(106): Wikimedia\Parsoid\Wt2Html\ParserPipeline->parse(string, array)
#40 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Ext/ParsoidExtensionAPI.php(397): Wikimedia\Parsoid\Utils\PipelineUtils::processContentInPipeline(Wikimedia\Parsoid\Config\Env, Wikimedia\Parsoid\Wt2Html\Frame, string, array)
#41 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Ext/ParsoidExtensionAPI.php(975): Wikimedia\Parsoid\Ext\ParsoidExtensionAPI->wikitextToDOM(string, array, boolean)
#42 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Ext/Gallery/Gallery.php(136): Wikimedia\Parsoid\Ext\ParsoidExtensionAPI->renderMedia(string, array, NULL, boolean, boolean)
#43 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Ext/Gallery/Gallery.php(212): Wikimedia\Parsoid\Ext\Gallery\Gallery::pLine(Wikimedia\Parsoid\Ext\ParsoidExtensionAPI, string, integer, Wikimedia\Parsoid\Ext\Gallery\Opts)
#44 [internal function]: Wikimedia\Parsoid\Ext\Gallery\Gallery->Wikimedia\Parsoid\Ext\Gallery\{closure}(array)
#45 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Ext/Gallery/Gallery.php(210): array_map(Closure, array)
#46 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/TT/ExtensionHandler.php(94): Wikimedia\Parsoid\Ext\Gallery\Gallery->sourceToDom(Wikimedia\Parsoid\Ext\ParsoidExtensionAPI, string, array)
#47 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/TT/ExtensionHandler.php(279): Wikimedia\Parsoid\Wt2Html\TT\ExtensionHandler->onExtension(Wikimedia\Parsoid\Tokens\SelfclosingTagTk)
#48 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/TT/TokenHandler.php(150): Wikimedia\Parsoid\Wt2Html\TT\ExtensionHandler->onTag(Wikimedia\Parsoid\Tokens\SelfclosingTagTk)
#49 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/TokenTransformManager.php(132): Wikimedia\Parsoid\Wt2Html\TT\TokenHandler->process(array)
#50 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/TokenTransformManager.php(195): Wikimedia\Parsoid\Wt2Html\TokenTransformManager->processChunk(array)
#51 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/TokenTransformManager.php(193): Wikimedia\Parsoid\Wt2Html\TokenTransformManager->processChunkily(string, array)
#52 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/TreeBuilder/TreeBuilderStage.php(497): Wikimedia\Parsoid\Wt2Html\TokenTransformManager->processChunkily(string, array)
#53 [internal function]: Wikimedia\Parsoid\Wt2Html\TreeBuilder\TreeBuilderStage->processChunkily(string, array)
#54 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/DOMPostProcessor.php(952): Generator->current()
#55 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/ParserPipeline.php(180): Wikimedia\Parsoid\Wt2Html\DOMPostProcessor->processChunkily(string, array)
#56 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wt2Html/ParserPipelineFactory.php(299): Wikimedia\Parsoid\Wt2Html\ParserPipeline->parseChunkily(string, array)
#57 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Wikitext/ContentModelHandler.php(130): Wikimedia\Parsoid\Wt2Html\ParserPipelineFactory->parse(string)
#58 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Parsoid.php(174): Wikimedia\Parsoid\Wikitext\ContentModelHandler->toDOM(Wikimedia\Parsoid\Ext\ParsoidExtensionAPI)
#59 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Parsoid.php(216): Wikimedia\Parsoid\Parsoid->parseWikitext(MediaWiki\Parser\Parsoid\Config\PageConfig, ParserOutput, array)
#60 /srv/mediawiki/php-1.41.0-wmf.17/includes/parser/Parsoid/ParsoidOutputAccess.php(298): Wikimedia\Parsoid\Parsoid->wikitext2html(MediaWiki\Parser\Parsoid\Config\PageConfig, array, NULL, ParserOutput)
#61 /srv/mediawiki/php-1.41.0-wmf.17/includes/parser/Parsoid/ParsoidOutputAccess.php(469): MediaWiki\Parser\Parsoid\ParsoidOutputAccess->parseInternal(MediaWiki\Parser\Parsoid\Config\PageConfig, array)
#62 /srv/mediawiki/php-1.41.0-wmf.17/includes/parser/Parsoid/ParsoidOutputAccess.php(244): MediaWiki\Parser\Parsoid\ParsoidOutputAccess->parse(MediaWiki\Page\PageStoreRecord, ParserOptions, array, MediaWiki\Revision\RevisionStoreRecord)
#63 /srv/mediawiki/php-1.41.0-wmf.17/includes/Rest/Handler/Helper/HtmlOutputRendererHelper.php(742): MediaWiki\Parser\Parsoid\ParsoidOutputAccess->getParserOutput(MediaWiki\Page\PageStoreRecord, ParserOptions, MediaWiki\Revision\RevisionStoreRecord, integer)
#64 /srv/mediawiki/php-1.41.0-wmf.17/includes/Rest/Handler/Helper/HtmlOutputRendererHelper.php(569): MediaWiki\Rest\Handler\Helper\HtmlOutputRendererHelper->getParserOutputInternal(ParserOptions)
#65 /srv/mediawiki/php-1.41.0-wmf.17/includes/Rest/Handler/Helper/HtmlOutputRendererHelper.php(665): MediaWiki\Rest\Handler\Helper\HtmlOutputRendererHelper->getParserOutput()
#66 /srv/mediawiki/php-1.41.0-wmf.17/includes/Rest/Handler/ParsoidHandler.php(909): MediaWiki\Rest\Handler\Helper\HtmlOutputRendererHelper->getPageBundle()
#67 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/extension/src/Rest/Handler/PageHandler.php(92): MediaWiki\Rest\Handler\ParsoidHandler->wt2html(MediaWiki\Parser\Parsoid\Config\PageConfig, array)
#68 /srv/mediawiki/php-1.41.0-wmf.17/includes/Rest/Router.php(517): MWParsoid\Rest\Handler\PageHandler->execute()
#69 /srv/mediawiki/php-1.41.0-wmf.17/includes/Rest/Router.php(422): MediaWiki\Rest\Router->executeHandler(MWParsoid\Rest\Handler\PageHandler)
#70 /srv/mediawiki/php-1.41.0-wmf.17/includes/Rest/EntryPoint.php(195): MediaWiki\Rest\Router->execute(MediaWiki\Rest\RequestFromGlobals)
#71 /srv/mediawiki/php-1.41.0-wmf.17/includes/Rest/EntryPoint.php(135): MediaWiki\Rest\EntryPoint->execute()
#72 /srv/mediawiki/php-1.41.0-wmf.17/rest.php(31): MediaWiki\Rest\EntryPoint::main()
#73 /srv/mediawiki/w/rest.php(3): require(string)
#74 {main}

Other related errors

Message	Count
[{reqId}] {exception_url} PHP Warning: DOMElement::setAttributeNS() expects parameter 3 to be string, object given	203
(104)Connection reset by peer: AH01075: Error dispatching request to :	45
AH01067: Failed to read FastCGI header	45
[WARNING] [pool www-7.4] child %d exited on signal 11 (SIGSEGV) after %g seconds from start	40
[error] Unexpected structure when adding media info.	24
[WARNING] failed processes threshold (40 in 60 sec) is reached, initiating reload	5

Details

Request URL: https://commons.wikimedia.org/w/rest.php/commons.wikimedia.org/v3/page/pagebundle/User%3ABernd_Schwabe_in_Hannover%2Fgallery/779634554

Related Objects

Mentioned In: T372503: ExcimerProfiler with PHP 8.2 JIT causes occasional memory corruption
T351245: Memory exhaustion fatal causes PHP to segfault with Excimer
Mentioned Here: T389734: Fatal exception of type "Wikimedia\RequestTimeout\EmergencyTimeoutException" errors
T293568: PHP Notice: Undefined offset in wikimedia/remex-html when rendering rest.php error page

Event Timeline

ssastry created this task.Jul 19 2023, 9:56 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 19 2023, 9:56 PM

MSantos moved this task from Needs Triage to Bugs & Crashers on the Parsoid board.Jul 20 2023, 2:11 PM

thcipriani moved this task from Untriaged to Jul 2023 on the Wikimedia-production-error board.Jul 20 2023, 3:29 PM

Krinkle added a project: Content-Transform-Team.Aug 17 2023, 11:42 PM

MSantos assigned this task to Arlolra.Aug 24 2023, 2:20 PM

MSantos removed a project: Content-Transform-Team.

If I'm reading this correctly,

#6 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/remex-html/src/TreeBuilder/BeforeHtml.php(33): Wikimedia\RemexHtml\TreeBuilder\TreeBuilder->insertElement(string, Wikimedia\RemexHtml\Tokenizer\PlainAttributes, boolean, integer, integer)

is,

$this->builder->insertElement( 'html', new PlainAttributes,	false,
				$sourceStart, 0 );

However, the throw from,

#1 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/remex-html/src/DOM/DOMBuilder.php(259): DOMElement->setAttributeNS(NULL, string, Wikimedia\RemexHtml\Tokenizer\Attribute)

is happening in a loop of those new PlainAttributes,

foreach ( $element->attrs->getObjects() as $attr ) {

which, you know, there shouldn't be any. That smells like memory corruption.

Looking earlier in the logs, there's an [error] Unexpected structure when adding media info.,
https://logstash.wikimedia.org/app/discover#/doc/logstash-*/logstash-mediawiki-1-7.0.0-1-2023.07.16?id=j7WzXokBzmHF6YkdaP2q

Before that though, on the same host (parse1007) there's a Wikimedia\RequestTimeout\RequestTimeoutException,
https://logstash.wikimedia.org/app/discover#/doc/logstash-*/logstash-deploy-1-7.0.0-1-2023.07.16?id=-xyzXokBuj8l-JjZaUxD

The trace there indicates it was also during rendering of a gallery,

#39 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Utils/PipelineUtils.php(106): Wikimedia\Parsoid\Wt2Html\ParserPipeline->parse(string, array)
#40 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Ext/ParsoidExtensionAPI.php(397): Wikimedia\Parsoid\Utils\PipelineUtils::processContentInPipeline(Wikimedia\Parsoid\Config\Env, Wikimedia\Parsoid\Wt2Html\Frame, string, array)
#41 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Ext/ParsoidExtensionAPI.php(975): Wikimedia\Parsoid\Ext\ParsoidExtensionAPI->wikitextToDOM(string, array, boolean)
#42 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Ext/Gallery/Gallery.php(136): Wikimedia\Parsoid\Ext\ParsoidExtensionAPI->renderMedia(string, array, NULL, boolean, boolean)
#43 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Ext/Gallery/Gallery.php(212): Wikimedia\Parsoid\Ext\Gallery\Gallery::pLine(Wikimedia\Parsoid\Ext\ParsoidExtensionAPI, string, integer, Wikimedia\Parsoid\Ext\Gallery\Opts)
#44 [internal function]: Wikimedia\Parsoid\Ext\Gallery\Gallery->Wikimedia\Parsoid\Ext\Gallery\{closure}(array)
#45 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/parsoid/src/Ext/Gallery/Gallery.php(210): array_map(Closure, array)

Similar to last time (T293568#7483468), this implicates Excimer

from /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/request-timeout/src/Detail/ExcimerTimerWrapper.php(97)
#0 /srv/mediawiki/php-1.41.0-wmf.17/vendor/wikimedia/request-timeout/src/Detail/ExcimerTimerWrapper.php(72): Wikimedia\RequestTimeout\Detail\ExcimerTimerWrapper->onTimeout(integer)
#1 /srv/mediawiki/php-1.41.0-wmf.17/includes/parser/Preprocessor_Hash.php(495): Wikimedia\RequestTimeout\Detail\ExcimerTimerWrapper->Wikimedia\RequestTimeout\Detail\{closure}(integer)
#2 /srv/mediawiki/php-1.41.0-wmf.17/includes/parser/Preprocessor_Hash.php(137): Preprocessor_Hash->buildDomTreeArrayFromText(string, integer)

/cc @tstarling

Arlolra removed Arlolra as the assignee of this task.Aug 24 2023, 10:05 PM

Arlolra triaged this task as Medium priority.

Arlolra subscribed.

Most likely opcache corruption. Errors started only 1ms after the request timeout at 12:36:03.013, and then there was a flood of segfaults over the next few hundred milliseconds, finishing with "[WARNING] failed processes threshold (40 in 60 sec) is reached, initiating reload" at 12:36:03.800.

It's unlikely the subsequent requests will help us to isolate the bug. I'll update the task description to show details of the timeout request.

The only slightly unusual thing in the timeout stack trace is that it's in a catch block (frame #12), catching a LuaError.

If there was a timeout in Lua, it would have thrown a LuaSandboxTimeoutError, which would have been caught and rethrown as a ScribuntoException. But we see it formatting a LuaError which suggests that there was some other error from Lua.

It's in a Lua module which was invoked from a template which was in a gallery. The page was [[commons:User:Bernd Schwabe in Hannover/gallery]].

tstarling renamed this task from PHP Warning: DOMElement::setAttributeNS() expects parameter 3 to be string, object given to Request timeout in LuaError::getScriptTraceHtml() leads to shared memory corruption, segfault flood.Aug 25 2023, 2:01 AM

tstarling claimed this task.

tstarling changed Request URL from https://commons.wikimedia.org/w/rest.php/commons.wikimedia.org/v3/page/pagebundle/File%3ABrezina_-_Brunelli(...)_-_NARA_-_108875456_(page_1133).jpg/783692778 to https://commons.wikimedia.org/w/rest.php/commons.wikimedia.org/v3/page/pagebundle/User%3ABernd_Schwabe_in_Hannover%2Fgallery/779634554.

tstarling updated the task description. (Show Details)

rmccue mentioned this in T351245: Memory exhaustion fatal causes PHP to segfault with Excimer.Nov 14 2023, 5:48 PM

MSantos removed a project: Parsoid.Dec 14 2023, 3:22 PM

TehKittyCat subscribed.Dec 23 2023, 1:15 AM

tstarling mentioned this in T372503: ExcimerProfiler with PHP 8.2 JIT causes occasional memory corruption.Aug 16 2024, 2:58 AM

Restricted Application added a project: MediaWiki-Platform-Team. · View Herald TranscriptSep 14 2024, 10:08 PM

Krinkle edited projects, added MediaWiki-Platform-Team (Radar); removed MediaWiki-Platform-Team.Sep 14 2024, 10:10 PM

All Excimer segfault bugs are now suspected duplicates of T389734, i.e. glibc timer aliasing, glibc#32833.

@tstarling: This task has been assigned to you a while ago. Is this still a valid (open) task which requires more work, or do you assume this got fixed by T389734?

Request timeout in LuaError::getScriptTraceHtml() leads to shared memory corruption, segfault floodOpen, MediumPublicPRODUCTION ERRORActions