Page MenuHomePhabricator

MWException: Parser state cleared while parsing. Did you call Parser::parse recursively?
Closed, ResolvedPublicPRODUCTION ERROR

Description

Error
normalized_message
[{reqId}] {exception_url}   MWException: Parser state cleared while parsing. Did you call Parser::parse recursively? Lock is held by: #0 /srv/mediawiki/php-1.38.0-wmf.17/includes/parser/Parser.php(666): Parser->lock()
exception.trace
from /srv/mediawiki/php-1.38.0-wmf.17/includes/parser/Parser.php(6320)
#0 /srv/mediawiki/php-1.38.0-wmf.17/includes/parser/Parser.php(666): Parser->lock()
#1 /srv/mediawiki/php-1.38.0-wmf.17/includes/content/WikitextContentHandler.php(294): Parser->parse(string, Title, ParserOptions, boolean, boolean, integer)
#2 /srv/mediawiki/php-1.38.0-wmf.17/includes/content/ContentHandler.php(1723): WikitextContentHandler->fillParserOutput(WikitextContent, MediaWiki\Content\Renderer\ContentParseParams, ParserOutput)
#3 /srv/mediawiki/php-1.38.0-wmf.17/includes/content/Renderer/ContentRenderer.php(47): ContentHandler->getParserOutput(WikitextContent, MediaWiki\Content\Renderer\ContentParseParams)
#4 /srv/mediawiki/php-1.38.0-wmf.17/includes/Revision/RenderedRevision.php(271): MediaWiki\Content\Renderer\ContentRenderer->getParserOutput(WikitextContent, Title, integer, ParserOptions, boolean)
#5 /srv/mediawiki/php-1.38.0-wmf.17/includes/Revision/RenderedRevision.php(238): MediaWiki\Revision\RenderedRevision->getSlotParserOutputUncached(WikitextContent, boolean)
#6 /srv/mediawiki/php-1.38.0-wmf.17/includes/Revision/RevisionRenderer.php(221): MediaWiki\Revision\RenderedRevision->getSlotParserOutput(string, array)
#7 /srv/mediawiki/php-1.38.0-wmf.17/includes/Revision/RevisionRenderer.php(158): MediaWiki\Revision\RevisionRenderer->combineSlotOutput(MediaWiki\Revision\RenderedRevision, array)
#8 [internal function]: MediaWiki\Revision\RevisionRenderer->MediaWiki\Revision\{closure}(MediaWiki\Revision\RenderedRevision, array)
#9 /srv/mediawiki/php-1.38.0-wmf.17/includes/Revision/RenderedRevision.php(200): call_user_func(Closure, MediaWiki\Revision\RenderedRevision, array)
#10 /srv/mediawiki/php-1.38.0-wmf.17/includes/poolcounter/PoolWorkArticleView.php(137): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput()
#11 /srv/mediawiki/php-1.38.0-wmf.17/includes/poolcounter/PoolCounterWork.php(152): PoolWorkArticleView->doWork()
#12 /srv/mediawiki/php-1.38.0-wmf.17/includes/page/ParserOutputAccess.php(279): PoolCounterWork->execute()
#13 /srv/mediawiki/php-1.38.0-wmf.17/includes/filerepo/file/LocalFile.php(2554): MediaWiki\Page\ParserOutputAccess->getParserOutput(MediaWiki\Page\PageStoreRecord, ParserOptions)
#14 /srv/mediawiki/php-1.38.0-wmf.17/extensions/CommonsMetadata/src/DataCollector.php(258): LocalFile->getDescriptionText(LanguageEn)
#15 /srv/mediawiki/php-1.38.0-wmf.17/extensions/CommonsMetadata/src/DataCollector.php(96): CommonsMetadata\DataCollector->getDescriptionText(LocalFile, LanguageEn)
#16 /srv/mediawiki/php-1.38.0-wmf.17/extensions/CommonsMetadata/src/HookHandler.php(75): CommonsMetadata\DataCollector->collect(array, LocalFile)
#17 /srv/mediawiki/php-1.38.0-wmf.17/includes/HookContainer/HookContainer.php(338): CommonsMetadata\HookHandler::onGetExtendedMetadata(array, LocalFile, DerivativeContext, boolean, integer)
#18 /srv/mediawiki/php-1.38.0-wmf.17/includes/HookContainer/HookContainer.php(137): MediaWiki\HookContainer\HookContainer->callLegacyHook(string, array, array, array)
#19 /srv/mediawiki/php-1.38.0-wmf.17/includes/HookContainer/HookRunner.php(1810): MediaWiki\HookContainer\HookContainer->run(string, array)
#20 /srv/mediawiki/php-1.38.0-wmf.17/includes/media/FormatMetadata.php(1842): MediaWiki\HookContainer\HookRunner->onGetExtendedMetadata(array, LocalFile, DerivativeContext, boolean, integer)
#21 /srv/mediawiki/php-1.38.0-wmf.17/includes/media/FormatMetadata.php(1759): FormatMetadata->getExtendedMetadataFromHook(LocalFile, array, integer)
#22 /srv/mediawiki/php-1.38.0-wmf.17/extensions/PageImages/includes/Hooks/ParserFileProcessingHookHandlers.php(369): FormatMetadata->fetchExtendedMetadata(LocalFile)
#23 /srv/mediawiki/php-1.38.0-wmf.17/extensions/PageImages/includes/Hooks/ParserFileProcessingHookHandlers.php(350): PageImages\Hooks\ParserFileProcessingHookHandlers->fetchFileMetadata(LocalFile)
#24 /srv/mediawiki/php-1.38.0-wmf.17/extensions/PageImages/includes/Hooks/ParserFileProcessingHookHandlers.php(202): PageImages\Hooks\ParserFileProcessingHookHandlers->isImageFree(string)
#25 /srv/mediawiki/php-1.38.0-wmf.17/extensions/PageImages/includes/Hooks/ParserFileProcessingHookHandlers.php(148): PageImages\Hooks\ParserFileProcessingHookHandlers->findBestImages(array)
#26 /srv/mediawiki/php-1.38.0-wmf.17/extensions/PageImages/includes/Hooks/ParserFileProcessingHookHandlers.php(66): PageImages\Hooks\ParserFileProcessingHookHandlers->doParserAfterTidy(Parser, string)
#27 /srv/mediawiki/php-1.38.0-wmf.17/includes/HookContainer/HookContainer.php(338): PageImages\Hooks\ParserFileProcessingHookHandlers::onParserAfterTidy(Parser, string)
#28 /srv/mediawiki/php-1.38.0-wmf.17/includes/HookContainer/HookContainer.php(137): MediaWiki\HookContainer\HookContainer->callLegacyHook(string, array, array, array)
#29 /srv/mediawiki/php-1.38.0-wmf.17/includes/HookContainer/HookRunner.php(2827): MediaWiki\HookContainer\HookContainer->run(string, array)
#30 /srv/mediawiki/php-1.38.0-wmf.17/includes/parser/Parser.php(1701): MediaWiki\HookContainer\HookRunner->onParserAfterTidy(Parser, string)
#31 /srv/mediawiki/php-1.38.0-wmf.17/includes/parser/Parser.php(693): Parser->internalParseHalfParsed(string, boolean, boolean)
#32 /srv/mediawiki/php-1.38.0-wmf.17/includes/content/WikitextContentHandler.php(294): Parser->parse(string, Title, ParserOptions, boolean, boolean, integer)
#33 /srv/mediawiki/php-1.38.0-wmf.17/includes/content/ContentHandler.php(1723): WikitextContentHandler->fillParserOutput(WikitextContent, MediaWiki\Content\Renderer\ContentParseParams, ParserOutput)
#34 /srv/mediawiki/php-1.38.0-wmf.17/includes/content/Renderer/ContentRenderer.php(47): ContentHandler->getParserOutput(WikitextContent, MediaWiki\Content\Renderer\ContentParseParams)
#35 /srv/mediawiki/php-1.38.0-wmf.17/includes/Revision/RenderedRevision.php(271): MediaWiki\Content\Renderer\ContentRenderer->getParserOutput(WikitextContent, Title, integer, ParserOptions, boolean)
#36 /srv/mediawiki/php-1.38.0-wmf.17/includes/Revision/RenderedRevision.php(238): MediaWiki\Revision\RenderedRevision->getSlotParserOutputUncached(WikitextContent, boolean)
#37 /srv/mediawiki/php-1.38.0-wmf.17/includes/Revision/RevisionRenderer.php(221): MediaWiki\Revision\RenderedRevision->getSlotParserOutput(string, array)
#38 /srv/mediawiki/php-1.38.0-wmf.17/includes/Revision/RevisionRenderer.php(158): MediaWiki\Revision\RevisionRenderer->combineSlotOutput(MediaWiki\Revision\RenderedRevision, array)
#39 [internal function]: MediaWiki\Revision\RevisionRenderer->MediaWiki\Revision\{closure}(MediaWiki\Revision\RenderedRevision, array)
#40 /srv/mediawiki/php-1.38.0-wmf.17/includes/Revision/RenderedRevision.php(200): call_user_func(Closure, MediaWiki\Revision\RenderedRevision, array)
#41 /srv/mediawiki/php-1.38.0-wmf.17/includes/poolcounter/PoolWorkArticleView.php(137): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput()
#42 /srv/mediawiki/php-1.38.0-wmf.17/includes/poolcounter/PoolCounterWork.php(162): PoolWorkArticleView->doWork()
#43 /srv/mediawiki/php-1.38.0-wmf.17/includes/page/ParserOutputAccess.php(279): PoolCounterWork->execute()
#44 /srv/mediawiki/php-1.38.0-wmf.17/includes/page/Article.php(705): MediaWiki\Page\ParserOutputAccess->getParserOutput(WikiPage, ParserOptions, MediaWiki\Revision\RevisionStoreRecord, integer)
#45 /srv/mediawiki/php-1.38.0-wmf.17/includes/page/Article.php(517): Article->generateContentOutput(User, ParserOptions, integer, OutputPage, array)
#46 /srv/mediawiki/php-1.38.0-wmf.17/includes/actions/ViewAction.php(80): Article->view()
#47 /srv/mediawiki/php-1.38.0-wmf.17/includes/MediaWiki.php(543): ViewAction->show()
#48 /srv/mediawiki/php-1.38.0-wmf.17/includes/MediaWiki.php(320): MediaWiki->performAction(Article, Title)
#49 /srv/mediawiki/php-1.38.0-wmf.17/includes/MediaWiki.php(903): MediaWiki->performRequest()
#50 /srv/mediawiki/php-1.38.0-wmf.17/includes/MediaWiki.php(563): MediaWiki->main()
#51 /srv/mediawiki/php-1.38.0-wmf.17/index.php(53): MediaWiki->run()
#52 /srv/mediawiki/php-1.38.0-wmf.17/index.php(46): wfIndexMain()
#53 /srv/mediawiki/w/index.php(3): require(string)
#54 {main}
Notes
  • Seeing a lot of these from group0 and group1 wikis on 1.38.0-wmf.17
  • Seems to happen more frequently on commonswiki

Details

Event Timeline

@tstarling can investigate this better but the PageImages patch may potentially need a revert to figure out the right strategy here since the metadata fetch is requiring a (recurisve) wikitext parse compared to before where the links-update hook invocation wasn't part of the main page parse. /cc @Krinkle

I'm not a big fan of a hook named onParserModifyImageHTML in any case. This is way too much access to the markup, and has the potential to totally break VisualEditor, eg, if/when Parsoid replaces the legacy parser. We shouldn't be adding new hooks which Parsoid can't reasonably implement.

It seems like the desired purpose could just as easily be done by (eg) allowing the hook to add special class attributes to the images. Defining a semantics for "extra extension-generated attributes" is something that could be tightly scoped and well-defined. With some more thought, coming up with a way to tag specific images in an abstract way that would work for both Parsoid and the legacy parser is also something that could work, and the legacy parser can do this with an embedded comment or something while Parsoid does it with an annotation or some other mechanism. But "Arbitrarily modify the HTML generated for images" is something I don't think we can/should ever support.

Maybe let us move the larger discussion to T296895 and keep this task restricted to the production error.

dduvall triaged this task as Unbreak Now! priority.Thu, Jan 13, 8:25 PM
dduvall added a subscriber: dduvall.

Promotion of group1 is causing a fairly high increase in these errors. I'm going to make this a UBN blocker for now.

Mentioned in SAL (#wikimedia-operations) [2022-01-13T20:28:59Z] <dduvall> rolling back wmf.17 from group1 due to a large increase in "Parser state cleared while parsing" across commons and group1 wikipedias (T293958, T299149)

@tstarling can investigate this better but the PageImages patch may potentially need a revert to figure out the right strategy here since the metadata fetch is requiring a (recurisve) wikitext parse compared to before where the links-update hook invocation wasn't part of the main page parse. /cc @Krinkle

Yes the PageImages patch is implicated, but it was a dependency for the LinksUpdate refactor, so the LinksUpdate refactor would have to be reverted as well. I can fix the error in another way.

Change 753850 had a related patch set uploaded (by Tim Starling; author: Tim Starling):

[mediawiki/core@master] In WikitextContentHandler always use getFreshParser()

https://gerrit.wikimedia.org/r/753850

It would have been complicated to set up CommonsMetadata locally in order to reproduce the bug, so I modelled the bug with a patch to FormatMetadata:

diff --git a/includes/media/FormatMetadata.php b/includes/media/FormatMetadata.php
index c270542f86d..bee9ab5ae1f 100644
--- a/includes/media/FormatMetadata.php
+++ b/includes/media/FormatMetadata.php
@@ -1770,6 +1770,14 @@ class FormatMetadata extends ContextSource {
 			$cache->set( $cacheKey, $valueToCache, $maxCacheTime );
 		}
 
+		$extendedMetadata['test'] = MediaWikiServices::getInstance()->getParserOutputAccess()
+			->getParserOutput(
+				WikiPage::factory( Title::newMainPage() ),
+				new ParserOptions( RequestContext::getMain()->getUser() ),
+				null,
+				\MediaWiki\Page\ParserOutputAccess::OPT_NO_CHECK_CACHE
+			);
+
 		return $extendedMetadata;
 	}

This generates an exception similar to the one reported above, and https://gerrit.wikimedia.org/r/753850 fixes it.

Change 753828 had a related patch set uploaded (by Dduvall; author: Tim Starling):

[mediawiki/core@wmf/1.38.0-wmf.17] In WikitextContentHandler always use getFreshParser()

https://gerrit.wikimedia.org/r/753828

Change 753850 merged by jenkins-bot:

[mediawiki/core@master] In WikitextContentHandler always use getFreshParser()

https://gerrit.wikimedia.org/r/753850

Change 753828 merged by jenkins-bot:

[mediawiki/core@wmf/1.38.0-wmf.17] In WikitextContentHandler always use getFreshParser()

https://gerrit.wikimedia.org/r/753828

Mentioned in SAL (#wikimedia-operations) [2022-01-14T00:08:59Z] <dduvall@deploy1002> Synchronized php-1.38.0-wmf.17/includes/content/WikitextContentHandler.php: Backport: [[gerrit:753828|In WikitextContentHandler always use getFreshParser() (T299149)]] (duration: 01m 07s)

dduvall claimed this task.

This error has not been seen since syncing the fix and re-rolling group1/group2.