Page MenuHomePhabricator
Create Task
Maniphest T299149

MWException: Parser state cleared while parsing. Did you call Parser::parse recursively?
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

Error
normalized_message
[{reqId}] {exception_url}   MWException: Parser state cleared while parsing. Did you call Parser::parse recursively? Lock is held by: #0 /srv/mediawiki/php-1.38.0-wmf.17/includes/parser/Parser.php(666): Parser->lock()
exception.trace
from /srv/mediawiki/php-1.38.0-wmf.17/includes/parser/Parser.php(6320)
#0 /srv/mediawiki/php-1.38.0-wmf.17/includes/parser/Parser.php(666): Parser->lock()
#1 /srv/mediawiki/php-1.38.0-wmf.17/includes/content/WikitextContentHandler.php(294): Parser->parse(string, Title, ParserOptions, boolean, boolean, integer)
#2 /srv/mediawiki/php-1.38.0-wmf.17/includes/content/ContentHandler.php(1723): WikitextContentHandler->fillParserOutput(WikitextContent, MediaWiki\Content\Renderer\ContentParseParams, ParserOutput)
#3 /srv/mediawiki/php-1.38.0-wmf.17/includes/content/Renderer/ContentRenderer.php(47): ContentHandler->getParserOutput(WikitextContent, MediaWiki\Content\Renderer\ContentParseParams)
#4 /srv/mediawiki/php-1.38.0-wmf.17/includes/Revision/RenderedRevision.php(271): MediaWiki\Content\Renderer\ContentRenderer->getParserOutput(WikitextContent, Title, integer, ParserOptions, boolean)
#5 /srv/mediawiki/php-1.38.0-wmf.17/includes/Revision/RenderedRevision.php(238): MediaWiki\Revision\RenderedRevision->getSlotParserOutputUncached(WikitextContent, boolean)
#6 /srv/mediawiki/php-1.38.0-wmf.17/includes/Revision/RevisionRenderer.php(221): MediaWiki\Revision\RenderedRevision->getSlotParserOutput(string, array)
#7 /srv/mediawiki/php-1.38.0-wmf.17/includes/Revision/RevisionRenderer.php(158): MediaWiki\Revision\RevisionRenderer->combineSlotOutput(MediaWiki\Revision\RenderedRevision, array)
#8 [internal function]: MediaWiki\Revision\RevisionRenderer->MediaWiki\Revision\{closure}(MediaWiki\Revision\RenderedRevision, array)
#9 /srv/mediawiki/php-1.38.0-wmf.17/includes/Revision/RenderedRevision.php(200): call_user_func(Closure, MediaWiki\Revision\RenderedRevision, array)
#10 /srv/mediawiki/php-1.38.0-wmf.17/includes/poolcounter/PoolWorkArticleView.php(137): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput()
#11 /srv/mediawiki/php-1.38.0-wmf.17/includes/poolcounter/PoolCounterWork.php(152): PoolWorkArticleView->doWork()
#12 /srv/mediawiki/php-1.38.0-wmf.17/includes/page/ParserOutputAccess.php(279): PoolCounterWork->execute()
#13 /srv/mediawiki/php-1.38.0-wmf.17/includes/filerepo/file/LocalFile.php(2554): MediaWiki\Page\ParserOutputAccess->getParserOutput(MediaWiki\Page\PageStoreRecord, ParserOptions)
#14 /srv/mediawiki/php-1.38.0-wmf.17/extensions/CommonsMetadata/src/DataCollector.php(258): LocalFile->getDescriptionText(LanguageEn)
#15 /srv/mediawiki/php-1.38.0-wmf.17/extensions/CommonsMetadata/src/DataCollector.php(96): CommonsMetadata\DataCollector->getDescriptionText(LocalFile, LanguageEn)
#16 /srv/mediawiki/php-1.38.0-wmf.17/extensions/CommonsMetadata/src/HookHandler.php(75): CommonsMetadata\DataCollector->collect(array, LocalFile)
#17 /srv/mediawiki/php-1.38.0-wmf.17/includes/HookContainer/HookContainer.php(338): CommonsMetadata\HookHandler::onGetExtendedMetadata(array, LocalFile, DerivativeContext, boolean, integer)
#18 /srv/mediawiki/php-1.38.0-wmf.17/includes/HookContainer/HookContainer.php(137): MediaWiki\HookContainer\HookContainer->callLegacyHook(string, array, array, array)
#19 /srv/mediawiki/php-1.38.0-wmf.17/includes/HookContainer/HookRunner.php(1810): MediaWiki\HookContainer\HookContainer->run(string, array)
#20 /srv/mediawiki/php-1.38.0-wmf.17/includes/media/FormatMetadata.php(1842): MediaWiki\HookContainer\HookRunner->onGetExtendedMetadata(array, LocalFile, DerivativeContext, boolean, integer)
#21 /srv/mediawiki/php-1.38.0-wmf.17/includes/media/FormatMetadata.php(1759): FormatMetadata->getExtendedMetadataFromHook(LocalFile, array, integer)
#22 /srv/mediawiki/php-1.38.0-wmf.17/extensions/PageImages/includes/Hooks/ParserFileProcessingHookHandlers.php(369): FormatMetadata->fetchExtendedMetadata(LocalFile)
#23 /srv/mediawiki/php-1.38.0-wmf.17/extensions/PageImages/includes/Hooks/ParserFileProcessingHookHandlers.php(350): PageImages\Hooks\ParserFileProcessingHookHandlers->fetchFileMetadata(LocalFile)
#24 /srv/mediawiki/php-1.38.0-wmf.17/extensions/PageImages/includes/Hooks/ParserFileProcessingHookHandlers.php(202): PageImages\Hooks\ParserFileProcessingHookHandlers->isImageFree(string)
#25 /srv/mediawiki/php-1.38.0-wmf.17/extensions/PageImages/includes/Hooks/ParserFileProcessingHookHandlers.php(148): PageImages\Hooks\ParserFileProcessingHookHandlers->findBestImages(array)
#26 /srv/mediawiki/php-1.38.0-wmf.17/extensions/PageImages/includes/Hooks/ParserFileProcessingHookHandlers.php(66): PageImages\Hooks\ParserFileProcessingHookHandlers->doParserAfterTidy(Parser, string)
#27 /srv/mediawiki/php-1.38.0-wmf.17/includes/HookContainer/HookContainer.php(338): PageImages\Hooks\ParserFileProcessingHookHandlers::onParserAfterTidy(Parser, string)
#28 /srv/mediawiki/php-1.38.0-wmf.17/includes/HookContainer/HookContainer.php(137): MediaWiki\HookContainer\HookContainer->callLegacyHook(string, array, array, array)
#29 /srv/mediawiki/php-1.38.0-wmf.17/includes/HookContainer/HookRunner.php(2827): MediaWiki\HookContainer\HookContainer->run(string, array)
#30 /srv/mediawiki/php-1.38.0-wmf.17/includes/parser/Parser.php(1701): MediaWiki\HookContainer\HookRunner->onParserAfterTidy(Parser, string)
#31 /srv/mediawiki/php-1.38.0-wmf.17/includes/parser/Parser.php(693): Parser->internalParseHalfParsed(string, boolean, boolean)
#32 /srv/mediawiki/php-1.38.0-wmf.17/includes/content/WikitextContentHandler.php(294): Parser->parse(string, Title, ParserOptions, boolean, boolean, integer)
#33 /srv/mediawiki/php-1.38.0-wmf.17/includes/content/ContentHandler.php(1723): WikitextContentHandler->fillParserOutput(WikitextContent, MediaWiki\Content\Renderer\ContentParseParams, ParserOutput)
#34 /srv/mediawiki/php-1.38.0-wmf.17/includes/content/Renderer/ContentRenderer.php(47): ContentHandler->getParserOutput(WikitextContent, MediaWiki\Content\Renderer\ContentParseParams)
#35 /srv/mediawiki/php-1.38.0-wmf.17/includes/Revision/RenderedRevision.php(271): MediaWiki\Content\Renderer\ContentRenderer->getParserOutput(WikitextContent, Title, integer, ParserOptions, boolean)
#36 /srv/mediawiki/php-1.38.0-wmf.17/includes/Revision/RenderedRevision.php(238): MediaWiki\Revision\RenderedRevision->getSlotParserOutputUncached(WikitextContent, boolean)
#37 /srv/mediawiki/php-1.38.0-wmf.17/includes/Revision/RevisionRenderer.php(221): MediaWiki\Revision\RenderedRevision->getSlotParserOutput(string, array)
#38 /srv/mediawiki/php-1.38.0-wmf.17/includes/Revision/RevisionRenderer.php(158): MediaWiki\Revision\RevisionRenderer->combineSlotOutput(MediaWiki\Revision\RenderedRevision, array)
#39 [internal function]: MediaWiki\Revision\RevisionRenderer->MediaWiki\Revision\{closure}(MediaWiki\Revision\RenderedRevision, array)
#40 /srv/mediawiki/php-1.38.0-wmf.17/includes/Revision/RenderedRevision.php(200): call_user_func(Closure, MediaWiki\Revision\RenderedRevision, array)
#41 /srv/mediawiki/php-1.38.0-wmf.17/includes/poolcounter/PoolWorkArticleView.php(137): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput()
#42 /srv/mediawiki/php-1.38.0-wmf.17/includes/poolcounter/PoolCounterWork.php(162): PoolWorkArticleView->doWork()
#43 /srv/mediawiki/php-1.38.0-wmf.17/includes/page/ParserOutputAccess.php(279): PoolCounterWork->execute()
#44 /srv/mediawiki/php-1.38.0-wmf.17/includes/page/Article.php(705): MediaWiki\Page\ParserOutputAccess->getParserOutput(WikiPage, ParserOptions, MediaWiki\Revision\RevisionStoreRecord, integer)
#45 /srv/mediawiki/php-1.38.0-wmf.17/includes/page/Article.php(517): Article->generateContentOutput(User, ParserOptions, integer, OutputPage, array)
#46 /srv/mediawiki/php-1.38.0-wmf.17/includes/actions/ViewAction.php(80): Article->view()
#47 /srv/mediawiki/php-1.38.0-wmf.17/includes/MediaWiki.php(543): ViewAction->show()
#48 /srv/mediawiki/php-1.38.0-wmf.17/includes/MediaWiki.php(320): MediaWiki->performAction(Article, Title)
#49 /srv/mediawiki/php-1.38.0-wmf.17/includes/MediaWiki.php(903): MediaWiki->performRequest()
#50 /srv/mediawiki/php-1.38.0-wmf.17/includes/MediaWiki.php(563): MediaWiki->main()
#51 /srv/mediawiki/php-1.38.0-wmf.17/index.php(53): MediaWiki->run()
#52 /srv/mediawiki/php-1.38.0-wmf.17/index.php(46): wfIndexMain()
#53 /srv/mediawiki/w/index.php(3): require(string)
#54 {main}
Notes
  • Seeing a lot of these from group0 and group1 wikis on 1.38.0-wmf.17
  • Seems to happen more frequently on commonswiki

Details

Request URL
https://office.wikimedia.org/w/index.php?title=*&direction=*&oldid=*
SubjectRepoBranchLines +/-
In WikitextContentHandler always use getFreshParser()mediawiki/coremaster+1 -1
In WikitextContentHandler always use getFreshParser()mediawiki/corewmf/1.38.0-wmf.17+1 -1
Customize query in gerrit

Related Objects
Search...

Event Timeline

thcipriani created this task.Jan 13 2022, 6:12 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 13 2022, 6:12 PM
thcipriani moved this task from Untriaged to Jan 2022 on the Wikimedia-production-error board.Jan 13 2022, 6:13 PM
Zabe updated the task description. (Show Details)Jan 13 2022, 6:13 PM
ssastry added subscribers: tstarling, ssastry.Jan 13 2022, 6:40 PM

Probably from https://gerrit.wikimedia.org/r/c/mediawiki/extensions/PageImages/+/743062/ and https://gerrit.wikimedia.org/r/c/mediawiki/core/+/743061

ssastry added a subscriber: Krinkle.EditedJan 13 2022, 6:58 PM

@tstarling can investigate this better but the PageImages patch may potentially need a revert to figure out the right strategy here since the metadata fetch is requiring a (recurisve) wikitext parse compared to before where the links-update hook invocation wasn't part of the main page parse. /cc @Krinkle

ssastry edited projects, added PageImages; removed Parsoid.Jan 13 2022, 7:10 PM
cscott subscribed.Jan 13 2022, 7:23 PM

I'm not a big fan of a hook named onParserModifyImageHTML in any case. This is way too much access to the markup, and has the potential to totally break VisualEditor, eg, if/when Parsoid replaces the legacy parser. We shouldn't be adding new hooks which Parsoid can't reasonably implement.

It seems like the desired purpose could just as easily be done by (eg) allowing the hook to add special class attributes to the images. Defining a semantics for "extra extension-generated attributes" is something that could be tightly scoped and well-defined. With some more thought, coming up with a way to tag specific images in an abstract way that would work for both Parsoid and the legacy parser is also something that could work, and the legacy parser can do this with an embedded comment or something while Parsoid does it with an annotation or some other mechanism. But "Arbitrarily modify the HTML generated for images" is something I don't think we can/should ever support.

ssastry added a comment.Jan 13 2022, 7:33 PM

Maybe let us move the larger discussion to T296895 and keep this task restricted to the production error.

cscott mentioned this in T296895: LinksUpdate hook review.Jan 13 2022, 8:14 PM
• dduvall triaged this task as Unbreak Now! priority.Jan 13 2022, 8:25 PM
• dduvall subscribed.

Promotion of group1 is causing a fairly high increase in these errors. I'm going to make this a UBN blocker for now.

• dduvall added a parent task: T293958: 1.38.0-wmf.17 deployment blockers.Jan 13 2022, 8:25 PM
Stashbot added a comment.Jan 13 2022, 8:28 PM

Mentioned in SAL (#wikimedia-operations) [2022-01-13T20:28:59Z] <dduvall> rolling back wmf.17 from group1 due to a large increase in "Parser state cleared while parsing" across commons and group1 wikipedias (T293958, T299149)

Stashbot mentioned this in T293958: 1.38.0-wmf.17 deployment blockers.Jan 13 2022, 8:29 PM
RhinosF1 subscribed.Jan 13 2022, 8:33 PM
Zabe subscribed.Jan 13 2022, 8:35 PM
brennen subscribed.Jan 13 2022, 9:48 PM
• dduvall added a comment.Jan 13 2022, 9:59 PM

Here's the logstash snapshot of the error spike that followed group1 deployment.

Agusbou2015 subscribed.Jan 13 2022, 10:05 PM
tstarling added a comment.Jan 13 2022, 10:13 PM
In T299149#7620431, @ssastry wrote:

@tstarling can investigate this better but the PageImages patch may potentially need a revert to figure out the right strategy here since the metadata fetch is requiring a (recurisve) wikitext parse compared to before where the links-update hook invocation wasn't part of the main page parse. /cc @Krinkle

Yes the PageImages patch is implicated, but it was a dependency for the LinksUpdate refactor, so the LinksUpdate refactor would have to be reverted as well. I can fix the error in another way.

gerritbot added a comment.Jan 13 2022, 10:42 PM

Change 753850 had a related patch set uploaded (by Tim Starling; author: Tim Starling):

[mediawiki/core@master] In WikitextContentHandler always use getFreshParser()

https://gerrit.wikimedia.org/r/753850

gerritbot added a project: Patch-For-Review.Jan 13 2022, 10:42 PM
tstarling added a comment.Jan 13 2022, 10:45 PM

It would have been complicated to set up CommonsMetadata locally in order to reproduce the bug, so I modelled the bug with a patch to FormatMetadata:

diff --git a/includes/media/FormatMetadata.php b/includes/media/FormatMetadata.php
index c270542f86d..bee9ab5ae1f 100644
--- a/includes/media/FormatMetadata.php
+++ b/includes/media/FormatMetadata.php
@@ -1770,6 +1770,14 @@ class FormatMetadata extends ContextSource {
 			$cache->set( $cacheKey, $valueToCache, $maxCacheTime );
 		}
 
+		$extendedMetadata['test'] = MediaWikiServices::getInstance()->getParserOutputAccess()
+			->getParserOutput(
+				WikiPage::factory( Title::newMainPage() ),
+				new ParserOptions( RequestContext::getMain()->getUser() ),
+				null,
+				\MediaWiki\Page\ParserOutputAccess::OPT_NO_CHECK_CACHE
+			);
+
 		return $extendedMetadata;
 	}

This generates an exception similar to the one reported above, and https://gerrit.wikimedia.org/r/753850 fixes it.

gerritbot added a comment.Jan 13 2022, 11:08 PM

Change 753828 had a related patch set uploaded (by Dduvall; author: Tim Starling):

[mediawiki/core@wmf/1.38.0-wmf.17] In WikitextContentHandler always use getFreshParser()

https://gerrit.wikimedia.org/r/753828

gerritbot added a comment.Jan 13 2022, 11:52 PM

Change 753850 merged by jenkins-bot:

[mediawiki/core@master] In WikitextContentHandler always use getFreshParser()

https://gerrit.wikimedia.org/r/753850

ReleaseTaggerBot added a project: MW-1.38-notes (1.38.0-wmf.18; 2022-01-17).Jan 14 2022, 12:00 AM
gerritbot added a comment.Jan 14 2022, 12:02 AM

Change 753828 merged by jenkins-bot:

[mediawiki/core@wmf/1.38.0-wmf.17] In WikitextContentHandler always use getFreshParser()

https://gerrit.wikimedia.org/r/753828

Stashbot added a comment.Jan 14 2022, 12:08 AM

Mentioned in SAL (#wikimedia-operations) [2022-01-14T00:08:59Z] <dduvall@deploy1002> Synchronized php-1.38.0-wmf.17/includes/content/WikitextContentHandler.php: Backport: [[gerrit:753828|In WikitextContentHandler always use getFreshParser() (T299149)]] (duration: 01m 07s)

Maintenance_bot removed a project: Patch-For-Review.Jan 14 2022, 12:10 AM
ReleaseTaggerBot edited projects, added MW-1.38-notes (1.38.0-wmf.17; 2022-01-10); removed MW-1.38-notes (1.38.0-wmf.18; 2022-01-17).Jan 14 2022, 1:00 AM
• dduvall closed this task as Resolved.Jan 14 2022, 1:05 AM
• dduvall claimed this task.

This error has not been seen since syncing the fix and re-rolling group1/group2.

cscott added a comment.Jan 18 2022, 3:39 PM

See also T250750: Parser::getFreshParser should always return a fresh parser, T250445: Deprecate and remove Parser::clearState(), T250447: Deprecate and remove Parser::resetOutput().

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL