Page MenuHomePhabricator
Create Task
Maniphest T376325

enwikinews generates DuplicateParse messages since 1.43.0-wmf.25
Open, Needs TriagePublicPRODUCTION ERROR
Actions

Description

After deploying 1.43.0-wmf.25, I have noticed an elevated rate of messages sent to the DuplicateParse logging bucket. That seems to come from enwikinews

https://logstash.wikimedia.org/goto/d284c53d2546842347e54ee4eacd4559

DuplicateParse_enwikinews.png (270×665 px, 25 KB)

Example:

trace
#0 /srv/mediawiki/php-1.43.0-wmf.25/includes/content/ContentHandler.php(1700): MediaWiki\Parser\ParserObserver->notifyParse(Object(MediaWiki\Title\Title), 4204913, Object(ParserOptions), Object(MediaWiki\Content\WikitextContent), Object(MediaWiki\Parser\ParserOutput))
#1 /srv/mediawiki/php-1.43.0-wmf.25/includes/content/Renderer/ContentRenderer.php(79): MediaWiki\Content\ContentHandler->getParserOutput(Object(MediaWiki\Content\WikitextContent), Object(MediaWiki\Content\Renderer\ContentParseParams))
#2 /srv/mediawiki/php-1.43.0-wmf.25/includes/Revision/RenderedRevision.php(262): MediaWiki\Content\Renderer\ContentRenderer->getParserOutput(Object(MediaWiki\Content\WikitextContent), Object(MediaWiki\Title\Title), Object(MediaWiki\Revision\RevisionStoreRecord), Object(ParserOptions), Array)
#3 /srv/mediawiki/php-1.43.0-wmf.25/includes/Revision/RenderedRevision.php(234): MediaWiki\Revision\RenderedRevision->getSlotParserOutputUncached(Object(MediaWiki\Content\WikitextContent), Array)
#4 /srv/mediawiki/php-1.43.0-wmf.25/includes/Revision/RevisionRenderer.php(234): MediaWiki\Revision\RenderedRevision->getSlotParserOutput('main', Array)
#5 /srv/mediawiki/php-1.43.0-wmf.25/includes/Revision/RevisionRenderer.php(170): MediaWiki\Revision\RevisionRenderer->combineSlotOutput(Object(MediaWiki\Revision\RenderedRevision), Object(ParserOptions), Array)
#6 [internal function]: MediaWiki\Revision\RevisionRenderer->MediaWiki\Revision\{closure}(Object(MediaWiki\Revision\RenderedRevision), Array)
#7 /srv/mediawiki/php-1.43.0-wmf.25/includes/Revision/RenderedRevision.php(199): call_user_func(Object(Closure), Object(MediaWiki\Revision\RenderedRevision), Array)
#8 /srv/mediawiki/php-1.43.0-wmf.25/includes/page/ParserOutputAccess.php(452): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput()
#9 /srv/mediawiki/php-1.43.0-wmf.25/includes/page/ParserOutputAccess.php(364): MediaWiki\Page\ParserOutputAccess->renderRevision(Object(MediaWiki\Page\PageStoreRecord), Object(ParserOptions), Object(MediaWiki\Revision\RevisionStoreRecord), 0, NULL)
#10 /srv/mediawiki/php-1.43.0-wmf.25/extensions/DiscussionTools/includes/Hooks/HookUtils.php(151): MediaWiki\Page\ParserOutputAccess->getParserOutput(Object(MediaWiki\Page\PageStoreRecord), Object(ParserOptions), Object(MediaWiki\Revision\RevisionStoreRecord), 0)
#11 /srv/mediawiki/php-1.43.0-wmf.25/extensions/DiscussionTools/includes/Hooks/DataUpdatesHooks.php(48): MediaWiki\Extension\DiscussionTools\Hooks\HookUtils::parseRevisionParsoidHtml(Object(MediaWiki\Revision\RevisionStoreRecord), 'MediaWiki\\Exten...')
#12 /srv/mediawiki/php-1.43.0-wmf.25/includes/deferred/MWCallableUpdate.php(52): MediaWiki\Extension\DiscussionTools\Hooks\DataUpdatesHooks->MediaWiki\Extension\DiscussionTools\Hooks\{closure}('MediaWiki\\Exten...')
#13 /srv/mediawiki/php-1.43.0-wmf.25/includes/deferred/DeferredUpdates.php(460): MediaWiki\Deferred\MWCallableUpdate->doUpdate()
#14 /srv/mediawiki/php-1.43.0-wmf.25/includes/deferred/RefreshSecondaryDataUpdate.php(112): MediaWiki\Deferred\DeferredUpdates::attemptUpdate(Object(MediaWiki\Deferred\MWCallableUpdate))
#15 /srv/mediawiki/php-1.43.0-wmf.25/includes/deferred/DeferredUpdates.php(460): MediaWiki\Deferred\RefreshSecondaryDataUpdate->doUpdate()
#16 /srv/mediawiki/php-1.43.0-wmf.25/includes/deferred/DeferredUpdates.php(204): MediaWiki\Deferred\DeferredUpdates::attemptUpdate(Object(MediaWiki\Deferred\RefreshSecondaryDataUpdate))
#17 /srv/mediawiki/php-1.43.0-wmf.25/includes/deferred/DeferredUpdates.php(291): MediaWiki\Deferred\DeferredUpdates::run(Object(MediaWiki\Deferred\RefreshSecondaryDataUpdate))
#18 /srv/mediawiki/php-1.43.0-wmf.25/includes/deferred/DeferredUpdatesScope.php(240): MediaWiki\Deferred\DeferredUpdates::MediaWiki\Deferred\{closure}(Object(MediaWiki\Deferred\RefreshSecondaryDataUpdate), 1)
#19 /srv/mediawiki/php-1.43.0-wmf.25/includes/deferred/DeferredUpdatesScope.php(172): MediaWiki\Deferred\DeferredUpdatesScope->processStageQueue(1, 1, Object(Closure))
#20 /srv/mediawiki/php-1.43.0-wmf.25/includes/deferred/DeferredUpdates.php(310): MediaWiki\Deferred\DeferredUpdatesScope->processUpdates(1, Object(Closure))
#21 /srv/mediawiki/php-1.43.0-wmf.25/includes/MediaWikiEntryPoint.php(306): MediaWiki\Deferred\DeferredUpdates::doUpdates(1)
#22 /srv/mediawiki/php-1.43.0-wmf.25/includes/MediaWikiEntryPoint.php(189): MediaWiki\MediaWikiEntryPoint->commitMainTransaction()
#23 /srv/mediawiki/php-1.43.0-wmf.25/includes/MediaWikiEntryPoint.php(172): MediaWiki\MediaWikiEntryPoint->doPrepareForOutput()
#24 /srv/mediawiki/php-1.43.0-wmf.25/includes/MediaWiki.php(90): MediaWiki\MediaWikiEntryPoint->prepareForOutput()
#25 /srv/mediawiki/php-1.43.0-wmf.25/includes/api/ApiMain.php(970): MediaWiki::preOutputCommit(Object(MediaWiki\Context\DerivativeContext))
#26 /srv/mediawiki/php-1.43.0-wmf.25/includes/api/ApiMain.php(913): MediaWiki\Api\ApiMain->executeActionWithErrorHandling()
#27 /srv/mediawiki/php-1.43.0-wmf.25/includes/api/ApiEntryPoint.php(152): MediaWiki\Api\ApiMain->execute()
#28 /srv/mediawiki/php-1.43.0-wmf.25/includes/MediaWikiEntryPoint.php(200): MediaWiki\Api\ApiEntryPoint->execute()
#29 /srv/mediawiki/php-1.43.0-wmf.25/api.php(44): MediaWiki\MediaWikiEntryPoint->run()
#30 /srv/mediawiki/w/api.php(3): require('/srv/mediawiki/...')
#31 {main}
previous-trace
#0 /srv/mediawiki/php-1.43.0-wmf.25/includes/content/ContentHandler.php(1700): MediaWiki\Parser\ParserObserver->notifyParse(Object(MediaWiki\Title\Title), 4204913, Object(ParserOptions), Object(MediaWiki\Content\WikitextContent), Object(MediaWiki\Parser\ParserOutput))
#1 /srv/mediawiki/php-1.43.0-wmf.25/extensions/Linter/includes/LintUpdate.php(98): MediaWiki\Content\ContentHandler->getParserOutput(Object(MediaWiki\Content\WikitextContent), Object(MediaWiki\Content\Renderer\ContentParseParams))
#2 /srv/mediawiki/php-1.43.0-wmf.25/includes/deferred/DeferredUpdates.php(460): MediaWiki\Linter\LintUpdate->doUpdate()
#3 /srv/mediawiki/php-1.43.0-wmf.25/includes/deferred/RefreshSecondaryDataUpdate.php(112): MediaWiki\Deferred\DeferredUpdates::attemptUpdate(Object(MediaWiki\Linter\LintUpdate))
#4 /srv/mediawiki/php-1.43.0-wmf.25/includes/deferred/DeferredUpdates.php(460): MediaWiki\Deferred\RefreshSecondaryDataUpdate->doUpdate()
#5 /srv/mediawiki/php-1.43.0-wmf.25/includes/deferred/DeferredUpdates.php(204): MediaWiki\Deferred\DeferredUpdates::attemptUpdate(Object(MediaWiki\Deferred\RefreshSecondaryDataUpdate))
#6 /srv/mediawiki/php-1.43.0-wmf.25/includes/deferred/DeferredUpdates.php(291): MediaWiki\Deferred\DeferredUpdates::run(Object(MediaWiki\Deferred\RefreshSecondaryDataUpdate))
#7 /srv/mediawiki/php-1.43.0-wmf.25/includes/deferred/DeferredUpdatesScope.php(240): MediaWiki\Deferred\DeferredUpdates::MediaWiki\Deferred\{closure}(Object(MediaWiki\Deferred\RefreshSecondaryDataUpdate), 1)
#8 /srv/mediawiki/php-1.43.0-wmf.25/includes/deferred/DeferredUpdatesScope.php(172): MediaWiki\Deferred\DeferredUpdatesScope->processStageQueue(1, 1, Object(Closure))
#9 /srv/mediawiki/php-1.43.0-wmf.25/includes/deferred/DeferredUpdates.php(310): MediaWiki\Deferred\DeferredUpdatesScope->processUpdates(1, Object(Closure))
#10 /srv/mediawiki/php-1.43.0-wmf.25/includes/MediaWikiEntryPoint.php(306): MediaWiki\Deferred\DeferredUpdates::doUpdates(1)
#11 /srv/mediawiki/php-1.43.0-wmf.25/includes/MediaWikiEntryPoint.php(189): MediaWiki\MediaWikiEntryPoint->commitMainTransaction()
#12 /srv/mediawiki/php-1.43.0-wmf.25/includes/MediaWikiEntryPoint.php(172): MediaWiki\MediaWikiEntryPoint->doPrepareForOutput()
#13 /srv/mediawiki/php-1.43.0-wmf.25/includes/MediaWiki.php(90): MediaWiki\MediaWikiEntryPoint->prepareForOutput()
#14 /srv/mediawiki/php-1.43.0-wmf.25/includes/api/ApiMain.php(970): MediaWiki::preOutputCommit(Object(MediaWiki\Context\DerivativeContext))
#15 /srv/mediawiki/php-1.43.0-wmf.25/includes/api/ApiMain.php(913): MediaWiki\Api\ApiMain->executeActionWithErrorHandling()
#16 /srv/mediawiki/php-1.43.0-wmf.25/includes/api/ApiEntryPoint.php(152): MediaWiki\Api\ApiMain->execute()
#17 /srv/mediawiki/php-1.43.0-wmf.25/includes/MediaWikiEntryPoint.php(200): MediaWiki\Api\ApiEntryPoint->execute()
#18 /srv/mediawiki/php-1.43.0-wmf.25/api.php(44): MediaWiki\MediaWikiEntryPoint->run()
#19 /srv/mediawiki/w/api.php(3): require('/srv/mediawiki/...')
#20 {main}

Details

MediaWiki Version
1.43.0-wmf.25
SubjectRepoBranchLines +/-
Use ParserOutputAccess for LintUpdate jobmediawiki/extensions/Lintermaster+39 -57
Customize query in gerrit

Related Objects

Event Timeline

hashar created this task.Oct 2 2024, 6:41 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 2 2024, 6:41 PM
hashar added a comment.Oct 2 2024, 6:41 PM

Note they all seem to come from requests to /w/api.php?action=purge&format=xml which I guess is due to some bot mass purging pages for some reason?

hashar mentioned this in T375656: 1.43.0-wmf.25 deployment blockers.Oct 2 2024, 6:46 PM
cscott subscribed.EditedOct 2 2024, 7:25 PM

The log is triggered due to https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Linter/+/1073493 which made the LintUpdates use the same Content/Parser codepath as other passes, and thus effectively opted-into the ParserObserver check here.

That said, the Linter patch didn't *cause* the duplicate parse; they were present previously, the patch just made them get noticed.

This should be low priority, unless log volume were to become a problem. @daniel might want to have a look when he gets back next week.

cscott added a subscriber: daniel.Oct 2 2024, 7:33 PM
hashar added a comment.Oct 2 2024, 7:38 PM

I have filed it primarily because the DuplicateParse rate raised and that looked suspicious. The root cause was purges being asked from a bot and now that the requests have stopped, the rate is back to normal:

DuplicateParse_enwikinews_finished.png (205×436 px, 12 KB)

cscott added a comment.Oct 2 2024, 7:50 PM

Yeah, to be clear it seems to be a legit duplicate parse: both the Linter and DiscussionTools are asking for Parsoid parses of the page in RefreshSecondaryDataUpdate. In theory this should be handled by the localCache in ParserOutputAccess, but both clients are deliberately bypassing the cache in order "not to pollute it". DT via OPT_NO_UPDATE_CACHE when $updateParserCacheFor is false (but NO_UPDATE_CACHE doesn't disable the localCache), and Linter by using Content methods directly and not ParserOutputAccess.

Probably Linter should be using ParserOutputAccess with NO_UPDATE_CACHE, which (despite the name) will ensure that the local cache is used at least when DT later asks for the same parse.

gerritbot added a comment.Oct 2 2024, 8:55 PM

Change #1077477 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/Linter@master] Use ParserOutputAccess for LintUpdate job

https://gerrit.wikimedia.org/r/1077477

gerritbot added a project: Patch-For-Review.Oct 2 2024, 8:55 PM
Agusbou2015 subscribed.Oct 2 2024, 10:33 PM
gerritbot added a comment.Oct 8 2024, 9:36 PM

Change #1077477 merged by jenkins-bot:

[mediawiki/extensions/Linter@master] Use ParserOutputAccess for LintUpdate job

https://gerrit.wikimedia.org/r/1077477

ReleaseTaggerBot added a project: MW-1.43-notes (1.43.0-wmf.27; 2024-10-15).Oct 8 2024, 10:00 PM
Maintenance_bot removed a project: Patch-For-Review.Oct 8 2024, 10:30 PM
Krinkle added a project: Content-Transform-Team.Oct 10 2024, 3:53 PM
Krinkle moved this task from Untriaged to Oct 2024 on the Wikimedia-production-error board.
MSantos edited projects, added Content-Transform-Team-WIP, Essential-Work; removed Content-Transform-Team.Oct 17 2024, 2:30 PM
MSantos moved this task from Backlog to To Verify on the Content-Transform-Team-WIP board.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits