Sanitizer.php on line 495
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Assigned To

Authored By

	ssastry
	Nov 10 2022, 10:38 PM

Description

Error

mwversion: 1.40.0-wmf.8
reqId: 9bf15c68-fd05-46c9-9114-41efadd3b3e7
Find reqId in Logstash

normalized_message

[{reqId}] {exception_url}   TypeError: Argument 1 passed to Wikimedia\Parsoid\Core\Sanitizer::validateCodepoint() must be of the type int, float given, called in /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Core/Sanitizer.php on line 495

exception.trace

from /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Core/Sanitizer.php(508)
#0 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Core/Sanitizer.php(495): Wikimedia\Parsoid\Core\Sanitizer::validateCodepoint(double)
#1 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Core/Sanitizer.php(440): Wikimedia\Parsoid\Core\Sanitizer::hexCharReference(string)
#2 [internal function]: Wikimedia\Parsoid\Core\Sanitizer::normalizeCharReferencesCallback(array)
#3 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Core/Sanitizer.php(424): preg_replace_callback(string, array, string)
#4 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Utils/Utils.php(243): Wikimedia\Parsoid\Core\Sanitizer::normalizeCharReferences(string)
#5 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/Grammar.php(850): Wikimedia\Parsoid\Utils\Utils::decodeWtEntities(string)
#6 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/Grammar.php(4806): Wikimedia\Parsoid\Wt2Html\Grammar->a54(string)
#7 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/Grammar.php(3398): Wikimedia\Parsoid\Wt2Html\Grammar->parseraw_htmlentity(boolean)
#8 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/Grammar.php(6061): Wikimedia\Parsoid\Wt2Html\Grammar->parsehtmlentity(boolean)
#9 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/Grammar.php(13814): Wikimedia\Parsoid\Wt2Html\Grammar->parseurltext(boolean, integer, string, NULL)
#10 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/Grammar.php(13173): Wikimedia\Parsoid\Wt2Html\Grammar->parselink_text_parameterized(boolean, integer, string, NULL)
#11 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/Grammar.php(12121): Wikimedia\Parsoid\Wt2Html\Grammar->parselink_text(boolean, integer, string, NULL)
#12 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/Grammar.php(11247): Wikimedia\Parsoid\Wt2Html\Grammar->parsewikilink_content(boolean, integer, string, NULL)
#13 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/Grammar.php(8546): Wikimedia\Parsoid\Wt2Html\Grammar->parsewikilink_preproc(boolean, integer, string, NULL)
#14 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/Grammar.php(6293): Wikimedia\Parsoid\Wt2Html\Grammar->parsewikilink(boolean, integer, NULL, NULL)
#15 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/Grammar.php(3832): Wikimedia\Parsoid\Wt2Html\Grammar->parseinline_element(boolean, integer, NULL, NULL)
#16 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/Grammar.php(7046): Wikimedia\Parsoid\Wt2Html\Grammar->parseinlineline(boolean, integer, NULL, NULL)
#17 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/Grammar.php(4368): Wikimedia\Parsoid\Wt2Html\Grammar->parseparagraph(boolean, integer, NULL, NULL)
#18 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/Grammar.php(2895): Wikimedia\Parsoid\Wt2Html\Grammar->parseblock(boolean, integer, NULL, NULL)
#19 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/Grammar.php(2057): Wikimedia\Parsoid\Wt2Html\Grammar->parsetlb(boolean, NULL)
#20 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/Grammar.php(14957): Wikimedia\Parsoid\Wt2Html\Grammar->parsestart(boolean, NULL)
#21 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/PegTokenizer.php(169): Wikimedia\Parsoid\Wt2Html\Grammar->parse(string, array)
#22 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/PegTokenizer.php(88): Wikimedia\Parsoid\Wt2Html\PegTokenizer->tokenizeSync(string, array)
#23 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/ParserPipeline.php(140): Wikimedia\Parsoid\Wt2Html\PegTokenizer->process(string, array)
#24 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Utils/PipelineUtils.php(106): Wikimedia\Parsoid\Wt2Html\ParserPipeline->parse(string, array)
#25 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/TT/TemplateHandler.php(617): Wikimedia\Parsoid\Utils\PipelineUtils::processContentInPipeline(Wikimedia\Parsoid\Config\Env, Wikimedia\Parsoid\Wt2Html\PageConfigFrame, string, array)
#26 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/TT/TemplateHandler.php(1121): Wikimedia\Parsoid\Wt2Html\TT\TemplateHandler->processTemplateSource(Wikimedia\Parsoid\Tokens\SelfclosingTagTk, array, string)
#27 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/TT/TemplateHandler.php(1151): Wikimedia\Parsoid\Wt2Html\TT\TemplateHandler->expandTemplate(Wikimedia\Parsoid\Wt2Html\TT\TemplateEncapsulator)
#28 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/TT/TemplateHandler.php(1196): Wikimedia\Parsoid\Wt2Html\TT\TemplateHandler->onTemplate(Wikimedia\Parsoid\Tokens\SelfclosingTagTk)
#29 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/TT/TokenHandler.php(150): Wikimedia\Parsoid\Wt2Html\TT\TemplateHandler->onTag(Wikimedia\Parsoid\Tokens\SelfclosingTagTk)
#30 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/TokenTransformManager.php(132): Wikimedia\Parsoid\Wt2Html\TT\TokenHandler->process(array)
#31 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/TokenTransformManager.php(195): Wikimedia\Parsoid\Wt2Html\TokenTransformManager->processChunk(array)
#32 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/TokenTransformManager.php(193): Wikimedia\Parsoid\Wt2Html\TokenTransformManager->processChunkily(string, array)
#33 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/TreeBuilder/TreeBuilderStage.php(497): Wikimedia\Parsoid\Wt2Html\TokenTransformManager->processChunkily(string, array)
#34 [internal function]: Wikimedia\Parsoid\Wt2Html\TreeBuilder\TreeBuilderStage->processChunkily(string, array)
#35 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/DOMPostProcessor.php(905): Generator->current()
#36 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/ParserPipeline.php(180): Wikimedia\Parsoid\Wt2Html\DOMPostProcessor->processChunkily(string, array)
#37 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wt2Html/ParserPipelineFactory.php(299): Wikimedia\Parsoid\Wt2Html\ParserPipeline->parseChunkily(string, array)
#38 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Wikitext/ContentModelHandler.php(123): Wikimedia\Parsoid\Wt2Html\ParserPipelineFactory->parse(string)
#39 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Parsoid.php(172): Wikimedia\Parsoid\Wikitext\ContentModelHandler->toDOM(Wikimedia\Parsoid\Ext\ParsoidExtensionAPI)
#40 /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Parsoid.php(210): Wikimedia\Parsoid\Parsoid->parseWikitext(MediaWiki\Parser\Parsoid\Config\PageConfig, ParserOutput, array)
#41 /srv/mediawiki/php-1.40.0-wmf.8/includes/Rest/Handler/ParsoidHandler.php(781): Wikimedia\Parsoid\Parsoid->wikitext2html(MediaWiki\Parser\Parsoid\Config\PageConfig, array, NULL, ParserOutput)
#42 /srv/mediawiki/php-1.40.0-wmf.8/includes/Rest/Handler/TransformHandler.php(106): MediaWiki\Rest\Handler\ParsoidHandler->wt2html(MediaWiki\Parser\Parsoid\Config\PageConfig, array, string)
#43 /srv/mediawiki/php-1.40.0-wmf.8/includes/Rest/Router.php(487): MediaWiki\Rest\Handler\TransformHandler->execute()
#44 /srv/mediawiki/php-1.40.0-wmf.8/includes/Rest/Router.php(406): MediaWiki\Rest\Router->executeHandler(MWParsoid\Rest\Handler\TransformHandler)
#45 /srv/mediawiki/php-1.40.0-wmf.8/includes/Rest/EntryPoint.php(170): MediaWiki\Rest\Router->execute(MediaWiki\Rest\RequestFromGlobals)
#46 /srv/mediawiki/php-1.40.0-wmf.8/includes/Rest/EntryPoint.php(135): MediaWiki\Rest\EntryPoint->execute()
#47 /srv/mediawiki/php-1.40.0-wmf.8/rest.php(31): MediaWiki\Rest\EntryPoint::main()
#48 /srv/mediawiki/w/rest.php(3): require(string)
#49 {main}

Impact

Notes

Details

Request URL: https://fr.wikipedia.org/w/rest.php/fr.wikipedia.org/v3/transform/wikitext/to/pagebundle/Drapeaux_transgenres

Subject	Repo	Branch	Lines +/-
Bump wikimedia/parsoid to 0.17.0-a9	mediawiki/vendor	master	+766 -278
Protect against long match length in CHAR_REFS_REGEX	mediawiki/services/parsoid	master	+167 -1
Protect against long match length in CHAR_REFS_REGEX	mediawiki/core	master	+32 -2

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		None	T336146 [EPIC] FY 23-24 unplanned and maintenance for Content Transform Team
		Resolved	PRODUCTION ERROR	cscott	T322892 TypeError: Argument 1 passed to Wikimedia\Parsoid\Core\Sanitizer::validateCodepoint() must be of the type int, float given, called in /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Core/Sanitizer.php on line 495

Event Timeline

ssastry created this task.Nov 10 2022, 10:38 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 10 2022, 10:38 PM

• JMcLeod_WMF moved this task from Needs Triage to Bugs & Crashers on the Parsoid board.Nov 17 2022, 3:28 PM

This looks like it might be a recent regression? Pretty clearly a bug in Parsoid, given that stack trace, though it wouldn't surprise me if there was some evil wikitext trigger.

so... PHP's `hexdec' function can return a double?
https://www.php.net/manual/en/function.hexdec.php says:

The function can convert numbers that are too large to fit into the platforms int type, larger values are returned as float in that case.

Wow. Ok, should be a straightforward fix, I guess? Probably core has this issue too, since this code is shared between core and PHP I think.

Change 858353 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/services/parsoid@master] Limit match length in CHAR_REFS_REGEX

https://gerrit.wikimedia.org/r/858353

gerritbot added a project: Patch-For-Review.Nov 17 2022, 3:48 PM

Change 858354 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/core@master] Limit match length in CHAR_REFS_REGEX

https://gerrit.wikimedia.org/r/858354

Krinkle moved this task from Untriaged to Nov 2022 on the Wikimedia-production-error board.Nov 17 2022, 4:47 PM

Change 858354 merged by jenkins-bot:

[mediawiki/core@master] Protect against long match length in CHAR_REFS_REGEX

https://gerrit.wikimedia.org/r/858354

ReleaseTaggerBot added a project: MW-1.40-notes (1.40.0-wmf.12; 2022-11-28).Nov 22 2022, 10:01 PM

Change 858353 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Protect against long match length in CHAR_REFS_REGEX

https://gerrit.wikimedia.org/r/858353

Maintenance_bot removed a project: Patch-For-Review.Dec 9 2022, 10:30 PM

Change 867276 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.17.0-a9

https://gerrit.wikimedia.org/r/867276

gerritbot added a project: Patch-For-Review.Dec 12 2022, 9:48 PM

Change 867276 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.17.0-a9

https://gerrit.wikimedia.org/r/867276

Maintenance_bot removed a project: Patch-For-Review.Dec 13 2022, 12:30 AM

Looks like this may have been resolved already. Moving to the to-verify column of the ctt wip board.

ssastry assigned this task to cscott.Jun 2 2023, 1:09 PM

ssastry moved this task from Backlog to To Verify on the Content-Transform-Team-WIP board.

MSantos added a parent task: T336146: [EPIC] FY 23-24 unplanned and maintenance for Content Transform Team.Jun 19 2023, 11:20 AM

MSantos added a project: Essential-Work.

Looks like this hasn't happened again since the patches were merged.

TypeError: Argument 1 passed to Wikimedia\Parsoid\Core\Sanitizer::validateCodepoint() must be of the type int, float given, called in /srv/mediawiki/php-1.40.0-wmf.8/vendor/wikimedia/parsoid/src/Core/Sanitizer.php on line 495Closed, ResolvedPublicPRODUCTION ERRORActions