Page MenuHomePhabricator
Create Task
Maniphest T306186

TypeError: Argument 1 passed to DOMNode::appendChild() must be an instance of DOMNode, null given
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

Error
normalized_message
[{reqId}] {exception_url}   TypeError: Argument 1 passed to DOMNode::appendChild() must be an instance of DOMNode, null given
exception.trace
from /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/PP/Processors/AnnotationDOMRangeBuilder.php(111)
#0 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/PP/Processors/AnnotationDOMRangeBuilder.php(111): DOMNode->appendChild(NULL)
#1 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/PP/Processors/AnnotationDOMRangeBuilder.php(78): Wikimedia\Parsoid\Wt2Html\PP\Processors\AnnotationDOMRangeBuilder->makeUneditable(Wikimedia\Parsoid\Wt2Html\PP\Processors\DOMRangeInfo, integer, integer)
#2 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/PP/Processors/AnnotationDOMRangeBuilder.php(313): Wikimedia\Parsoid\Wt2Html\PP\Processors\AnnotationDOMRangeBuilder->wrapAnnotationsInTree(Wikimedia\Parsoid\DOM\Element)
#3 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/PP/Processors/WrapAnnotations.php(22): Wikimedia\Parsoid\Wt2Html\PP\Processors\AnnotationDOMRangeBuilder->execute(Wikimedia\Parsoid\DOM\Element)
#4 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/DOMPostProcessor.php(158): Wikimedia\Parsoid\Wt2Html\PP\Processors\WrapAnnotations->run(Wikimedia\Parsoid\Config\Env, Wikimedia\Parsoid\DOM\Element, array, boolean)
#5 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/DOMPostProcessor.php(848): Wikimedia\Parsoid\Wt2Html\DOMPostProcessor->Wikimedia\Parsoid\Wt2Html\{closure}(Wikimedia\Parsoid\DOM\Element, array, boolean)
#6 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/DOMPostProcessor.php(889): Wikimedia\Parsoid\Wt2Html\DOMPostProcessor->doPostProcess(Wikimedia\Parsoid\DOM\Element)
#7 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/DOMPostProcessor.php(907): Wikimedia\Parsoid\Wt2Html\DOMPostProcessor->process(Wikimedia\Parsoid\DOM\Element)
#8 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/ParserPipeline.php(180): Wikimedia\Parsoid\Wt2Html\DOMPostProcessor->processChunkily(string, array)
#9 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/ParserPipelineFactory.php(308): Wikimedia\Parsoid\Wt2Html\ParserPipeline->parseChunkily(string, array)
#10 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wikitext/ContentModelHandler.php(123): Wikimedia\Parsoid\Wt2Html\ParserPipelineFactory->parse(string)
#11 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Parsoid.php(172): Wikimedia\Parsoid\Wikitext\ContentModelHandler->toDOM(Wikimedia\Parsoid\Ext\ParsoidExtensionAPI)
#12 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Parsoid.php(210): Wikimedia\Parsoid\Parsoid->parseWikitext(MediaWiki\Parser\Parsoid\Config\PageConfig, ParserOutput, array)
#13 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/extension/src/Rest/Handler/ParsoidHandler.php(587): Wikimedia\Parsoid\Parsoid->wikitext2html(MediaWiki\Parser\Parsoid\Config\PageConfig, array, NULL, ParserOutput)
#14 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/extension/src/Rest/Handler/PageHandler.php(88): MWParsoid\Rest\Handler\ParsoidHandler->wt2html(MediaWiki\Parser\Parsoid\Config\PageConfig, array)
#15 /srv/mediawiki/php-1.39.0-wmf.7/includes/Rest/Router.php(414): MWParsoid\Rest\Handler\PageHandler->execute()
#16 /srv/mediawiki/php-1.39.0-wmf.7/includes/Rest/Router.php(338): MediaWiki\Rest\Router->executeHandler(MWParsoid\Rest\Handler\PageHandler)
#17 /srv/mediawiki/php-1.39.0-wmf.7/includes/Rest/EntryPoint.php(166): MediaWiki\Rest\Router->execute(MediaWiki\Rest\RequestFromGlobals)
#18 /srv/mediawiki/php-1.39.0-wmf.7/includes/Rest/EntryPoint.php(131): MediaWiki\Rest\EntryPoint->execute()
#19 /srv/mediawiki/php-1.39.0-wmf.7/rest.php(31): MediaWiki\Rest\EntryPoint::main()
#20 /srv/mediawiki/w/rest.php(3): require(string)
#21 {main}
Notes

76 hits for this error today. There are no other hits in the last 30 days.

Details

Request URL
https://meta.wikimedia.org/w/rest.php/meta.wikimedia.org/v3/page/pagebundle/United_States_non-acceptance_of_the_rule_of_the_shorter_term/23140651
ProjectBranchLines +/-Subject
mediawiki/vendormaster+362 -3 KBump parsoid to 0.16.0-a7
mediawiki/services/parsoidmaster+11 -2Workaround for DOMNode::apendChild() production crasher
mediawiki/services/parsoidmaster+78 -79Use DOMRangeBuilder "top level range" algorithm for nested ranges
Customize query in gerrit

Related Objects

Event Timeline

dancy created this task.Apr 14 2022, 2:10 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 14 2022, 2:10 PM
dancy triaged this task as Unbreak Now! priority.Apr 14 2022, 2:32 PM
dancy added a parent task: T305213: 1.39.0-wmf.7 deployment blockers.

I've made this task a train blocker for 1.39.0-wmf.7 since it appears to be a new problem. Please fix or advise.

dancy updated the task description. (Show Details)Apr 14 2022, 2:35 PM
ihurbain added subscribers: cscott, ssastry, Arlolra, ihurbain.Apr 14 2022, 3:08 PM

It's an issue that seems limited to a small subset of translatable pages - and it may only trigger on wikis where the Translate extension is running, which I believe would all be group0/group1. Also note that VE is not officially supported yet for editing these pages (although this was supposed to be the version that would make it so, basically :( ) - so this reduces the impact of Parsoid crashes.
From my point of view, this is not a train blocker due to limited impact, but I don't feel comfortable making that call definitely. @ssastry, @cscott, @Arlolra - opinions?

Krinkle moved this task from Untriaged to Apr 2022 on the Wikimedia-production-error board.Apr 14 2022, 3:11 PM
Krinkle added a subscriber: Krinkle.Apr 14 2022, 3:22 PM

The same requests also first log this non-fatal error, which might help narrow it down:

PHP Notice: Trying to get property 'nextSibling' of non-object

from /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/PP/Processors/AnnotationDOMRangeBuilder.php(110)
#0 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/PP/Processors/AnnotationDOMRangeBuilder.php(110): MWExceptionHandler::handleError(integer, string, string, integer, array)
#1 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/PP/Processors/AnnotationDOMRangeBuilder.php(78): Wikimedia\Parsoid\Wt2Html\PP\Processors\AnnotationDOMRangeBuilder->makeUneditable(Wikimedia\Parsoid\Wt2Html\PP\Processors\DOMRangeInfo, integer, integer)
#2 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/PP/Processors/AnnotationDOMRangeBuilder.php(313): Wikimedia\Parsoid\Wt2Html\PP\Processors\AnnotationDOMRangeBuilder->wrapAnnotationsInTree(Wikimedia\Parsoid\DOM\Element)
#3 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/PP/Processors/WrapAnnotations.php(22): Wikimedia\Parsoid\Wt2Html\PP\Processors\AnnotationDOMRangeBuilder->execute(Wikimedia\Parsoid\DOM\Element)
#4 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/DOMPostProcessor.php(158): Wikimedia\Parsoid\Wt2Html\PP\Processors\WrapAnnotations->run(Wikimedia\Parsoid\Config\Env, Wikimedia\Parsoid\DOM\Element, array, boolean)
#5 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/DOMPostProcessor.php(848): Wikimedia\Parsoid\Wt2Html\DOMPostProcessor->Wikimedia\Parsoid\Wt2Html\{closure}(Wikimedia\Parsoid\DOM\Element, array, boolean)
#6 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/DOMPostProcessor.php(889): Wikimedia\Parsoid\Wt2Html\DOMPostProcessor->doPostProcess(Wikimedia\Parsoid\DOM\Element)
#7 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/DOMPostProcessor.php(907): Wikimedia\Parsoid\Wt2Html\DOMPostProcessor->process(Wikimedia\Parsoid\DOM\Element)
#8 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/ParserPipeline.php(180): Wikimedia\Parsoid\Wt2Html\DOMPostProcessor->processChunkily(string, array)
#9 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/ParserPipelineFactory.php(308): Wikimedia\Parsoid\Wt2Html\ParserPipeline->parseChunkily(string, array)
#10 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Wikitext/ContentModelHandler.php(123): Wikimedia\Parsoid\Wt2Html\ParserPipelineFactory->parse(string)
#11 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Parsoid.php(172): Wikimedia\Parsoid\Wikitext\ContentModelHandler->toDOM(Wikimedia\Parsoid\Ext\ParsoidExtensionAPI)
#12 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/src/Parsoid.php(210): Wikimedia\Parsoid\Parsoid->parseWikitext(MediaWiki\Parser\Parsoid\Config\PageConfig, ParserOutput, array)
#13 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/extension/src/Rest/Handler/ParsoidHandler.php(587): Wikimedia\Parsoid\Parsoid->wikitext2html(MediaWiki\Parser\Parsoid\Config\PageConfig, array, NULL, ParserOutput)
#14 /srv/mediawiki/php-1.39.0-wmf.7/vendor/wikimedia/parsoid/extension/src/Rest/Handler/PageHandler.php(88): MWParsoid\Rest\Handler\ParsoidHandler->wt2html(MediaWiki\Parser\Parsoid\Config\PageConfig, array)
#15 /srv/mediawiki/php-1.39.0-wmf.7/includes/Rest/Router.php(414): MWParsoid\Rest\Handler\PageHandler->execute()
RhinosF1 added a subscriber: RhinosF1.Apr 14 2022, 3:24 PM
ihurbain added a comment.Apr 14 2022, 3:26 PM

Yeah, the ->nextSibling goes to null, and in the next loop iteration crashes violently.

I'm preparing a revert patch for this as we speak, and I'll try to prepare a hack to keep the feature while not breaking on this thing - there's a deeper bug there to be fixed, but there may be a mitigation in the meantime.

gerritbot added a comment.Apr 14 2022, 3:51 PM

Change 780885 had a related patch set uploaded (by Isabelle Hurbain-Palatin; author: Isabelle Hurbain-Palatin):

[mediawiki/services/parsoid@master] Workaround for DOMNode::apendChild() production crasher

https://gerrit.wikimedia.org/r/780885

gerritbot added a project: Patch-For-Review.Apr 14 2022, 3:51 PM
ihurbain added a comment.Apr 14 2022, 3:56 PM

I have two patches that should fix the issue - but I don't know which one we want to deploy.

It's also unclear to me what the procedure is - I guess we'd have to release a new version of Parsoid with one of these patches and update vendor, but I'm not clear on how to handle that; I'm deferring to my teammates expertise on the topic.

ssastry added a comment.Apr 14 2022, 4:14 PM

My recommendation is: given that VE editing is not recommended on these pages and the only impact is VE editability, and it is a small number of pages, we don't block the train. But, we can figure out if we want to rollout a fix that @ihurbain was worked on. I don't think a revert is warranted either.

We can even edit those pages and fix them .. if they are just a handful.

ssastry added a comment.Apr 14 2022, 4:17 PM

My recommendation for now is: unblock the train and proceed. We can figure out separately how best to minimize the impact.

dancy added a comment.Apr 14 2022, 4:32 PM

Thanks. I will go with this advice if nothing changes within 1.5 hours.

dancy removed a parent task: T305213: 1.39.0-wmf.7 deployment blockers.Apr 14 2022, 5:17 PM
ssastry lowered the priority of this task from Unbreak Now! to High.Apr 14 2022, 5:26 PM
ssastry added a comment.Apr 14 2022, 5:48 PM

Looking at logstash, so far it looks about 4 metawiki pages are impacted. So, we'll deal with this as part of next week's train.

cscott added a comment.Apr 14 2022, 6:14 PM

(Might be worth considering adding more "pages of this type" (whatever that is) to rt testing?)

gerritbot added a comment.Apr 20 2022, 4:40 PM

Change 784731 had a related patch set uploaded (by Isabelle Hurbain-Palatin; author: Isabelle Hurbain-Palatin):

[mediawiki/services/parsoid@master] Use DOMRangeBuilder "top level range" algorithm for nested ranges

https://gerrit.wikimedia.org/r/784731

gerritbot added a comment.Apr 21 2022, 10:21 AM

Change 784731 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Use DOMRangeBuilder "top level range" algorithm for nested ranges

https://gerrit.wikimedia.org/r/784731

gerritbot added a comment.Apr 21 2022, 12:23 PM

Change 780885 abandoned by Isabelle Hurbain-Palatin:

[mediawiki/services/parsoid@master] Workaround for DOMNode::apendChild() production crasher

Reason:

https://gerrit.wikimedia.org/r/780885

Maintenance_bot removed a project: Patch-For-Review.Apr 21 2022, 12:32 PM
jeena added a subscriber: jeena.Apr 21 2022, 6:29 PM

Still happening in 1.39.0-wmf.8
Find reqId in Logstash

brennen added a subscriber: brennen.Apr 21 2022, 6:31 PM

Noting a handful of these ongoing in 1.39.0-wmf.8 (T305214).

Arlolra added a comment.Apr 21 2022, 6:54 PM

Should be fixed by T306186#7870786 which will ride the train next week

ssastry closed this task as Resolved.Apr 22 2022, 3:39 AM
ssastry assigned this task to ihurbain.
gerritbot added a comment.Apr 25 2022, 5:21 PM

Change 785896 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/vendor@master] Bump parsoid to 0.16.0-a7

https://gerrit.wikimedia.org/r/785896

gerritbot added a project: Patch-For-Review.Apr 25 2022, 5:21 PM
gerritbot added a comment.Apr 25 2022, 6:30 PM

Change 785896 merged by jenkins-bot:

[mediawiki/vendor@master] Bump parsoid to 0.16.0-a7

https://gerrit.wikimedia.org/r/785896

ReleaseTaggerBot added a project: MW-1.39-notes (1.39.0-wmf.9; 2022-04-25).Apr 25 2022, 7:00 PM
Maintenance_bot removed a project: Patch-For-Review.Apr 25 2022, 7:30 PM
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL