Page MenuHomePhabricator
Create Task
Maniphest T284693

Wikimedia\Assert\InvariantException: Invariant failed: Bad UTF-8 at end of string (3 byte sequence)
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

Error
normalized_message
[{reqId}] {exception_url}   Wikimedia\Assert\InvariantException: Invariant failed: Bad UTF-8 at end of string (3 byte sequence)
exception.trace
from /srv/mediawiki/php-1.37.0-wmf.7/vendor/wikimedia/assert/src/Assert.php(224)
#0 /srv/mediawiki/php-1.37.0-wmf.7/vendor/wikimedia/parsoid/src/Utils/PHPUtils.php(218): Wikimedia\Assert\Assert::invariant(boolean, string)
#1 /srv/mediawiki/php-1.37.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/PP/Processors/WrapTemplates.php(963): Wikimedia\Parsoid\Utils\PHPUtils::safeSubstr(string, integer, integer)
#2 /srv/mediawiki/php-1.37.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/PP/Processors/WrapTemplates.php(1226): Wikimedia\Parsoid\Wt2Html\PP\Processors\WrapTemplates::encapsulateTemplates(DOMDocument, Wikimedia\Parsoid\Wt2Html\PageConfigFrame, array, array)
#3 /srv/mediawiki/php-1.37.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/PP/Processors/WrapTemplates.php(1239): Wikimedia\Parsoid\Wt2Html\PP\Processors\WrapTemplates::wrapTemplatesInTree(DOMDocument, Wikimedia\Parsoid\Wt2Html\PageConfigFrame, DOMElement)
#4 /srv/mediawiki/php-1.37.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/DOMPostProcessor.php(158): Wikimedia\Parsoid\Wt2Html\PP\Processors\WrapTemplates->run(Wikimedia\Parsoid\Config\Env, DOMElement, array, boolean)
#5 /srv/mediawiki/php-1.37.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/DOMPostProcessor.php(853): Wikimedia\Parsoid\Wt2Html\DOMPostProcessor->Wikimedia\Parsoid\Wt2Html\{closure}(DOMElement, array, boolean)
#6 /srv/mediawiki/php-1.37.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/DOMPostProcessor.php(903): Wikimedia\Parsoid\Wt2Html\DOMPostProcessor->doPostProcess(DOMElement)
#7 /srv/mediawiki/php-1.37.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/DOMPostProcessor.php(920): Wikimedia\Parsoid\Wt2Html\DOMPostProcessor->process(DOMElement)
#8 /srv/mediawiki/php-1.37.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/ParserPipeline.php(178): Wikimedia\Parsoid\Wt2Html\DOMPostProcessor->processChunkily(string, array)
#9 /srv/mediawiki/php-1.37.0-wmf.7/vendor/wikimedia/parsoid/src/Wt2Html/ParserPipelineFactory.php(307): Wikimedia\Parsoid\Wt2Html\ParserPipeline->parseChunkily(string, array)
#10 /srv/mediawiki/php-1.37.0-wmf.7/vendor/wikimedia/parsoid/src/Core/WikitextContentModelHandler.php(106): Wikimedia\Parsoid\Wt2Html\ParserPipelineFactory->parse(string)
#11 /srv/mediawiki/php-1.37.0-wmf.7/vendor/wikimedia/parsoid/src/Parsoid.php(162): Wikimedia\Parsoid\Core\WikitextContentModelHandler->toDOM(Wikimedia\Parsoid\Config\Env)
#12 /srv/mediawiki/php-1.37.0-wmf.7/vendor/wikimedia/parsoid/src/Parsoid.php(194): Wikimedia\Parsoid\Parsoid->parseWikitext(MWParsoid\Config\PageConfig, array)
#13 /srv/mediawiki/php-1.37.0-wmf.7/vendor/wikimedia/parsoid/extension/src/Rest/Handler/ParsoidHandler.php(589): Wikimedia\Parsoid\Parsoid->wikitext2html(MWParsoid\Config\PageConfig, array, NULL)
#14 /srv/mediawiki/php-1.37.0-wmf.7/vendor/wikimedia/parsoid/extension/src/Rest/Handler/PageHandler.php(88): MWParsoid\Rest\Handler\ParsoidHandler->wt2html(MWParsoid\Config\PageConfig, array)
#15 /srv/mediawiki/php-1.37.0-wmf.7/includes/Rest/Router.php(395): MWParsoid\Rest\Handler\PageHandler->execute()
#16 /srv/mediawiki/php-1.37.0-wmf.7/includes/Rest/Router.php(322): MediaWiki\Rest\Router->executeHandler(MWParsoid\Rest\Handler\PageHandler)
#17 /srv/mediawiki/php-1.37.0-wmf.7/includes/Rest/EntryPoint.php(165): MediaWiki\Rest\Router->execute(MediaWiki\Rest\RequestFromGlobals)
#18 /srv/mediawiki/php-1.37.0-wmf.7/includes/Rest/EntryPoint.php(130): MediaWiki\Rest\EntryPoint->execute()
#19 /srv/mediawiki/php-1.37.0-wmf.7/rest.php(31): MediaWiki\Rest\EntryPoint::main()
#20 /srv/mediawiki/w/rest.php(3): require(string)
#21 {main}
Impact
  • ~50 times in the last 30 min
Notes

Details

Request URL
https://ml.wikipedia.org/w/rest.php/ml.wikipedia.org/v3/page/pagebundle/%E0%B4%B5%E0%B4%BF%E0%B4%95%E0%B5%8D%E0%B4%95%E0%B4%BF%E0%B4%AA%E0%B5%80%E0%B4%A1%E0%B4%BF%E0%B4%AF%3A%E0%B4%B5%E0%B4%BF%E0%B4%95%E0%B5%8D%E0%B4%95%E0%B4%BF%E0%B4%B8%E0%B4%82%E0%B4%97%E0%B4%AE%E0%B5%8B%E0%B4%A4%E0%B5%8D%E0%B4%B8%E0%B4%B5%E0%B4%82_-_2013%2F%E0%B4%AA%E0%B4%99%E0%B5%8D%E0%B4%95%E0%B5%86%E0%B4%9F%E0%B5%81%E0%B4%95%E0%B5%8D%E0%B4%95%E0%B5%81%E0%B4%B5%E0%B4%BE%E0%B5%BB/1882310
Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Bump parsoid to 0.17.0-a5mediawiki/vendormaster+285 -82
Ensure DSR computation is accurate if an unclosed comment is presentmediawiki/services/parsoidmaster+73 -8
Customize query in gerrit

Related Objects

Event Timeline

jeena created this task.Jun 9 2021, 7:55 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 9 2021, 7:55 PM
ssastry subscribed.Jun 9 2021, 8:00 PM

For the benefit of train engineers, I wanted to note that these UTF-8 errors seen in Parsoid are expected to come up in spurts (lots of errors on the same page OR repeated errors because of retries). So, unless there is a significant spike, this shouldn't be a concern for deployments.

Arlolra merged a task: T284738: Parsoid breaks at rendering br.wikisource.org page.Jun 10 2021, 3:36 PM
Arlolra added subscribers: Kelson, Arlolra.
Arlolra triaged this task as Medium priority.Jun 10 2021, 6:51 PM
Arlolra moved this task from Needs Triage to Bugs & Crashers on the Parsoid board.
• Krinkle subscribed.Jun 11 2021, 5:08 PM

@ssastry I don't know much about these errors, but it sounds like they have to do with bad user input. Are these generally solved by improving the parser to support more input, or also by categorically letting these fail in a different way? It seems like a categorical change may be appropiate here such that these kinds of issues more generally won't result in an HTTP 5xx and exception log entry, but instead e.g. to a Parsoid-specific log channel with an INFO or WARNING severity for your team to monitor. Possibly also a 4xx status response, but that's a different question.

Deployers and people outside the team have not, and imho should not, try to memorise internals of extensions and what kinds of fatals are "real" fatals. These should be fixed on the producer side instead. Usually that means fixing the root cause if it's easy, but if that's non-trivial or if it's a sustained category of erorrs like this one, then maybe a different mitigation could take place first to ensure they get recategorised. That might change nothing for end-users, but would mean Parsoid plays along better with how other MediaWiki components express themselves in Logstash.

ssastry added a comment.Jun 11 2021, 6:14 PM

These utf-8 errors are caused by a combination of things: either bad encoding stored in the db, "bad markup", or real bugs in Parsoid. These errors helped us find and fix most of these. The remaining ones are still some lingering ones we haven't investigated because there are far fewer of them now. But, yes I agree that it is probably time to migrate these remaining errors to a different log channel. We'll chat about this in the coming week.

• Krinkle moved this task from Untriaged to June 2021 on the Wikimedia-production-error board.Jun 12 2021, 12:16 AM
Kelson added a comment.Aug 19 2021, 12:57 PM

@Krinkle Thx

ssastry added a project: Parsoid-Read-Views.Feb 8 2022, 10:14 PM
• JMcLeod_WMF moved this task from Uncategorized to Phase 2 - testwiki Main namespace support on the Parsoid-Read-Views board.Jul 28 2022, 2:18 PM
• JMcLeod_WMF edited projects, added Parsoid-Read-Views (Phase 2 - testwiki Main namespace support); removed Parsoid-Read-Views.
ssastry claimed this task.Oct 12 2022, 12:50 AM
gerritbot added a comment.Oct 12 2022, 12:57 AM

Change 841598 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/services/parsoid@master] Ensure DSR computation is accurate if an unclosed comment is present

https://gerrit.wikimedia.org/r/841598

gerritbot added a project: Patch-For-Review.Oct 12 2022, 12:57 AM
ssastry moved this task from Backlog to In Progress on the Parsoid-Read-Views (Phase 2 - testwiki Main namespace support) board.Oct 12 2022, 2:12 AM
ssastry moved this task from In Progress to Code Review on the Parsoid-Read-Views (Phase 2 - testwiki Main namespace support) board.
gerritbot added a comment.Oct 24 2022, 11:47 PM

Change 841598 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Ensure DSR computation is accurate if an unclosed comment is present

https://gerrit.wikimedia.org/r/841598

Maintenance_bot removed a project: Patch-For-Review.Oct 25 2022, 12:30 AM
Kelson added a comment.Oct 29 2022, 1:06 PM

See also https://github.com/openzim/mwoffliner/issues/1482

ssastry added a comment.Oct 30 2022, 3:46 AM

https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/+/842879 might take care of that one.

gerritbot added a comment.Oct 31 2022, 10:16 PM

Change 851140 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/vendor@master] Bump parsoid to 0.17.0-a5

https://gerrit.wikimedia.org/r/851140

gerritbot added a project: Patch-For-Review.Oct 31 2022, 10:16 PM
gerritbot added a comment.Oct 31 2022, 10:35 PM

Change 851140 merged by jenkins-bot:

[mediawiki/vendor@master] Bump parsoid to 0.17.0-a5

https://gerrit.wikimedia.org/r/851140

Maintenance_bot removed a project: Patch-For-Review.Oct 31 2022, 11:30 PM
ssastry added a subscriber: Content-Transform-Team-WIP.Nov 7 2022, 4:24 PM
Aklapper added a project: Content-Transform-Team-WIP.Nov 7 2022, 4:38 PM
Aklapper removed a subscriber: Content-Transform-Team-WIP.
ssastry moved this task from Backlog to To Verify on the Content-Transform-Team-WIP board.Nov 7 2022, 4:44 PM
ssastry closed this task as Resolved.Nov 10 2022, 10:43 PM

I am going to resolve this instance of the phab task. There are now instances that purport to be on main pages of ruwiki, bewiki, ukwiki .. which I'll look at separately but they may be parses for posted wikitext vs page wikitext. It is a bit hard to track those without logging the posted wikitext.

Kelson added a comment.Dec 1 2022, 11:19 AM

On my end, this looks better now. Thx!

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits