Wikidata's content handlers trigger many "ParserOutput does not specify a language and no page language set in helper" warnings in production
Open, Needs TriagePublic
Actions

Assigned To

Authored By

	Ladsgroup
	Feb 27 2025, 1:37 PM

Description

Causing 60K warnings every hour: https://logstash.wikimedia.org/goto/1aaf1ee5c9e4a202c576ccf5533e1a68

An example
https://logstash.wikimedia.org/app/discover#/doc/logstash-*/logstash-mediawiki-1-7.0.0-1-2025.02.27?id=gOSeR5UBRNrRHqkykcu7

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	HtmlOutputRendererHelper::getHtmlOutputContentLanguage: Add a stack trace for huntability	mediawiki/core	master	+4 -1
	Set ParserOutput language in FullEntityParserOutputGenerator	mediawiki/extensions/Wikibase	master	+13 -2

Customize query in gerrit

Related Objects

Mentioned In: T362312: VE triggers internal warning: "ParserOutput does not specify a language"
Mentioned Here: rMWe7f21f6e6438: HtmlOutputRendererHelper: fall back to page language

Event Timeline

Ladsgroup created this task.Feb 27 2025, 1:37 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 27 2025, 1:37 PM

All the log entries I'm seeing are to requests core REST routes, e.g. /w/rest.php/v1/revision/2317716284/html. I don't think we're the best team to look into this.

Ah thanks! They all were to wikidata.org so that confused me.

In T387453#10586891, @Jakob_WMDE wrote:

All the log entries I'm seeing are to requests core REST routes, e.g. /w/rest.php/v1/revision/2317716284/html. I don't think we're the best team to look into this.

Does Wikibase specify the output language when generating a ParserOutput object for a wikibase entity? Recording the actual output language in the ParserOutput is a relatively new feature (see https://gerrit.wikimedia.org/r/c/mediawiki/core/+/836297), so I suspect that Wikibase doesn't use it. That may cause the warning.

This warning dates from e7f21f6e6438 by @daniel; it looks like we now expect callers at this point to have set the language via calling setPageLanguage() on the helper. Obviously we can just hack back in the fallback language and revert Daniel's warning, but I imagine this wants actually fixing? Will push a patch adding a trace.

Change #1124106 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] HtmlOutputRendererHelper::getHtmlOutputContentLanguage: Add a stack trace for huntability

https://gerrit.wikimedia.org/r/1124106

gerritbot added a project: Patch-For-Review.Mar 3 2025, 2:14 PM

I had a look, and could repro locally (and plop a debugger in there).

The (probable) chain is ContentHandler::getParserOutput calls ContentHandler::fillParserOutput, which passes the $parserOutput as reference. fillParserOutput is overriden by Wikibase EntityHandler::fillParserOutput, which indeed does not set a language. Setting the language there fixes the issue (it's quite unclear to me *which* language we want there, so I'll let people who know have a proper look at it.)

I assume there's probably a similar thing on PropertyHandler, but I have not checked.

Of note: the issue only reproduces (on my setup) if I send a request without a Accept-Language header - otherwise the Accept-Language header is used as $targetLanguage somewhere in the HtmlOutputRendererHelper, and the ParserOutput language gets defined along the way.

Jdforrester-WMF renamed this task from ParserOutput does not specify a language and no page language set in helper to Wikidata's content handlers trigger many "ParserOutput does not specify a language and no page language set in helper" warnings in production.Mar 3 2025, 6:57 PM

Tagging Content-Transform-Team for awareness. Fixing this is best done by the Wikibase team.

Jakob_WMDE added a project: Wikibase Reuse Team.Mar 17 2025, 10:20 AM

Ifrahkhanyaree_WMDE moved this task from Incoming tickets outside WPP to To polish on the Wikibase Reuse Team board.Mar 18 2025, 11:38 AM

ihurbain moved this task from Backlog to Tracking on the Content-Transform-Team board.Mar 20 2025, 2:21 PM

Ifrahkhanyaree_WMDE moved this task from To polish to Polished on the Wikibase Reuse Team board.Mar 25 2025, 10:35 AM

Change #1131021 had a related patch set uploaded (by Jakob; author: Jakob):

[mediawiki/extensions/Wikibase@master] Set ParserOutput language in EntityHandler

https://gerrit.wikimedia.org/r/1131021

Jakob_WMDE claimed this task.Mar 27 2025, 4:44 PM

Jakob_WMDE moved this task from Polished to Sprint 41 on the Wikibase Reuse Team board.

Jakob_WMDE edited projects, added Wikibase Reuse Team (Sprint 41); removed Wikibase Reuse Team.

Jakob_WMDE moved this task from To Do to Peer Review on the Wikibase Reuse Team (Sprint 41) board.

Change #1131021 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Set ParserOutput language in FullEntityParserOutputGenerator

https://gerrit.wikimedia.org/r/1131021

ReleaseTaggerBot added a project: MW-1.44-notes (1.44.0-wmf.23; 2025-04-01).Mar 28 2025, 3:00 PM

Jakob_WMDE moved this task from Peer Review to Done on the Wikibase Reuse Team (Sprint 41) board.Mar 31 2025, 9:01 AM

Ifrahkhanyaree_WMDE closed this task as Resolved.Apr 1 2025, 10:32 AM

For record

Ollie.Shotton_WMDE awarded a token.Apr 3 2025, 1:15 PM

ihurbain mentioned this in T362312: VE triggers internal warning: "ParserOutput does not specify a language".Apr 11 2025, 2:25 PM

https://logstash.wikimedia.org/goto/92bdfde8121cc98a9b7cee445b010666 seems to show still ~1500 of these a day coming from Wikidata - I think it's worth re-opening this one, if only because James' "stack trace" patch is attached to it. (Feel free to re-close and if I'm wrong in my assessment, obviously.)

Change #1124106 merged by jenkins-bot: