LanguageConverter: Call to member function replace() on null
Closed, ResolvedPublic5 Estimated Story PointsPRODUCTION ERROR
Actions

Description

Error

Call to member function replace() on null

trace

from /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(713)
#0 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(683): LanguageConverter->translate(string, string)
#1 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(943): LanguageConverter->autoConvert(string, string)
#2 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(665): LanguageConverter->recursiveConvertTopLevel(string, string)
#3 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(943): LanguageConverter->autoConvert(string, string)
#4 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(907): LanguageConverter->recursiveConvertTopLevel(string, string)
#5 /srv/mediawiki/php-1.41.0-wmf.10/extensions/CentralNotice/includes/BannerRenderer.php(170): LanguageConverter->convertTo(string, string)
#6 /srv/mediawiki/php-1.41.0-wmf.10/extensions/CentralNotice/includes/specials/SpecialBannerLoader.php(230): BannerRenderer->toHtml()
#7 /srv/mediawiki/php-1.41.0-wmf.10/extensions/CentralNotice/includes/specials/SpecialBannerLoader.php(55): SpecialBannerLoader->getJsNotice()
#8 /srv/mediawiki/php-1.41.0-wmf.10/includes/specialpage/SpecialPage.php(701): SpecialBannerLoader->execute(NULL)
#9 /srv/mediawiki/php-1.41.0-wmf.10/includes/specialpage/SpecialPageFactory.php(1554): SpecialPage->run(NULL)
#10 /srv/mediawiki/php-1.41.0-wmf.10/includes/MediaWiki.php(328): MediaWiki\SpecialPage\SpecialPageFactory->executePath(string, RequestContext)
#11 /srv/mediawiki/php-1.41.0-wmf.10/includes/MediaWiki.php(925): MediaWiki->performRequest()
#12 /srv/mediawiki/php-1.41.0-wmf.10/includes/MediaWiki.php(579): MediaWiki->main()
#13 /srv/mediawiki/php-1.41.0-wmf.10/index.php(50): MediaWiki->run()
#14 /srv/mediawiki/php-1.41.0-wmf.10/index.php(46): wfIndexMain()
#15 /srv/mediawiki/w/index.php(3): require(string)
#16 {main}

Impact

Referrer was from zh-yue wiki.

Notes

Also spotted at the same time, likely related: PHP Notice: Undefined index: yue with the following stacktrace:

trace2

from /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(713)
#0 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(713): MWExceptionHandler::handleError(integer, string, string, integer, array)
#1 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(683): LanguageConverter->translate(string, string)
#2 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(943): LanguageConverter->autoConvert(string, string)
#3 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(665): LanguageConverter->recursiveConvertTopLevel(string, string)
#4 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(943): LanguageConverter->autoConvert(string, string)
#5 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(907): LanguageConverter->recursiveConvertTopLevel(string, string)
#6 /srv/mediawiki/php-1.41.0-wmf.10/extensions/CentralNotice/includes/BannerRenderer.php(170): LanguageConverter->convertTo(string, string)
#7 /srv/mediawiki/php-1.41.0-wmf.10/extensions/CentralNotice/includes/specials/SpecialBannerLoader.php(230): BannerRenderer->toHtml()
#8 /srv/mediawiki/php-1.41.0-wmf.10/extensions/CentralNotice/includes/specials/SpecialBannerLoader.php(55): SpecialBannerLoader->getJsNotice()
#9 /srv/mediawiki/php-1.41.0-wmf.10/includes/specialpage/SpecialPage.php(701): SpecialBannerLoader->execute(NULL)
#10 /srv/mediawiki/php-1.41.0-wmf.10/includes/specialpage/SpecialPageFactory.php(1554): SpecialPage->run(NULL)
#11 /srv/mediawiki/php-1.41.0-wmf.10/includes/MediaWiki.php(328): MediaWiki\SpecialPage\SpecialPageFactory->executePath(string, RequestContext)
#12 /srv/mediawiki/php-1.41.0-wmf.10/includes/MediaWiki.php(925): MediaWiki->performRequest()
#13 /srv/mediawiki/php-1.41.0-wmf.10/includes/MediaWiki.php(579): MediaWiki->main()
#14 /srv/mediawiki/php-1.41.0-wmf.10/index.php(50): MediaWiki->run()
#15 /srv/mediawiki/php-1.41.0-wmf.10/index.php(46): wfIndexMain()
#16 /srv/mediawiki/w/index.php(3): require(string)
#17 {main}

Details

MediaWiki Version: 1.41.0-wmf.10
Request URL: https://meta.wikimedia.org/w/index.php?banner=wle_2023_tw&campaign=wle_2023_tw&debug=false&title=Special:BannerLoader&uselang=yue

Related Changes in Gerrit:

Subject	Repo	Branch	Lines +/-
BannerRenderer: Make sure the language variant is valid	mediawiki/extensions/CentralNotice	wmf/1.41.0-wmf.10	+4 -1
BannerRenderer: Make sure the language variant is valid	mediawiki/extensions/CentralNotice	wmf_deploy	+4 -1
BannerRenderer: Make sure the language variant is valid	mediawiki/extensions/CentralNotice	master	+4 -1

Customize query in gerrit

Related Objects
Search...

Status	Subtype	Assigned	Task
Resolved	Release	• demon	T330216 1.41.0-wmf.10 deployment blockers
Resolved	PRODUCTION ERROR	Func	T337427 LanguageConverter: Call to member function replace() on null
Open		None	T354061 LanguageConverter: Add validation and debug log for target language code

Event Timeline

• demon created this task.May 24 2023, 7:07 PM

• demon created this object with edit policy "Custom Policy".

Restricted Application added a project: Wikimedia-production-error. · View Herald TranscriptMay 24 2023, 7:07 PM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Presumably fallout from 504c1a9faf6f62a629bc3f4eb4b91213a1fd006a or e69ed0fee9c72aa8d54fd6e81ceefc1c23fd8200, the two new patches touching LanguageConverter that are riding this train.

Change 922925 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] LanguageConverter::translate: Debug log if no variant table set

https://gerrit.wikimedia.org/r/922925

gerritbot added a project: Patch-For-Review.May 24 2023, 7:22 PM

Ladsgroup subscribed.May 24 2023, 7:39 PM

We can pull that patch in mwdebug but I can't reproduce the error/warning with the given URL. Give me a bit.

There is no error showing up anymore. I think it was something with old caches not being invalidated.

https://logstash.wikimedia.org/goto/1e68b2f14a3fecec8bc44366de679ffb

In T337427#8878206, @Ladsgroup wrote:

There is no error showing up anymore. I think it was something with old caches not being invalidated.

Looking, I'm agreed that it's probably a deployment gremlin; happy for my patch to just land as a safety-check, or be ignored. :-)

Zabe changed the edit policy from "Custom Policy" to "All Users".May 24 2023, 10:38 PM

Winston_Sung moved this task from Backlog to Core on the MediaWiki-Language-converter board.May 25 2023, 3:00 AM

In T337427#8878150, @gerritbot wrote:

Change 922925 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] LanguageConverter::translate: Debug log if no variant table set

https://gerrit.wikimedia.org/r/922925

Probably just simply add hasVariant( $variant ) checks?

Train blockers are Unbreak Now! priority.

https://meta.wikimedia.org/w/index.php?banner=wle_2023_tw&campaign=wle_2023_tw&debug=false&title=Special:BannerLoader&uselang=yue

Cannot reproduce.

In T337427#8879085, @hashar wrote:

Train blockers are Unbreak Now! priority.

It stopped happening after ten minutes after the deployment: https://logstash.wikimedia.org/goto/92507b2358ce3044a882beb540ba47ee

I don't think this should be a blocker, it stopped right after the deployment, it's not reproducible, simply should be closed as invalid.

But we should have a stable way to invalidate the cache after new converter introduced though.

In T337427#8879138, @Func wrote:

But we should have a stable way to invalidate the cache after new converter introduced though.

for five to ten minutes? I don't think so tbh. A bit of 500 flood after each deployment is natural and part of the error budget.

Lower the priority and remove the blocker parent task per discussion.

I noticed I have some WIP on this caching logic, I will upload a patch for review anyway.

Wait - no more errors because they reverted group 2 to the wmf.9 branch.

Func triaged this task as Unbreak Now! priority.May 25 2023, 2:57 PM

Group2 never got wmf.10. It will get it later today. Group1 was deployed and never reverted: T330216#8878084 (I'm not seeing any revert of any kind)

~~Then why the trace is from wmf.10, I didn't get it.~~
Oh, I mean group 1, reverted with T330216#8878101
And they are indeed on wmf.9, can check from https://versions.toolforge.org/ and meta:Special:Version

Anyway, I can reproduce this bug with lang=sh, which has another language converter without main code as its variant.

Change 922538 had a related patch set uploaded (by Func; author: Func):

[mediawiki/extensions/CentralNotice@master] BannerRenderer: Make sure the language variant is valid

https://gerrit.wikimedia.org/r/922538

Func added a project: MediaWiki-extensions-CentralAuth.May 25 2023, 3:26 PM

Winston_Sung edited projects, added MediaWiki-extensions-CentralNotice; removed MediaWiki-extensions-CentralAuth.May 25 2023, 3:28 PM

Restricted Application added a subscriber: Pcoombe. · View Herald TranscriptMay 25 2023, 3:28 PM

Change 923276 had a related patch set uploaded (by Ladsgroup; author: Func):

[mediawiki/extensions/CentralNotice@wmf/1.41.0-wmf.10] BannerRenderer: Make sure the language variant is valid

https://gerrit.wikimedia.org/r/923276

Change 923277 had a related patch set uploaded (by Winston Sung; author: Winston Sung):

[mediawiki/core@master] LanguageConverter::translate: Use short-circuit evaluation for source === target situations

https://gerrit.wikimedia.org/r/923277

Change 922538 merged by jenkins-bot:

[mediawiki/extensions/CentralNotice@master] BannerRenderer: Make sure the language variant is valid

https://gerrit.wikimedia.org/r/922538

Change 923279 had a related patch set uploaded (by Func; author: Func):

[mediawiki/extensions/CentralNotice@wmf_deploy] BannerRenderer: Make sure the language variant is valid

https://gerrit.wikimedia.org/r/923279

It seems the wmf_deploy branch should be used instead. Not sure how that works.

Change 923276 abandoned by Ladsgroup:

[mediawiki/extensions/CentralNotice@wmf/1.41.0-wmf.10] BannerRenderer: Make sure the language variant is valid

Reason:

not a deploy branch

https://gerrit.wikimedia.org/r/923276

ReleaseTaggerBot added a project: MW-1.41-notes (1.41.0-wmf.11; 2023-05-30).May 25 2023, 5:00 PM

greg added a project: Fundraising Tech - Chaos Crew.May 25 2023, 5:20 PM

greg moved this task from Backlog to Ready for Review on the Fundraising Tech - Chaos Crew board.

Change 923279 merged by jenkins-bot:

[mediawiki/extensions/CentralNotice@wmf_deploy] BannerRenderer: Make sure the language variant is valid

https://gerrit.wikimedia.org/r/923279

In T337427#8879118, @Ladsgroup wrote:

I don't think this should be a blocker, it stopped right after the deployment, it's not reproducible, simply should be closed as invalid.

In T337427#8880263, @Ladsgroup wrote:

Group2 never got wmf.10. It will get it later today. Group1 was deployed and never reverted: T330216#8878084 (I'm not seeing any revert of any kind)

It was rolled back 10 minutes after the deployment:

2023-05-24 18:55 Group1 → wmf.10: https://sal.toolforge.org/log/IEMdT4gBGiVuUzOdfWXr
2023-05-24 19:12: Group1 → wmf.9: https://sal.toolforge.org/log/l5wtT4gBxE1_1c7sLc_a

Does https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralNotice/+/923279/ need to be cherry-picked to wmf/1.41.0-wmf.10 and deployed (or one of the other patches here?)

It's in the wmf_deploy branch (CentralNotice) and that is merged but I don't know how that gets deployed.

We branch CentralNotice each week from wmf_deploy (instead of master like the other extensions). So, they get a wmf/1.41.0-wmf.10 like other extensions—we'd need to cherry-pick there for deployment.

Change 923276 restored by Ladsgroup:

[mediawiki/extensions/CentralNotice@wmf/1.41.0-wmf.10] BannerRenderer: Make sure the language variant is valid

https://gerrit.wikimedia.org/r/923276

hrm, seems like the backport to 1.41.0-wmf.10 causes failing tests somehow (even though it merged just fine into wmf_deploy: ChoiceDataProviderTest::testProviderResponse—did something get refactored somewhere?

In T337427#8881790, @thcipriani wrote:

hrm, seems like the backport to 1.41.0-wmf.10 causes failing tests somehow (even though it merged just fine into wmf_deploy: ChoiceDataProviderTest::testProviderResponse—did something get refactored somewhere?

The message in the CI logs is Use of Hooks::runWithoutAbort was deprecated in MediaWiki 1.35., the replacement it is part of https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralNotice/+/922624 as merge to wmf_deploy. But it should be possible to just cherry-pick the replacement from https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralNotice/+/912954

Change 923276 merged by jenkins-bot:

[mediawiki/extensions/CentralNotice@wmf/1.41.0-wmf.10] BannerRenderer: Make sure the language variant is valid

https://gerrit.wikimedia.org/r/923276

Mentioned in SAL (#wikimedia-operations) [2023-05-25T22:05:28Z] <zabe@deploy1002> Started scap: Backport for [[gerrit:923283|Replace deprecated Hooks::runWithoutAbort (T335536)]], [[gerrit:923276|BannerRenderer: Make sure the language variant is valid (T337427)]]

Mentioned in SAL (#wikimedia-operations) [2023-05-25T22:06:59Z] <zabe@deploy1002> zabe and ladsgroup: Backport for [[gerrit:923283|Replace deprecated Hooks::runWithoutAbort (T335536)]], [[gerrit:923276|BannerRenderer: Make sure the language variant is valid (T337427)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-05-25T22:14:42Z] <zabe@deploy1002> Finished scap: Backport for [[gerrit:923283|Replace deprecated Hooks::runWithoutAbort (T335536)]], [[gerrit:923276|BannerRenderer: Make sure the language variant is valid (T337427)]] (duration: 09m 14s)