Page MenuHomePhabricator

LanguageConverter: Call to member function replace() on null
Closed, ResolvedPublic5 Estimated Story PointsPRODUCTION ERROR

Description

Error
trace
from /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(713)
#0 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(683): LanguageConverter->translate(string, string)
#1 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(943): LanguageConverter->autoConvert(string, string)
#2 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(665): LanguageConverter->recursiveConvertTopLevel(string, string)
#3 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(943): LanguageConverter->autoConvert(string, string)
#4 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(907): LanguageConverter->recursiveConvertTopLevel(string, string)
#5 /srv/mediawiki/php-1.41.0-wmf.10/extensions/CentralNotice/includes/BannerRenderer.php(170): LanguageConverter->convertTo(string, string)
#6 /srv/mediawiki/php-1.41.0-wmf.10/extensions/CentralNotice/includes/specials/SpecialBannerLoader.php(230): BannerRenderer->toHtml()
#7 /srv/mediawiki/php-1.41.0-wmf.10/extensions/CentralNotice/includes/specials/SpecialBannerLoader.php(55): SpecialBannerLoader->getJsNotice()
#8 /srv/mediawiki/php-1.41.0-wmf.10/includes/specialpage/SpecialPage.php(701): SpecialBannerLoader->execute(NULL)
#9 /srv/mediawiki/php-1.41.0-wmf.10/includes/specialpage/SpecialPageFactory.php(1554): SpecialPage->run(NULL)
#10 /srv/mediawiki/php-1.41.0-wmf.10/includes/MediaWiki.php(328): MediaWiki\SpecialPage\SpecialPageFactory->executePath(string, RequestContext)
#11 /srv/mediawiki/php-1.41.0-wmf.10/includes/MediaWiki.php(925): MediaWiki->performRequest()
#12 /srv/mediawiki/php-1.41.0-wmf.10/includes/MediaWiki.php(579): MediaWiki->main()
#13 /srv/mediawiki/php-1.41.0-wmf.10/index.php(50): MediaWiki->run()
#14 /srv/mediawiki/php-1.41.0-wmf.10/index.php(46): wfIndexMain()
#15 /srv/mediawiki/w/index.php(3): require(string)
#16 {main}
Impact

Referrer was from zh-yue wiki.

Notes

Also spotted at the same time, likely related: PHP Notice: Undefined index: yue with the following stacktrace:

trace2
from /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(713)
#0 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(713): MWExceptionHandler::handleError(integer, string, string, integer, array)
#1 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(683): LanguageConverter->translate(string, string)
#2 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(943): LanguageConverter->autoConvert(string, string)
#3 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(665): LanguageConverter->recursiveConvertTopLevel(string, string)
#4 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(943): LanguageConverter->autoConvert(string, string)
#5 /srv/mediawiki/php-1.41.0-wmf.10/includes/language/LanguageConverter.php(907): LanguageConverter->recursiveConvertTopLevel(string, string)
#6 /srv/mediawiki/php-1.41.0-wmf.10/extensions/CentralNotice/includes/BannerRenderer.php(170): LanguageConverter->convertTo(string, string)
#7 /srv/mediawiki/php-1.41.0-wmf.10/extensions/CentralNotice/includes/specials/SpecialBannerLoader.php(230): BannerRenderer->toHtml()
#8 /srv/mediawiki/php-1.41.0-wmf.10/extensions/CentralNotice/includes/specials/SpecialBannerLoader.php(55): SpecialBannerLoader->getJsNotice()
#9 /srv/mediawiki/php-1.41.0-wmf.10/includes/specialpage/SpecialPage.php(701): SpecialBannerLoader->execute(NULL)
#10 /srv/mediawiki/php-1.41.0-wmf.10/includes/specialpage/SpecialPageFactory.php(1554): SpecialPage->run(NULL)
#11 /srv/mediawiki/php-1.41.0-wmf.10/includes/MediaWiki.php(328): MediaWiki\SpecialPage\SpecialPageFactory->executePath(string, RequestContext)
#12 /srv/mediawiki/php-1.41.0-wmf.10/includes/MediaWiki.php(925): MediaWiki->performRequest()
#13 /srv/mediawiki/php-1.41.0-wmf.10/includes/MediaWiki.php(579): MediaWiki->main()
#14 /srv/mediawiki/php-1.41.0-wmf.10/index.php(50): MediaWiki->run()
#15 /srv/mediawiki/php-1.41.0-wmf.10/index.php(46): wfIndexMain()
#16 /srv/mediawiki/w/index.php(3): require(string)
#17 {main}

Details

Event Timeline

demon created this object with edit policy "Custom Policy".
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Presumably fallout from 504c1a9faf6f62a629bc3f4eb4b91213a1fd006a or e69ed0fee9c72aa8d54fd6e81ceefc1c23fd8200, the two new patches touching LanguageConverter that are riding this train.

Change 922925 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] LanguageConverter::translate: Debug log if no variant table set

https://gerrit.wikimedia.org/r/922925

We can pull that patch in mwdebug but I can't reproduce the error/warning with the given URL. Give me a bit.

There is no error showing up anymore. I think it was something with old caches not being invalidated.

There is no error showing up anymore. I think it was something with old caches not being invalidated.

Looking, I'm agreed that it's probably a deployment gremlin; happy for my patch to just land as a safety-check, or be ignored. :-)

Zabe changed the edit policy from "Custom Policy" to "All Users".May 24 2023, 10:38 PM

Change 922925 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] LanguageConverter::translate: Debug log if no variant table set

https://gerrit.wikimedia.org/r/922925

Probably just simply add hasVariant( $variant ) checks?

hashar triaged this task as Unbreak Now! priority.May 25 2023, 8:20 AM
hashar subscribed.

Train blockers are Unbreak Now! priority.

Train blockers are Unbreak Now! priority.

It stopped happening after ten minutes after the deployment: https://logstash.wikimedia.org/goto/92507b2358ce3044a882beb540ba47ee

I don't think this should be a blocker, it stopped right after the deployment, it's not reproducible, simply should be closed as invalid.

But we should have a stable way to invalidate the cache after new converter introduced though.

But we should have a stable way to invalidate the cache after new converter introduced though.

for five to ten minutes? I don't think so tbh. A bit of 500 flood after each deployment is natural and part of the error budget.

Func lowered the priority of this task from Unbreak Now! to Needs Triage.May 25 2023, 1:24 PM

Lower the priority and remove the blocker parent task per discussion.

I noticed I have some WIP on this caching logic, I will upload a patch for review anyway.

Wait - no more errors because they reverted group 2 to the wmf.9 branch.

Func triaged this task as Unbreak Now! priority.May 25 2023, 2:57 PM

Group2 never got wmf.10. It will get it later today. Group1 was deployed and never reverted: T330216#8878084 (I'm not seeing any revert of any kind)

Then why the trace is from wmf.10, I didn't get it.
Oh, I mean group 1, reverted with T330216#8878101
And they are indeed on wmf.9, can check from https://versions.toolforge.org/ and meta:Special:Version

Anyway, I can reproduce this bug with lang=sh, which has another language converter without main code as its variant.

Change 922538 had a related patch set uploaded (by Func; author: Func):

[mediawiki/extensions/CentralNotice@master] BannerRenderer: Make sure the language variant is valid

https://gerrit.wikimedia.org/r/922538

Change 923276 had a related patch set uploaded (by Ladsgroup; author: Func):

[mediawiki/extensions/CentralNotice@wmf/1.41.0-wmf.10] BannerRenderer: Make sure the language variant is valid

https://gerrit.wikimedia.org/r/923276

Change 923277 had a related patch set uploaded (by Winston Sung; author: Winston Sung):

[mediawiki/core@master] LanguageConverter::translate: Use short-circuit evaluation for source === target situations

https://gerrit.wikimedia.org/r/923277

Change 922538 merged by jenkins-bot:

[mediawiki/extensions/CentralNotice@master] BannerRenderer: Make sure the language variant is valid

https://gerrit.wikimedia.org/r/922538

Change 923279 had a related patch set uploaded (by Func; author: Func):

[mediawiki/extensions/CentralNotice@wmf_deploy] BannerRenderer: Make sure the language variant is valid

https://gerrit.wikimedia.org/r/923279

It seems the wmf_deploy branch should be used instead. Not sure how that works.

Change 923276 abandoned by Ladsgroup:

[mediawiki/extensions/CentralNotice@wmf/1.41.0-wmf.10] BannerRenderer: Make sure the language variant is valid

Reason:

not a deploy branch

https://gerrit.wikimedia.org/r/923276

Change 923279 merged by jenkins-bot:

[mediawiki/extensions/CentralNotice@wmf_deploy] BannerRenderer: Make sure the language variant is valid

https://gerrit.wikimedia.org/r/923279

I don't think this should be a blocker, it stopped right after the deployment, it's not reproducible, simply should be closed as invalid.

Group2 never got wmf.10. It will get it later today. Group1 was deployed and never reverted: T330216#8878084 (I'm not seeing any revert of any kind)

It was rolled back 10 minutes after the deployment:

Does https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralNotice/+/923279/ need to be cherry-picked to wmf/1.41.0-wmf.10 and deployed (or one of the other patches here?)

It's in the wmf_deploy branch (CentralNotice) and that is merged but I don't know how that gets deployed.

We branch CentralNotice each week from wmf_deploy (instead of master like the other extensions). So, they get a wmf/1.41.0-wmf.10 like other extensions—we'd need to cherry-pick there for deployment.

Change 923276 restored by Ladsgroup:

[mediawiki/extensions/CentralNotice@wmf/1.41.0-wmf.10] BannerRenderer: Make sure the language variant is valid

https://gerrit.wikimedia.org/r/923276

hrm, seems like the backport to 1.41.0-wmf.10 causes failing tests somehow (even though it merged just fine into wmf_deploy: ChoiceDataProviderTest::testProviderResponse—did something get refactored somewhere?

hrm, seems like the backport to 1.41.0-wmf.10 causes failing tests somehow (even though it merged just fine into wmf_deploy: ChoiceDataProviderTest::testProviderResponse—did something get refactored somewhere?

The message in the CI logs is Use of Hooks::runWithoutAbort was deprecated in MediaWiki 1.35., the replacement it is part of https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralNotice/+/922624 as merge to wmf_deploy. But it should be possible to just cherry-pick the replacement from https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralNotice/+/912954

Change 923276 merged by jenkins-bot:

[mediawiki/extensions/CentralNotice@wmf/1.41.0-wmf.10] BannerRenderer: Make sure the language variant is valid

https://gerrit.wikimedia.org/r/923276

Mentioned in SAL (#wikimedia-operations) [2023-05-25T22:05:28Z] <zabe@deploy1002> Started scap: Backport for [[gerrit:923283|Replace deprecated Hooks::runWithoutAbort (T335536)]], [[gerrit:923276|BannerRenderer: Make sure the language variant is valid (T337427)]]

Mentioned in SAL (#wikimedia-operations) [2023-05-25T22:06:59Z] <zabe@deploy1002> zabe and ladsgroup: Backport for [[gerrit:923283|Replace deprecated Hooks::runWithoutAbort (T335536)]], [[gerrit:923276|BannerRenderer: Make sure the language variant is valid (T337427)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-05-25T22:14:42Z] <zabe@deploy1002> Finished scap: Backport for [[gerrit:923283|Replace deprecated Hooks::runWithoutAbort (T335536)]], [[gerrit:923276|BannerRenderer: Make sure the language variant is valid (T337427)]] (duration: 09m 14s)

Zabe assigned this task to Func.