Page MenuHomePhabricator

Newly installed extensions don't pick up localization messages
Closed, ResolvedPublicBUG REPORT

Description

Setting up mediawiki for local development in Docker, per the instructions in DEVELOPERS.md, installing new extensions seems to fail to pick up the new localization messages. All of the extension messages are in the ⧼message...⧽ form, and extensions which add special pages cause PHP warnings about their aliases not being found.

It's fixed if you run the rebuildLocalisationCache.php maintenance script with --force. This wasn't necessary before, so presumably there's some sort of regression around detecting whether it's needed to be automatically triggered somewhere (in update.php?)

Event Timeline

The LocalisationCache tracks the i18n json files in use, and if/when those change, it invalidates the l10n cache and rebuilds it on the fly, much like how it was also built during the first request after installing the wiki.

To my knowledge this still works, including for example after you git pull and receive new messages or new translations in MediaWiki core or any extension you have installed.

What I suspect is happening is, if you clone an extension at time A, then at time B do something that warrants an update, and then at time C edit LocalSettings.php to enable the new extension, then the modified time of the LocalisationCache is newer than any of the i18n files that were created by git clone at time A.

At time B, it would likely be a git pull in MediaWIki core, or a git pull in an extension you already had installed. Or alternatively, perhaps LocalSettings.php was edited to enable the extension in some kind of conditional way that allowed you to make a web request that didn't error, refresh the LocalisationCache automatically, and then later when cloning the extension you make that conditional true without an edit to LocalSettings.php.

Testing locally on latest master, the mechanism seems to generally work, including for new extensions.

  • View Main_Page?action=edit
  • Clone a WikiEditor extension repo (if already, then first: rm-rf it, disable in LocalSettings, and view Main_Page)
  • Edit LocalSettings.php to enable it.
  • Hard refresh Main_Page?action=edit. Toolbar and labels show up.

I do feel like I did it in the sequence it said to in the installation instructions for the extension, at least.

I wonder if Docker might be introducing some level of host-client time drift that could cause confusion.

I've also just seen someone have this issue while running mw-cli.

That would be me, I was installing a fresh environment using mw-cli. This is what I did:

  1. export NETWORK_SUBNET_PREFIX=10.1.1; export PORT=8080
  2. mw docker mediawiki get-code
  3. mw docker mediawiki create
  4. mw docker mysql create
  5. mw docker mediawiki install --dbtype=mysql

Check that the installation worked as expected, then install CheckUser:

  1. mw docker mediawiki get-code --extension CheckUser
  2. mv composer.local.json-sample composer.local.json
  3. mw docker mediawiki composer update
  4. Add this to local settings, so my admin account gets CheckUser rights:
# CHECKUSER SETTINGS
wfLoadExtension('CheckUser');

$wgGroupPermissions['sysop']['checkuser'] = true;
$wgGroupPermissions['sysop']['checkuser-log'] = true;
$wgGroupPermissions['sysop']['investigate'] = true;

$wgCheckUserLogLogins = true;
$wgCheckUserEnableSpecialInvestigate = true;
$wgGroupPermissions['sysop']['checkuser-temporary-account'] = true;
  1. mw docker mediawiki exec php maintenance/run.php update.php

After that, I get the "Did not find alias..." warning when accessing the Special:CheckUser page. It's fixed by running rebuildLocalisationCache.php with --force

I'm able to reproduce this by simply following the install instructions linked from our MediaWiki-Docker page.
(I'm using macOS 13.6.4 with the latest Docker Desktop (4.27.2), although I also tried it on Windows 11 and saw the same behavior)

  • Install MediaWiki in docker by following the quickstart instructions in DEVELOPERS.md
  • Install the MobileFrontend extension by following the instructions
    • Clone the extension
    • Add it to LocalSettings.php
    • Run update.php
  • Reloading the site in my browser produces Notice: Did not find alias for special page 'MobileOptions' at the top, and messages are in the un-expanded form ⧼...⧽.
  • Running rebuildLocalisationCache.php --force fixes it.

I am seeing this on macOS host system PHP (no Docker).

I cloned SpamBlacklist, enabled in LocalSettings.php, added a URL to the blacklist page, then attempted to add the link containing the URL. I see ⧼spam-blacklisted-link⧽ in the page output. After I run php maintenance/rebuildLocalisationCache.php --lang=en --force, I see the en.json value for spam-blacklisted-link.

Some values in my local setup:

> $wgLocalisationCacheConf
= [
    "class" => "LocalisationCache",
    "store" => "detect",
    "storeClass" => false,
    "storeDirectory" => false,
    "storeServer" => [],
    "forceRecache" => false,
    "manualRecache" => false,
  ]

> $wgMainCacheType
= "memcached-php"

Thanks. I suggest someone in our team take a look next week to see if we can determine the root cause and then address/escalate/delegate accordingly.

Krinkle triaged this task as Medium priority.Mar 12 2024, 5:32 PM

After some digging, I think the regression happened in January (Cc @matmarex) here: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/981576.

Before, we used to use MainConfigDependency (accessing global state) but with the patch linked above, we switched to injecting the config/settings (via ServiceOptions) into ConfigDependency but the problem that caused is https://gerrit.wikimedia.org/g/mediawiki/core/+/7086e5613110552d7303454c540fe6f318f3714a/includes/language/dependency/ConfigDependency.php#39 whereby the cached dependency MessagesDir gets compared to itself (effectively https://gerrit.wikimedia.org/g/mediawiki/core/+/7086e5613110552d7303454c540fe6f318f3714a/includes/language/dependency/ConfigDependency.php#35 and https://gerrit.wikimedia.org/g/mediawiki/core/+/7086e5613110552d7303454c540fe6f318f3714a/includes/language/dependency/ConfigDependency.php#39), meaning that the check in https://gerrit.wikimedia.org/g/mediawiki/core/+/d7cb31fd1db354a974fef0e82692162124436e51/includes/language/LocalisationCache.php#578 would not get into the loop thereby making the isExpired() method in LocalisationCache return false in this case.

I don't see a clean way we can fix this without accessing MainConfig in the global services container. The solution I see here is we need a fresh (uncached) instance of MainConfigNames::MessagesDir to compare with what we have in the cache and if there is a difference, then recompute the localisation cache.

Let me know your thoughts if there is a better way to resolve the issue differently.

NOTE: This issue will happen in all environments: bare metal, MW docker, MW cli etc. And it only happens when new extensions are registered and used immediately, but if we do a git pull in core, and en.json with other message files changes, after registering the new extension, the problem be hidden due to a modified time on the localisation files in core there by triggering a recache.

The difficulty here is that ConfigDependency is supposed to be a generic container for a value from any ServiceOptions config store. As such, altering its isExpired method to refetch its config store would require that it also knows which method to call to do so.

Perhaps the cleanest way to deal with this would be to add a refresh method to ServiceOptions so any given object could know how to refetch itself (with data it was provided when it was created, presumably)?

In the interim, here's a patch that'd just switch back to using MainConfigDependency...

Change 1010913 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/core@master] LocalizationCache: fix non-detection of changes in messages

https://gerrit.wikimedia.org/r/1010913

Can we update documentation to instruct users to run rebuildLocalisationCache.php (or update.php with some custom flag to clear the localisation cache) after adding wfLoadExtension() in LocalSettings.php? I think it's pretty common for extension registration in other platforms to require an additional command to properly register an extension, and it doesn't seem that onerous.

Having it be an optional missable step feels less-ideal. Would there be any drawback to just making it be something that update.php always does?

I mean, it'd be slightly slower. But it's an extremely infrequently run script anyway...

Having it be an optional missable step feels less-ideal. Would there be any drawback to just making it be something that update.php always does?

I think for local development environments, doing this unconditionally isn't very terrible. On one hand, I see that recently per: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/657105, we're already beginning to unconditionally purge other caches (objectcache, RL mesasge cache, module dependendies etc) in update.php? @DLynch, on another hand, I think if we want update.php to do this always we can configure a setting in LocalSettings.php to set $wgLocalisationCacheConf['manualRecache'] to true.

I know that in WMF production, we rebuild localisation caches during scap deployments, so, that is a complete different matter and this issue will never happen in production as of today.

I'm not sure how this impacts 3-rd parties if they're using the default configuration of MW (maybe they're having the same issue)?

Maybe others have smatter ideas on the way forward but I as a developer would expect that if I register an extension in MW, the l10n caches should automatically have all messages in the newly registered extension because not all extensions need the update script to run smoothly because some extensions may not need any DB changes to function smoothly.

update.php already clears most object caches, including to rebuild LocalisationCache if manualRecache is enabled. There is not a currently a known way for LocalisationCache to become out of date without automatically fixing itself — except for the (one) bug described in this task.

Requiring update.php or rebuildLocalisationCache.php to be run regularly imho significantly takes away from the two-step developer experience of "just" installing the vast vast majority of extensions by simply downloading and enabling them, and updating them merely via git pull (and maybe compose update and update.php for the small number of extensions with custom deps and/or recent schema changes).

In particular, because once we recommend running scripts "just in case", this becomes the default experience. It very quickly will feel like it's not worth skipping or doing only when needed because its recommended, and who knows maybe you did need it. This cultivates a feeling of instability, that I expet would last many instability years after it has stopped being needed, because we're told it is needed. So the entire period you did the step for would be remembered as being needed.

The current task represents a bug in LocalisationCache, which affects not only local development but MediaWiki in general and is in need of fixing. It was a fairly recent regression, and a fairly trivial one. The Dependency subclass just stopped working entirely, because "dependency injection" was applied a bit too far, causing a copy of the entire service config to be stored as PHP-serialized string in the cache itself, thus isExpired could never return anything other than false for that object.

Change 1010913 merged by jenkins-bot:

[mediawiki/core@master] language: switch LocalisationCache back to MainConfigDependency

https://gerrit.wikimedia.org/r/1010913

DLynch claimed this task.
DLynch added a subscriber: DAlangi_WMF.

@Krinkle thanks for summarizing everything in the end. 👍🏻

Change 1011040 had a related patch set uploaded (by D3r1ck01; author: Derick Alangi):

[mediawiki/core@master] Restore release notes as the class has been undeprecated

https://gerrit.wikimedia.org/r/1011040

Oops, sorry about that. Thanks for fixing it for me.

Change 1011040 merged by jenkins-bot:

[mediawiki/core@master] Partial revert of REL-NOT, class has been undeprecated

https://gerrit.wikimedia.org/r/1011040