Scap should not know or care about which LCStore subclass is selected as the $wgLocalisationCacheConf['storeClass']. At the moment scap assumes LCStoreCDB is in use. This assumption is manifest in hard-coded checks for l10n_cache-en.cdb, etc. This prevents us from migrating to LCStoreStaticArray.
|Open||None||T212460 Adopt static array files for local disk storage of values (epic)|
|Stalled||None||T99740 Use static php array files for l10n cache at WMF (instead of CDB)|
|Open||Ladsgroup||T105683 Add Scap support for static-array format of LCStore|
|Resolved||LarsWirzenius||T243358 Tag a new release of scap|
- Mentioned In
- T99740: Use static php array files for l10n cache at WMF (instead of CDB)
rMSCA045d2d0fc9a1: Don't assume current l10n cache files are .cdb
rMSCAf92d080b61b7: Don't assume current l10n cache files are .cdb
rMSCAd68141c95d5b: Don't assume current l10n cache files are .cdb
rMSCAe391ba0d70d7: Don't assume current l10n cache files are .cdb
rMSCAd7db8de6e496: Don't assume current l10n cache files are .cdb
- Mentioned Here
- T243358: Tag a new release of scap
T99740: Use static php array files for l10n cache at WMF (instead of CDB)
T53174: Scap broken for deploying new versions of MediaWiki due to ExtensionMessage file not being created
The specific check that exists for l10n_cache-en.cdb is to determine if the --force flag needs to be passed to rebuildLocalisationCache.php or not. To see why this is needed you have to jump in the wayback machine and travel back to the bash implementation of mw-update-l10n and the fix for T53174: Scap broken for deploying new versions of MediaWiki due to ExtensionMessage file not being created (https://gerrit.wikimedia.org/r/#/c/113260/). Removing the check entirely doesn't seem like a good idea, but maybe we can find a way to generalize it?
It seems that there are other changes that will be needed in scap however if we are going to replace LCStoreCDB with LCStoreStaticArray. See scap.tasks._call_rebuildLocalisationCache() and scap.tasks.merge_cdb_updates() as a couple of things that are obviously related to the CDB cache layer.
In practicality I think it actually is scap's job to understand the build artifacts in use for a MediaWiki deployment and how to properly manage them. Scap is not a general purpose tool that is agnostic of usage. It is a very purpose built tool to manage MediaWiki deployments for the WMF production cluster. It might be better to reword this feature request as "Add support for LCStoreStaticArray l10n cache" and determine if we need to continue to support LCStoreCDB or can unilaterally convert the beta cluster and production usage to the new storage backend.
It was (sort of) implemented in https://gerrit.wikimedia.org/r/#/c/mediawiki/tools/scap/+/224520/ (and several follow up patches). Then it was removed in https://gerrit.wikimedia.org/r/#/c/mediawiki/tools/scap/+/240440/.
Below are ideas from the parent task about implementing transition stage in Scap.
I like Ori's version better here as it allows for generating the array format ahead of any wiki using it (not even testwiki). And then doing the switch gradually separate from that. Ori's version also allows us to be 100% on the new array format and still (for a short time) generate the cdb files. Thus allowing an instance switch-back using nothing more but a config patch and SWAT (no full Scap l10n-rebuild required).
I deployed it but I'm not sure if it works with mwscript:
ladsgroup@mwmaint1002:~$ MW_FORCE_LC_CLASS="really" mwscript eval.php --wiki=wikidatawiki > getenv( 'MW_FORCE_LC_CLASS' ); > ^D ladsgroup@mwmaint1002:~$ export MW_FORCE_LC_CLASS="really" ladsgroup@mwmaint1002:~$ mwscript eval.php --wiki=wikidatawiki > getenv( 'MW_FORCE_LC_CLASS' );
Ah right. We run mediawiki code as a separate user and a clean shell environment with its env variables puppetized (eg. MW STAGING DIR etc.). This is to ensure consistency between users and avoid e.g. poisoning umask and causing files to be made personal or without the correct group.
I assumed both ENV and CLI option would work equally easily, but given not, perhaps go for the rebuildLocalisationCache.php CLI option instead?