In order to experiment with different methods for building and integrating l10n caches in a containerized world, we'll need to know what the current compilation process looks like in terms of process and computation.
Description
Description
Details
Details
Customize query in gerrit
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | akosiaris | T198901 Migrate production services to kubernetes using the pipeline | |||
| Resolved | Clement_Goubert | T238770 Deploy MediaWiki to Wikimedia production in containers | |||
| Resolved | dancy | T260827 l10n process for MW containerization | |||
| Resolved | dduvall | T261360 Get scap-vagrant working for use in l10n tracing | |||
| Resolved | dancy | T268698 Add flag to rebuildLocalisationCache.php to skip MessageBlobStore::clearGlobalCacheEntry | |||
| Resolved | dancy | T238436 Allow running mergeMessageFileList (or any other maintenance script) without needing a DB connection | |||
| Resolved | dancy | T237148 Allow running rebuildLocalizationCache with Gadgets extension loaded and no DB connection |
Event Timeline
Comment Actions
My notes after tracing the process:
MessageBlobStore clear
Why is this done during LC rebuild? LC is built upon the first scap sync of
test wikis on Tuesdays long before update-wikiversions has been performed.
- What benefit is there to clearing the message blob store cache at this point?
- Is it in fact a race condition regardless of when it's cleared? Wouldn't new requests result in re-cached messages from the previous version of MW before the new version is promoted via sync-wikiversions?
- The cache is cleared again by scap explicitly following a sync-wikiversions so this really seems redundant.
Merge order
The order is which l10n sources for each language are processed determines the
aggregate messages produced. Use of an array concatenation operation
(aggregate_messages + source_messages) means that entries (by key) in each
source_messages that already exist in aggregate_messages will be ignored.
Hooks
There are two hooks run during this process,
LocalisationCacheRecacheFallback and LocalisationCacheRecache. Neither
appear to have be used significantly by existing extensions.[1][2]
[1]: https://codesearch.wmcloud.org/search/?q=LocalisationCacheRecacheFallback&i=nope&files=&repos=
[2]: https://codesearch.wmcloud.org/search/?q=LocalisationCacheRecache&i=nope&files=&repos=
Comment Actions
@tstarling can you provide insight into why the MessageBlobStore is cleared during this process and whether we might be able to remove/decouple it from localization rebuilds? (See above questions/comments.)
For more context, my motivation here to see what parts of localization precaching can be done in a containerized MediaWiki build pipeline (see T259817#6395133) and how far upstream. Before/after all extensions/skins/vendor are integrated? Do we absolutely need all prod configuration or only a list of enabled exts? Do we need to allow extensions to modify messages during compilation or can we deprecate those hooks? (To the latter, it seems like they aren't used in any prod-enabled extension.)
Comment Actions
MessageBlobStore was @Catrope's work, he added this feature to rebuildLocalisationCache in 2010, so he can probably explain better than I can.
Comment Actions
Thanks for taking a look, @tstarling. It looks like there's more context for the MBS cache clear in T222539: Scap deployments are not purging MessageBlobStore (was: Stale localized messages) and the related patchset to scap as well.
I'm handing this task off to @dancy as he's expressed interest in tackling l10n issues related to current deployments and containerization.
Just to summarize my current understanding and seek clarification for the handoff, @Catrope, it seems from your comments in that task and associated commit message that performing the MBS cache clear in LocalisationCache::recache is not correct because recaching is performed on the deployment host long before a sync—i.e. there is a race condition between new production requests and a sync or group promotion. Is that accurate? And is it accurate to say that it should instead be performed in the following two cases?
- Whenever LC files are rebuilt and synced for a currently deployed release. Seems like your patch already addresses this.
- Whenever a release is promoted (following sync-wikiversions). It's not clear to me whether this is already the case as a result of that patchset.
And finally, can we remove the MBS cache clear from LocalisationCache::recache?
Thanks for taking this on, @dancy.
Comment Actions
Talked to @Catrope yesterday and he confirmed that the early MBS cache clear is unnecessary.
Comment Actions
Change 667244 had a related patch set uploaded (by Ahmon Dancy; owner: Ahmon Dancy):
[operations/mediawiki-config@master] wmf-config/CommonSettings.php: Add MW_NO_ETCD handling
Comment Actions
Change 667734 had a related patch set uploaded (by Dduvall; owner: Dduvall):
[mediawiki/core@master] maintenance: Skip setAgentAndTriggers for DB_NONE maintenance tasks
Comment Actions
Change 667735 had a related patch set uploaded (by Dduvall; owner: Dduvall):
[mediawiki/core@master] maintenance: rebuildLocalisationCache should be DB_NONE if possible
Comment Actions
Change 667906 had a related patch set uploaded (by Dduvall; owner: Dduvall):
[mediawiki/core@master] maintenance: Disable storage backend for DB_NONE tasks
Comment Actions
Change 667734 merged by jenkins-bot:
[mediawiki/core@master] maintenance: Skip setAgentAndTriggers for DB_NONE maintenance tasks
Comment Actions
Change 667921 had a related patch set uploaded (by Dduvall; owner: Dduvall):
[mediawiki/core@master] maintenance: mergeMessageFileList should be DB_NONE
Comment Actions
Change 667906 merged by jenkins-bot:
[mediawiki/core@master] maintenance: Disable storage backend for DB_NONE tasks
Comment Actions
Change 667921 merged by jenkins-bot:
[mediawiki/core@master] maintenance: mergeMessageFileList should be DB_NONE
Comment Actions
Change 667735 merged by jenkins-bot:
[mediawiki/core@master] maintenance: rebuildLocalisationCache should be DB_NONE if possible
Comment Actions
Change 668514 had a related patch set uploaded (by Dduvall; owner: Dduvall):
[mediawiki/core@wmf/1.36.0-wmf.33] maintenance: mergeMessageFileList should be DB_NONE
Comment Actions
Change 668516 had a related patch set uploaded (by Dduvall; owner: Dduvall):
[mediawiki/core@wmf/1.36.0-wmf.33] maintenance: rebuildLocalisationCache should be DB_NONE if possible
Comment Actions
Change 669790 had a related patch set uploaded (by Dduvall; owner: Dduvall):
[mediawiki/core@wmf/1.36.0-wmf.33] maintenance: Skip setAgentAndTriggers for DB_NONE maintenance tasks
Comment Actions
Change 667244 merged by jenkins-bot:
[operations/mediawiki-config@master] wmf-config/CommonSettings.php: Add WMF_MAINTENANCE_OFFLINE handling
Comment Actions
Change 669790 merged by jenkins-bot:
[mediawiki/core@wmf/1.36.0-wmf.33] maintenance: Skip setAgentAndTriggers for DB_NONE maintenance tasks
Comment Actions
Change 668514 merged by jenkins-bot:
[mediawiki/core@wmf/1.36.0-wmf.33] maintenance: mergeMessageFileList should be DB_NONE
Comment Actions
Change 668516 merged by jenkins-bot:
[mediawiki/core@wmf/1.36.0-wmf.33] maintenance: rebuildLocalisationCache should be DB_NONE if possible