Page MenuHomePhabricator

l10n generation in git-deploy
Closed, ResolvedPublic

Description

Incorporate the generation and distribution of l10n caches in git-deploy.


Version: unspecified
Severity: enhancement

Details

Reference
bz43614

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 1:19 AM
bzimport added a project: Deployments.
bzimport set Reference to bz43614.

Currently, this is slow, but working in the new system. git-deploy sync in the core repos will cause the cache to rebuild if needed, and all of the salt minions will fetch the appropriate l10n repo. Currently, only English is rebuilt during deployment, and a cron is used to rebuild the other languages daily

A full rebuild takes about 10 mins in eqiad.

In the near future, we will probably try to use something else to only move the parts of the cdb files that have changed.

Technically, all the languages are rebuilt if necessary on deploy. It's just that almost all of the non-English messages will be coming from the LocalisationUpdate extension's cache (which is not being updated on deploy) instead of the deployed code's messages, so they'll still have the old messages.

Gerrit change 42777 should speed up the cron script that rebuilds the LocalisationUpdate cache in many cases, by making it skip extensions that have not changed and by preventing it from invalidating the cache for languages whose messages have not changed. Right now it processes every extension message file, even if the message file has not changed since the previous run, and invalidates every language in the message file, even if none of those translations have actually changed.

Brad tells me this work is done. There still may be some loose ends on cron generation of the l10n caches, but the core task described here is done.