Page MenuHomePhabricator

Automatically update Apertium language pairs
Open, NormalPublic

Description

Apertium language pairs in Content Translation mostly gets outdated quickly, we need solution that quickly updates language pairs:

Proposed solution:

a. Fetch Apertium language pairs data from upstream repositories,
b. Put data into WMF Git repo (probably as submodule),
c. Either build automatic packages from the fetched language pairs data,
OR
c. Put apertium language pairs data directly into 'mode_path' to use in Content Translation.

This should be approved by TechOps to avoid security/other issues.

Event Timeline

Restricted Application added subscribers: Zppix, Aklapper. · View Herald TranscriptJul 5 2016, 6:50 AM

apertium-apy supports custom mode path (default: /usr/share/apertium/modes) so, we can put language pairs data in any directory. This need some careful testing though.

KartikMistry updated the task description. (Show Details)Jul 5 2016, 7:13 AM

First step towards this to make mode path as configurable.

Change 297350 had a related patch set uploaded (by KartikMistry):
WIP: Configurable mode_patch for apertium

https://gerrit.wikimedia.org/r/297350

I have trouble understanding a bit the solution statements.

Are a, b, c, d mutually exlusive ? Because neither of those 4 is strictly a solution on its own. I also have trouble understanding d. What does d entail ?

KartikMistry updated the task description. (Show Details)Jul 7 2016, 1:15 PM

@akosiaris Updated description for clarity.

Amire80 triaged this task as Normal priority.Jul 22 2016, 8:41 PM

I got a different proposal for this. Migrate at some point apertium from the Debian package distribution way we have today to a more conformant with the rest of the services we have way. That is use a git repo that contains apertium and all language dependencies and deploy it with scap3. Pretty similar to the way we deploy cxserver and the other nodejs/python applications we have. It will probably take some time and a migration process but I don't see why it should not be doable. It can even be done in stages, migrating the language pairs first with something similar to the proposed approach - the one with the second (c), and apertium itself later on.

@KartikMistry what do you think ?

Change 297350 abandoned by KartikMistry:
Configurable mode_path for apertium

Reason:
Not suitable at moment.

https://gerrit.wikimedia.org/r/297350