Page MenuHomePhabricator
Create Task
Maniphest T352690

Evaluate the integration of the new IndicTrans model (IndicTrans2-M2M) into MinT
Closed, ResolvedPublic
Actions

Description

A new version of the IndicTrans model is available. While the original version integrated in MinT supported the translation of 22 Indic languages to and from English, the new version supports all combinations. This is particularly relevant for combinations not covered by other models such as NLLB-200 for example those involving Santali, Bodo and Dogri, which which were not supported when translating from Hindi, Malayalam or other Indic languages.

This task proposes to explore the new model and evaluate whether it can be incorporated into MinT to improve language support.

__

Initial integration in MinT of the English-to/from-Indic models were supported in T337656: Explore using IndicTrans2 - better model supporting 22 Indic languages

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Update MinT to 2023-12-08-151348-productionoperations/deployment-chartsmaster+1 -1
IndicTrans2 indic-indic model integrationmediawiki/services/machinetranslationmaster+643 -38
Customize query in gerrit

Related Objects

Event Timeline

Pginer-WMF created this task.Dec 4 2023, 4:33 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 4 2023, 4:33 PM
Pginer-WMF triaged this task as Medium priority.Dec 4 2023, 4:33 PM
Pginer-WMF updated the task description. (Show Details)
Pginer-WMF moved this task from Quarter Backlog to Priority: Translation on the Language-Team (Language-2023-October-December) board.Dec 4 2023, 4:46 PM
santhosh claimed this task.Dec 5 2023, 10:46 AM
santhosh merged a task: T352741: Support Indic-Indic translation using IndicTrans2 indic-indic model.
santhosh added a subscriber: KartikMistry.
gerritbot added a comment.Dec 5 2023, 11:24 AM

Change 980356 had a related patch set uploaded (by Santhosh; author: Santhosh):

[mediawiki/services/machinetranslation@master] IndicTrans2 indic-indic model integration

https://gerrit.wikimedia.org/r/980356

gerritbot added a project: Patch-For-Review.Dec 5 2023, 11:24 AM
gerritbot added a comment.Dec 8 2023, 3:13 PM

Change 980356 merged by jenkins-bot:

[mediawiki/services/machinetranslation@master] IndicTrans2 indic-indic model integration

https://gerrit.wikimedia.org/r/980356

Maintenance_bot removed a project: Patch-For-Review.Dec 8 2023, 3:30 PM
KartikMistry moved this task from Priority: Translation to Check after deployment on the Language-Team (Language-2023-October-December) board.Dec 9 2023, 7:19 AM
gerritbot added a comment.Dec 11 2023, 4:51 AM

Change 981709 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update MinT to 2023-12-08-151348-production

https://gerrit.wikimedia.org/r/981709

gerritbot added a project: Patch-For-Review.Dec 11 2023, 4:51 AM
gerritbot added a comment.Dec 11 2023, 6:08 AM

Change 981709 merged by jenkins-bot:

[operations/deployment-charts@master] Update MinT to 2023-12-08-151348-production

https://gerrit.wikimedia.org/r/981709

Maintenance_bot removed a project: Patch-For-Review.Dec 11 2023, 6:10 AM
Pginer-WMF updated the task description. (Show Details)Dec 11 2023, 10:42 AM
Pginer-WMF mentioned this in T337656: Explore using IndicTrans2 - better model supporting 22 Indic languages.
Pginer-WMF added a comment.Dec 11 2023, 10:51 AM

As it can be seen below, IndicTrans2 is now used for language pairs such as Hindi-Kashmiri.
However, when inspecting the target language selector you can notice that Santali (sat) is not listed.
We may need to update MinT configuration to indicate that Santali is supported also for all combinations of the IndicTrans2 supported languages.

Screenshot 2023-12-11 at 11.47.41 2.png (657×1 px, 196 KB)

Pginer-WMF mentioned this in T353136: Set MinT as the default service in Content Translation for languages supported by IndicTrans2.Dec 11 2023, 12:00 PM
santhosh added a comment.Dec 12 2023, 5:23 AM

However, when inspecting the target language selector you can notice that Santali (sat) is not listed.

Santali is present.

image.png (817×952 px, 196 KB)

Nikerabbit changed the task status from Open to In Progress.Jan 8 2024, 7:54 AM
Pginer-WMF edited projects, added Language-Team (Language-2024-January-March); removed Language-Team (Language-2023-October-December).Jan 8 2024, 10:35 AM
Pginer-WMF moved this task from Quarter Backlog to Check after deployment on the Language-Team (Language-2024-January-March) board.
Pginer-WMF closed this task as Resolved.Jan 9 2024, 8:33 AM
Nikerabbit moved this task from Check after deployment to Done on the Language-Team (Language-2024-January-March) board.Jan 22 2024, 7:42 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits