Page MenuHomePhabricator
Create Task
Maniphest T340236

MinT translates to English when Hindi-Santali or any other language-Santali is selected
Closed, ResolvedPublicBUG REPORT
Actions

Description

It was reported that when you start a translation from Hindi or Bengali to Santali, the Machine translation is in English instead of Santali.

Steps to replicate the issue (include links if applicable):

  • Select any language article except English and translate to Santali (ᱥᱟᱱᱛᱟᱲᱤ).
  • Select MinT.
  • The translation will be in English instead of Santali (Ol Chiki).

What happens?:
The translation will be in English instead of Santali (Ol Chiki).

What should have happened instead?:
It should be translated to Santali (Ol Chiki).

Details

SubjectRepoBranchLines +/-
Update MinT to 2023-06-27-053706-productionoperations/deployment-chartsmaster+1 -1
Update cxserver to 2023-06-26-050753-productionoperations/deployment-chartsmaster+1 -1
Remove Santali from NLLB base model as it need wrong script codemediawiki/services/machinetranslationmaster+0 -1
MinT: Reconfigure santali as en<->santali onlymediawiki/services/cxservermaster+3 -1
Customize query in gerrit

Related Objects

Event Timeline

UOzurumba created this task.Jun 23 2023, 4:42 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 23 2023, 4:42 PM
UOzurumba updated the task description. (Show Details)Jun 23 2023, 4:59 PM
gerritbot added a comment.Jun 26 2023, 3:57 AM

Change 932681 had a related patch set uploaded (by Santhosh; author: Santhosh):

[mediawiki/services/machinetranslation@master] Remove Santali from NLLB base model as it need wrong script code

https://gerrit.wikimedia.org/r/932681

gerritbot added a project: Patch-For-Review.Jun 26 2023, 3:57 AM
gerritbot added a comment.Jun 26 2023, 4:05 AM

Change 932682 had a related patch set uploaded (by Santhosh; author: Santhosh):

[mediawiki/services/cxserver@master] MinT: Reconfigure santali as en<->santali only

https://gerrit.wikimedia.org/r/932682

• santhosh claimed this task.Jun 26 2023, 4:06 AM
• santhosh triaged this task as Medium priority.
• santhosh moved this task from Needs Triage to MT on the ContentTranslation board.
• santhosh added a project: Language-Team (Language-2023-April-June).
• santhosh moved this task from Quarter Backlog to In Review on the Language-Team (Language-2023-April-June) board.
gerritbot added a comment.Jun 26 2023, 5:07 AM

Change 932682 merged by jenkins-bot:

[mediawiki/services/cxserver@master] MinT: Reconfigure santali as en<->santali only

https://gerrit.wikimedia.org/r/932682

gerritbot added a comment.Jun 26 2023, 5:07 AM

Change 932681 merged by jenkins-bot:

[mediawiki/services/machinetranslation@master] Remove Santali from NLLB base model as it need wrong script code

https://gerrit.wikimedia.org/r/932681

Maintenance_bot removed a project: Patch-For-Review.Jun 26 2023, 5:10 AM
gerritbot added a comment.Jun 26 2023, 5:13 AM

Change 932683 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2023-06-26-050753-production

https://gerrit.wikimedia.org/r/932683

gerritbot added a project: Patch-For-Review.Jun 26 2023, 5:13 AM
gerritbot added a comment.Jun 26 2023, 6:09 AM

Change 932683 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2023-06-26-050753-production

https://gerrit.wikimedia.org/r/932683

Maintenance_bot removed a project: Patch-For-Review.Jun 26 2023, 6:10 AM
Stashbot mentioned this in T339896: Enable MinT for all languages supported by IndicTrans2.Jun 26 2023, 6:28 AM

Mentioned in SAL (#wikimedia-operations) [2023-06-26T06:28:05Z] <kart_> Updated cxserver to 2023-06-26-050753-production (T340236, T339896)

KartikMistry added a comment.Jun 26 2023, 6:30 AM

@UOzurumba We have updated the config for Santhali. Currently, only English -> Santhali translation is supported by MinT.

UOzurumba added a comment.Jun 26 2023, 12:08 PM
In T340236#8962978, @KartikMistry wrote:

@UOzurumba We have updated the config for Santhali. Currently, only English -> Santhali translation is supported by MinT.

Thank @KartikMistry for the information I will let the community know.

gerritbot added a comment.Jun 27 2023, 5:59 AM

Change 933221 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update MinT to 2023-06-27-053706-production

https://gerrit.wikimedia.org/r/933221

gerritbot added a project: Patch-For-Review.Jun 27 2023, 5:59 AM
gerritbot added a comment.Jun 27 2023, 8:41 AM

Change 933221 merged by jenkins-bot:

[operations/deployment-charts@master] Update MinT to 2023-06-27-053706-production

https://gerrit.wikimedia.org/r/933221

Maintenance_bot removed a project: Patch-For-Review.Jun 27 2023, 9:10 AM
Stashbot added a comment.Jun 27 2023, 9:11 AM

Mentioned in SAL (#wikimedia-operations) [2023-06-27T09:11:33Z] <kart_> Updated MinT to 2023-06-27-053706-production (T339896, T340236)

KartikMistry moved this task from In Review to Done on the Language-Team (Language-2023-April-June) board.Jun 27 2023, 12:51 PM
Pginer-WMF added a comment.Jun 28 2023, 7:39 PM
In T340236#8962978, @KartikMistry wrote:

@UOzurumba We have updated the config for Santhali. Currently, only English -> Santhali translation is supported by MinT.

The current status is that only en-sat language pair is supported.
My understanding, is that in order to support other pairs, we need to translate the code used for Santali into the one expected by NLLB-200. Should this be part of the current ticket or a separate follow-up ticket is preferred?

• santhosh added a comment.Jun 29 2023, 4:15 AM

My understanding, is that in order to support other pairs, we need to translate the code used for Santali into the one expected by NLLB-200. Should this be part of the current ticket or a separate follow-up ticket is preferred?

Translating between a non-english language and santali is supported by NLLB200 using sat_deva code(Wrong because it is not devanagari). But IndicTrans2 uses larger and better corpus for Santali and offeres en->sat or sat->en. I wont recommened using NLLB200 for other combinations because that require complicating the configuration to use two different language codes for two models. From my conversations to IndicTrans2 team, they are working on Indic->Indic translations.

Pginer-WMF closed this task as Resolved.Jun 29 2023, 8:19 AM
In T340236#8975147, @santhosh wrote:

My understanding, is that in order to support other pairs, we need to translate the code used for Santali into the one expected by NLLB-200. Should this be part of the current ticket or a separate follow-up ticket is preferred?

Translating between a non-english language and santali is supported by NLLB200 using sat_deva code(Wrong because it is not devanagari). But IndicTrans2 uses larger and better corpus for Santali and offeres en->sat or sat->en. I wont recommened using NLLB200 for other combinations because that require complicating the configuration to use two different language codes for two models. From my conversations to IndicTrans2 team, they are working on Indic->Indic translations.

Thanks for the context. I think it makes sense to wait, and check if the next developments of IndicTrans2 covers the needs of the community. At that point, if there are some needs not covered (e.g., translations from French to Santali being common) we can reconsider how to expose NLLB-200, but I'd expect most translations to be from English or other Indic languages. So resolving this for now.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL