Page MenuHomePhabricator

MinT has ambiguous language code "gan" as output language, which is used for the mix-script language converter
Open, MediumPublicBUG REPORT

Description

Steps to replicate the issue (include links if applicable):

  • Check the list of supported languages by language code

What happens?:

  • gan is listed as one of the translation target languages

What should have happened instead?:

We should only use explicit gan-hans , gan-hant language codes for translation target language

Event Timeline

I've looked at the code and there are a few things that need to be decided before we can implement a fix. Specifically:

  1. Which NLLB code should Gan Chinese map to? NLLB-200 has no dedicated code for Gan Chinese. The current mapping gan → zho_Hant (standard Traditional Chinese) is clearly incorrect. The realistic options are:
    • Map to yue_Hant (Yue/Cantonese), which is a closer Sinitic relative than standard Chinese
    • Drop gan support entirely until a model with proper Gan Chinese support is available
  2. Should both gan-hans and gan-hant be exposed, or only gan-hant? The Gan Wikipedia predominantly uses Traditional script, so gan-hant alone may be sufficient. That said, adding both symmetrically would be consistent with how other multi-script languages are handled in MinT.
  3. source language support — gan currently only appears as a target language. Should the fix also add gan-hans/gan-hant as source languages?
  4. MADLAD-400 — gan is absent from the MADLAD section of the configuration. Should it be added there as well, or is NLLB the only relevant model here?
  1. Do not mapping Gan as either Mandarin or Cantonese, that's creating more problems (just like what whould happen if mapping Franch language to English language).
  2. Should both gan-hans and gan-hant be exposed? => It should be decided based on "what translation language models we have" instead of just blindly "mapping one to another without proper support". If we are just making a language list without thinking the support status, then we should just list based on the translatewiki's MediaWiki system message translation target languages and remove those disabled instead of listing the language tags that are just a shallow code of language converters.
  3. Still the "support status/strategy" thing.
  4. Still the "support status/strategy" thing.