Page MenuHomePhabricator
Create Task
Maniphest T308248

Newly supported languages in Google Translate
Closed, ResolvedPublic
Actions

Description

Recently Google Translate added machine translate supports in 24 new languages, 21 of which have a Wikipedia project. This should be enabled in ContentTranslation:

  • Assamese (as)
  • Aymara (ay)
  • Bhojpuri (bho)
  • Ewe (ee)
  • Guarani (gn)
  • Sorani (ckb)
  • Lingala (ln)
  • Luganda (lg)
  • Maithili (mai)
  • Maldivian (dv)
  • Northern Sotho (nso)
  • Oromo (om)
  • Quechua (qu)
  • Sanskrit (sa)
  • Tigrinya (ti)
  • Tsonga (ts)
  • Twi (tw) (Google API seems to be using "ak" code)

Not working:

  • Bambara (bm)
  • Iloko (ilo)
  • Konkani (gom)
  • Meitei (mni)

Further work for the languages with issues will be done as part of T314557: Enable Content and Section translation on wikipedias with new MT support from Google for languages once it is working

Details

SubjectRepoBranchLines +/-
Update cxserver to 2022-08-22-093815-productionoperations/deployment-chartsmaster+1 -1
Google: Removed unused mapping for Goan Konkani (gom) languagemediawiki/services/cxservermaster+0 -1
Update cxserver to 2022-08-04-022612-productionoperations/deployment-chartsmaster+1 -1
MT: Add mapping for mni for Googlemediawiki/services/cxservermaster+1 -0
MT: Map Google MT language code with Wikipediamediawiki/services/cxservermaster+9 -6
Update cxserver to 2022-07-27-220330-productionoperations/deployment-chartsmaster+1 -1
Add newly supported languages in Google Translatemediawiki/services/cxservermaster+21 -0
Customize query in gerrit

Related Objects

Event Timeline

Bugreporter created this task.May 12 2022, 2:25 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 12 2022, 2:25 PM
Bugreporter updated the task description. (Show Details)May 12 2022, 2:27 PM
Amire80 subscribed.May 16 2022, 7:29 AM
Pginer-WMF triaged this task as Medium priority.May 27 2022, 8:32 AM
Pginer-WMF moved this task from Needs Triage to MT on the ContentTranslation board.
Pginer-WMF mentioned this in T310773: Re-run and update the MT service usage report.Jun 16 2022, 9:21 AM
Pginer-WMF assigned this task to KartikMistry.Jul 14 2022, 10:13 AM
Pginer-WMF added a project: Language-Team (Language-2022-July-September).
Pginer-WMF moved this task from Quarter Backlog to Priority: Unified Content & Section Translation on the Language-Team (Language-2022-July-September) board.
Pginer-WMF mentioned this in T313296: Enable Content and Section translation on wikipedias with new MT support from Google.Jul 19 2022, 10:24 AM
Pginer-WMF subscribed.EditedJul 19 2022, 12:13 PM

Checking the Google API it seems the newly listed languages are already available. I also noticed that Akan was also supported but was not enabled yet in Content Translation and it may be the code used for Twi. As well as others for which there is no Wikipedia yet (Hmong, and Krio/Sierra Leonean Creole)

Pginer-WMF updated the task description. (Show Details)Jul 19 2022, 12:14 PM
Pginer-WMF updated the task description. (Show Details)
Pginer-WMF updated the task description. (Show Details)Jul 19 2022, 12:20 PM
Pginer-WMF updated the task description. (Show Details)
Pginer-WMF updated the task description. (Show Details)
Pginer-WMF updated the task description. (Show Details)Jul 19 2022, 12:27 PM
Pginer-WMF raised the priority of this task from Medium to High.Jul 27 2022, 8:22 AM
gerritbot added a comment.Jul 27 2022, 9:34 AM

Change 817734 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[mediawiki/services/cxserver@master] Add newly supported languages in Google Translate

https://gerrit.wikimedia.org/r/817734

gerritbot added a project: Patch-For-Review.Jul 27 2022, 9:34 AM
KartikMistry moved this task from Priority: Unified Content & Section Translation to In Review on the Language-Team (Language-2022-July-September) board.Jul 27 2022, 9:34 AM
gerritbot added a comment.Jul 27 2022, 6:51 PM

Change 817734 merged by jenkins-bot:

[mediawiki/services/cxserver@master] Add newly supported languages in Google Translate

https://gerrit.wikimedia.org/r/817734

Maintenance_bot removed a project: Patch-For-Review.Jul 27 2022, 7:30 PM
gerritbot added a comment.Jul 28 2022, 6:01 AM

Change 817931 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2022-07-27-220330-production

https://gerrit.wikimedia.org/r/817931

gerritbot added a project: Patch-For-Review.Jul 28 2022, 6:01 AM
gerritbot added a comment.Jul 28 2022, 8:42 AM

Change 817931 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2022-07-27-220330-production

https://gerrit.wikimedia.org/r/817931

Stashbot added a comment.Jul 28 2022, 8:57 AM

Mentioned in SAL (#wikimedia-operations) [2022-07-28T08:57:31Z] <kart_> Updated cxserver to 2022-07-27-220330-production (T308248)

KartikMistry moved this task from In Review to Done on the Language-Team (Language-2022-July-September) board.Jul 28 2022, 8:59 AM
Maintenance_bot removed a project: Patch-For-Review.Jul 28 2022, 9:30 AM
gerritbot added a comment.Jul 28 2022, 11:06 AM

Change 818087 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[mediawiki/services/cxserver@master] MT: Map Google MT language code with Wikipedia

https://gerrit.wikimedia.org/r/818087

gerritbot added a project: Patch-For-Review.Jul 28 2022, 11:06 AM
KartikMistry moved this task from Done to In Review on the Language-Team (Language-2022-July-September) board.Jul 28 2022, 1:23 PM
gerritbot added a comment.Aug 1 2022, 11:32 AM

Change 818087 merged by jenkins-bot:

[mediawiki/services/cxserver@master] MT: Map Google MT language code with Wikipedia

https://gerrit.wikimedia.org/r/818087

gerritbot added a comment.Aug 1 2022, 12:18 PM

Change 819045 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[mediawiki/services/cxserver@master] MT: Add mapping for mni for Google

https://gerrit.wikimedia.org/r/819045

gerritbot added a comment.Aug 3 2022, 8:23 AM

Change 819045 merged by jenkins-bot:

[mediawiki/services/cxserver@master] MT: Add mapping for mni for Google

https://gerrit.wikimedia.org/r/819045

Maintenance_bot removed a project: Patch-For-Review.Aug 3 2022, 8:30 AM
gerritbot added a comment.Aug 3 2022, 8:35 AM

Change 820075 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2022-08-03-082610-production

https://gerrit.wikimedia.org/r/820075

gerritbot added a project: Patch-For-Review.Aug 3 2022, 8:35 AM
gerritbot added a comment.Aug 4 2022, 5:36 AM

Change 820075 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2022-08-04-022612-production

https://gerrit.wikimedia.org/r/820075

Stashbot added a comment.Aug 4 2022, 5:49 AM

Mentioned in SAL (#wikimedia-operations) [2022-08-04T05:49:40Z] <kart_> Updated cxserver to 2022-08-04-022612-production (T313296, T308248)

KartikMistry added a comment.Aug 4 2022, 5:55 AM

@Pginer-WMF The following languages seem not working with Google MT.

  1. Bambara (bm)

Error:

"Translation with Google failed for en > bm. Error: 503 : The service is unavailable at this time."
  1. Manipuri Meitei (mni) (Google MT code: mni-Mtei)

Error:

"Translation with Google failed for en > mni-Mtei. Error: 503 : The service is unavailable at this time."
  1. Goan Konkani (gom)

Error:

"Translation with Google failed for en > kon. Error: 400 : Invalid Value"
KartikMistry added a comment.Aug 4 2022, 6:21 AM
In T308248#8130240, @KartikMistry wrote:
  1. Goan Konkani (gom)

Error:

"Translation with Google failed for en > kon. Error: 400 : Invalid Value"

It seems Google is using 'gom' code only, but with that too,

"Translation with Google failed for en > gom. Error: 503 : The service is unavailable at this time."
Pginer-WMF mentioned this in T314557: Enable Content and Section translation on wikipedias with new MT support from Google for languages once it is working.Aug 4 2022, 7:35 AM
Pginer-WMF added a comment.Aug 4 2022, 7:44 AM

I tested Iloko (ilo) and it seems not to be working. Note that testing Iloko on the tool may require to switch the MT service to Google in the drop-down since it is not the current default.

DraconicDark subscribed.Aug 18 2022, 10:19 PM
In T308248#8130250, @KartikMistry wrote:
In T308248#8130240, @KartikMistry wrote:
  1. Goan Konkani (gom)

Error:

"Translation with Google failed for en > kon. Error: 400 : Invalid Value"

It seems Google is using 'gom' code only, but with that too,

"Translation with Google failed for en > gom. Error: 503 : The service is unavailable at this time."

Should note here that the ISO 639-3 code for Konkani is kok, not kon (kon is the 3-letter code for Kongo, which has two-letter code kg). In any case, gom is the code that Konkani Wikipedia uses, so Google Translate only using gom shouldn't be an issue.

gerritbot added a comment.Aug 22 2022, 6:24 AM

Change 825200 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[mediawiki/services/cxserver@master] Google: Removed unused mapping for Goan Konkani (gom) language

https://gerrit.wikimedia.org/r/825200

KartikMistry moved this task from In Review to In Progress on the Language-Team (Language-2022-July-September) board.Aug 22 2022, 6:25 AM
gerritbot added a comment.Aug 22 2022, 8:28 AM

Change 825200 merged by jenkins-bot:

[mediawiki/services/cxserver@master] Google: Removed unused mapping for Goan Konkani (gom) language

https://gerrit.wikimedia.org/r/825200

gerritbot added a comment.Aug 22 2022, 12:07 PM

Change 825336 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2022-08-22-093815-production

https://gerrit.wikimedia.org/r/825336

gerritbot added a comment.Aug 23 2022, 6:37 AM

Change 825336 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2022-08-22-093815-production

https://gerrit.wikimedia.org/r/825336

Stashbot mentioned this in T308371: Migrate node-based services in production to node16.Aug 23 2022, 6:50 AM

Mentioned in SAL (#wikimedia-operations) [2022-08-23T06:50:31Z] <kart_> Updated cxserver to 2022-08-22-093815-production (T308248, T308371)

Pginer-WMF mentioned this in T316617: Translation suggestion loading forever when a machine translation service fails.Aug 30 2022, 8:49 AM
KartikMistry updated the task description. (Show Details)Sep 5 2022, 5:50 AM
KartikMistry updated the task description. (Show Details)
KartikMistry updated the task description. (Show Details)Sep 5 2022, 5:53 AM
KartikMistry updated the task description. (Show Details)Sep 5 2022, 5:56 AM

We still have Bambara (bm), Iloko (ilo), Konkani (gom) and, Meitei (mni) not working despite using the same code as the Google Translation API service.

Pginer-WMF updated the task description. (Show Details)Sep 5 2022, 10:54 AM
Pginer-WMF added a comment.Sep 5 2022, 10:58 AM

I added a note for the languages not working. We may want to close this ticket and continue the work on T314557: Enable Content and Section translation on wikipedias with new MT support from Google for languages once it is working

KartikMistry closed this task as Resolved.Sep 5 2022, 11:42 AM
In T308248#8211224, @Pginer-WMF wrote:

I added a note for the languages not working. We may want to close this ticket and continue the work on T314557: Enable Content and Section translation on wikipedias with new MT support from Google for languages once it is working

Thanks a lot, Pau!

KartikMistry moved this task from In Progress to Done on the Language-Team (Language-2022-July-September) board.Sep 5 2022, 11:43 AM
• Nikerabbit removed a project: Patch-For-Review.Sep 7 2022, 11:08 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits