Page MenuHomePhabricator
Create Task
Maniphest T290847

Generate template parameter alignments for languages of interest to Section Translation
Closed, ResolvedPublic
Actions

Description

To improve template support a machine learning approach (T221211) was applied to identify the mappings/alignments of parameters for the most used templates and language pairs (generated alignments). We want to generate additional mappings of Urdu, Kurdish, Igbo, Haitian Creole, Czech, Slovak, Turkish, Xhosa, Zulu, Shona, Luganda, Hausa, Occitan, Yoruba, Bashkir, Azerbaijani, and Malay.

In particular, we want support these additional pairs:

Run 1:

  • en → az
  • en → ba
  • en → cs (Current mapping: 243)
  • en → ha
  • en → ht
  • en → ig
  • en → ku
  • en → lg
  • en → ms
  • en → oc
  • en → sk
  • en → sn
  • en → tr (Current mapping: 387)
  • en → ur (Current mapping: 82)
  • en → xh
  • en → yo
  • en → zu

Run 2:

  • ca → oc
  • cs → sk
  • fr → ht
  • fr → oc
  • ru → ba
  • sk → cs

Run 3:

  • simple → xh

These are languages used on wikis that may be interested in Section Translation. The set includes the target wikis where the Growth team is deploying their recent tools (which may integrate with Section Translation in the future).

Details

SubjectRepoBranchLines +/-
Update cxserver to 2022-05-22-062659-productionoperations/deployment-chartsmaster+1 -1
Added template parameter alignments for simple -> xh pairmediawiki/services/cxservermaster+0 -0
Added template parameter alignments for 14 pairsmediawiki/services/cxservermaster+0 -0
Customize query in gerrit

Related Objects

Event Timeline

Pginer-WMF created this task.Sep 13 2021, 9:55 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 13 2021, 9:55 AM
Pginer-WMF updated the task description. (Show Details)Sep 13 2021, 9:55 AM
Pginer-WMF triaged this task as Medium priority.Sep 13 2021, 10:01 AM
Pginer-WMF renamed this task from Generate template parameter alignments for Urdu, Kurdish, Igbo, Haitian Creole, Kikuyu, Czech, Slovak, and Turkish to Generate template parameter alignments for Urdu, Kurdish, Igbo, Haitian Creole, Kikuyu, Czech, Slovak, Turkish, Xhosa, Zulu, Shona, Luganda, Hausa, and Occitan.Sep 13 2021, 1:57 PM
Pginer-WMF updated the task description. (Show Details)
nshahquinn-wmf subscribed.Sep 13 2021, 4:11 PM
Pginer-WMF edited projects, added Language-Team (Language-2021-October-December); removed Language-Team (Language-2021-July-September).Oct 1 2021, 12:36 PM
Pginer-WMF moved this task from Needs Triage to Enhancements on the ContentTranslation board.Oct 11 2021, 10:48 AM
Pginer-WMF edited projects, added Language-Team (Language-2022-January-March); removed Language-Team (Language-2021-October-December).Dec 3 2021, 12:03 PM
Pginer-WMF renamed this task from Generate template parameter alignments for Urdu, Kurdish, Igbo, Haitian Creole, Kikuyu, Czech, Slovak, Turkish, Xhosa, Zulu, Shona, Luganda, Hausa, and Occitan to Generate template parameter alignments for languages of interest to Section Translation.Mar 29 2022, 10:46 AM
Pginer-WMF updated the task description. (Show Details)
Pginer-WMF edited projects, added Language-Team (Language-2022-April-June); removed Language-Team (Language-2022-January-March).
Pginer-WMF moved this task from Quarter Backlog to Priority: SX Adoption on the Language-Team (Language-2022-April-June) board.Mar 30 2022, 11:54 AM
KartikMistry claimed this task.May 19 2022, 1:59 PM
KartikMistry updated the task description. (Show Details)
KartikMistry updated the task description. (Show Details)
KartikMistry changed the task status from Open to In Progress.May 20 2022, 5:40 AM
KartikMistry moved this task from Priority: SX Adoption to In Progress on the Language-Team (Language-2022-April-June) board.
KartikMistry added a comment.May 20 2022, 12:08 PM

Result of first set:

sqlite> select count(*) from templates where source_lang='en' and target_lang='az';
188
sqlite> select count(*) from templates where source_lang='en' and target_lang='ba';
64
sqlite> select count(*) from templates where source_lang='en' and target_lang='ha';
18
sqlite> select count(*) from templates where source_lang='en' and target_lang='ht';
3
sqlite> select count(*) from templates where source_lang='en' and target_lang='ig';
7
sqlite> select count(*) from templates where source_lang='en' and target_lang='ku';
38
sqlite> select count(*) from templates where source_lang='en' and target_lang='lg';
4
sqlite> select count(*) from templates where source_lang='en' and target_lang='ms';
307
sqlite> select count(*) from templates where source_lang='en' and target_lang='oc';
8
sqlite> select count(*) from templates where source_lang='en' and target_lang='sk';
58
sqlite> select count(*) from templates where source_lang='en' and target_lang='sn';
5
sqlite> select count(*) from templates where source_lang='en' and target_lang='xh';
8
sqlite> select count(*) from templates where source_lang='en' and target_lang='yo';
50
sqlite> select count(*) from templates where source_lang='en' and target_lang='zu';
14
gerritbot added a comment.May 20 2022, 12:22 PM

Change 793768 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[mediawiki/services/cxserver@master] Added template parameter alignments for 14 pairs

https://gerrit.wikimedia.org/r/793768

gerritbot added a project: Patch-For-Review.May 20 2022, 12:22 PM
gerritbot added a comment.May 21 2022, 4:29 AM

Change 793768 merged by jenkins-bot:

[mediawiki/services/cxserver@master] Added template parameter alignments for 14 pairs

https://gerrit.wikimedia.org/r/793768

Maintenance_bot removed a project: Patch-For-Review.May 21 2022, 4:30 AM
KartikMistry updated the task description. (Show Details)May 21 2022, 8:43 AM
KartikMistry added a comment.May 21 2022, 11:51 AM

Along with planned template parameter alignment generation, we've also improved results for some already done pairs as a side result:

  • en > ba: Old: 64, New: 65
  • en > cs: Old: 243, New: 244
  • en > fr: Old: 190, New: 220
  • en > ru: Old: 371, New: 431
  • fr > en: Old: 133, New: 150
  • ru > ca: Old: 137, New: 145
  • ru > en: Old: 298, New: 306
  • ru > fr: Old: 59, New: 71

New pairs:

  • ca > oc: 8
  • cs > sk: 60
  • fr > ht: 3
  • ru > ba: 149
  • sk > cs: 38
KartikMistry updated the task description. (Show Details)May 21 2022, 4:22 PM
gerritbot added a comment.May 21 2022, 4:39 PM

Change 794598 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[mediawiki/services/cxserver@master] Added template parameter alignments for simple -> xh pair

https://gerrit.wikimedia.org/r/794598

gerritbot added a project: Patch-For-Review.May 21 2022, 4:39 PM
gerritbot added a comment.May 22 2022, 6:26 AM

Change 794598 merged by jenkins-bot:

[mediawiki/services/cxserver@master] Added template parameter alignments for simple -> xh pair

https://gerrit.wikimedia.org/r/794598

gerritbot added a comment.May 22 2022, 6:36 AM

Change 794890 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2022-05-22-062659-production

https://gerrit.wikimedia.org/r/794890

KartikMistry updated the task description. (Show Details)May 23 2022, 5:12 AM
KartikMistry moved this task from In Progress to In Review on the Language-Team (Language-2022-April-June) board.
gerritbot added a comment.May 23 2022, 5:14 AM

Change 794890 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2022-05-22-062659-production

https://gerrit.wikimedia.org/r/794890

Stashbot added a comment.May 23 2022, 5:33 AM

Mentioned in SAL (#wikimedia-operations) [2022-05-23T05:33:01Z] <kart_> Updated cxserver to 2022-05-22-062659-production (T290847)

KartikMistry moved this task from In Review to Done on the Language-Team (Language-2022-April-June) board.May 23 2022, 5:33 AM
KartikMistry mentioned this in T304834: Enable Content and Section Translation for Zulu Wikipedia.May 25 2022, 3:50 AM
Pginer-WMF closed this task as Resolved.May 27 2022, 9:10 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL