Page MenuHomePhabricator

[LEX] Wikibase Lexeme should not localize languages names supported by cldr
Closed, ResolvedPublic

Description

Wikibase Lexeme has its own facility to localize language names (which is partly redundant to the localization provided by the cldr extension). This facility should not be used for languages supported by cldr, once these language names are localized there. For languages supported by the cldr extension, see $additionalLocalizedLanguagesin WikibaseLexeme.mediawiki-services.php or for a more up to date list $languageNames in CldrNamesEn.php.

This concerns the localization messages like wikibase-lexeme-language-name-… defined in i18n/language-names/….json and used in LexemeLanguageNameLookup.

Acceptance Criteria

  • Localization messages with keys like wikibase-lexeme-language-name-XXX are imported into the cldr extension (as LocalNames{...}.php entries)
  • Localization messages with keys like wikibase-lexeme-language-name-XXX, where XXX is a language supported by the cldr extension, are removed from i18n/language-names/*.json.
  • $additionalLocalizedLanguages is removed from WikibaseLexeme.mediawiki-services.php

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
mediawiki/extensions/cldrmaster+6 -0
mediawiki/extensions/WikibaseLexememaster+1 -2 K
mediawiki/extensions/WikibaseLexememaster+18 -239
mediawiki/extensions/cldrmaster+57 -0
translatewikimaster+0 -4
mediawiki/extensions/WikibaseLexememaster+3 -193
mediawiki/extensions/cldrmaster+7 -0
mediawiki/extensions/cldrmaster+2 -0
mediawiki/extensions/cldrmaster+38 -0
mediawiki/extensions/cldrmaster+43 -0
mediawiki/extensions/cldrmaster+1 -0
mediawiki/extensions/cldrmaster+19 -0
mediawiki/extensions/cldrmaster+2 -0
mediawiki/extensions/cldrmaster+55 -0
mediawiki/extensions/cldrmaster+6 -0
mediawiki/extensions/cldrmaster+34 -0
mediawiki/extensions/cldrmaster+55 -1
mediawiki/extensions/cldrmaster+56 -0
mediawiki/extensions/cldrmaster+7 -0
mediawiki/extensions/cldrmaster+40 -0
mediawiki/extensions/cldrmaster+37 -0
mediawiki/extensions/cldrmaster+6 -0
mediawiki/extensions/cldrmaster+14 -0
mediawiki/extensions/cldrmaster+6 -0
mediawiki/extensions/cldrmaster+7 -0
mediawiki/extensions/cldrmaster+10 -0
mediawiki/extensions/cldrmaster+16 -0
mediawiki/extensions/cldrmaster+8 -0
mediawiki/extensions/cldrmaster+73 -0
mediawiki/extensions/cldrmaster+67 -0
mediawiki/extensions/cldrmaster+67 -0
mediawiki/extensions/cldrmaster+12 -0
mediawiki/extensions/cldrmaster+8 -0
mediawiki/extensions/cldrmaster+7 -0
mediawiki/extensions/cldrmaster+7 -0
mediawiki/extensions/cldrmaster+65 -0
mediawiki/extensions/cldrmaster+13 -0
mediawiki/extensions/cldrmaster+16 -0
mediawiki/extensions/cldrmaster+7 -0
mediawiki/extensions/cldrmaster+14 -0
mediawiki/extensions/cldrmaster+12 -0
mediawiki/extensions/cldrmaster+61 -0
mediawiki/extensions/cldrmaster+61 -0
mediawiki/extensions/cldrmaster+45 -0
mediawiki/extensions/cldrmaster+59 -0
mediawiki/extensions/cldrmaster+45 -0
mediawiki/extensions/cldrmaster+18 -0
mediawiki/extensions/cldrmaster+20 -0
mediawiki/extensions/cldrmaster+12 -0
mediawiki/extensions/cldrmaster+6 -0
mediawiki/extensions/cldrmaster+55 -0
mediawiki/extensions/cldrmaster+8 -0
mediawiki/extensions/cldrmaster+45 -0
mediawiki/extensions/cldrmaster+53 -0
mediawiki/extensions/cldrmaster+7 -0
mediawiki/extensions/cldrmaster+6 -0
mediawiki/extensions/cldrmaster+36 -0
mediawiki/extensions/cldrmaster+6 -0
mediawiki/extensions/cldrmaster+61 -0
mediawiki/extensions/cldrmaster+37 -0
mediawiki/extensions/cldrmaster+6 -0
mediawiki/extensions/cldrmaster+7 -0
mediawiki/extensions/cldrmaster+11 -0
mediawiki/extensions/cldrmaster+11 -0
mediawiki/extensions/cldrmaster+52 -0
mediawiki/extensions/cldrmaster+67 -0
mediawiki/extensions/cldrmaster+6 -0
mediawiki/extensions/cldrmaster+7 -0
mediawiki/extensions/cldrmaster+18 -0
mediawiki/extensions/cldrmaster+45 -0
mediawiki/extensions/cldrmaster+15 -0
mediawiki/extensions/cldrmaster+43 -0
mediawiki/extensions/cldrmaster+18 -0
mediawiki/extensions/cldrmaster+6 -0
mediawiki/extensions/cldrmaster+10 -0
mediawiki/extensions/cldrmaster+29 -0
mediawiki/extensions/cldrmaster+7 -0
mediawiki/extensions/cldrmaster+20 -0
mediawiki/extensions/cldrmaster+6 -0
mediawiki/extensions/cldrmaster+61 -0
Show related patches Customize query in gerrit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change #1102745 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for ar from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102745

Change #1102746 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for az from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102746

Change #1102747 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for be from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102747

Change #1102748 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for br from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102748

Change #1102749 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for ca from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102749

Change #1102750 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for ce from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102750

Change #1102751 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for cy from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102751

Change #1102752 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for da from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102752

Change #1102753 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for eo from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102753

Change #1102754 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for es from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102754

Change #1102755 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for fi from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102755

Change #1102756 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for hi from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102756

Change #1102757 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for hr from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102757

Change #1102758 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for hu from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102758

Change #1102759 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for hy from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102759

Change #1102769 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for ia from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102769

Change #1102770 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for io from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102770

Change #1102771 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for it from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102771

Change #1102772 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for kcg from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102772

Change #1102773 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for ko from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102773

Change #1102774 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for lb from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102774

Change #1102775 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for lv from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102775

Change #1102776 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for mk from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102776

Change #1102777 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for ms from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102777

Change #1102778 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for my from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102778

Change #1102779 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for nqo from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102779

Change #1102780 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for pl from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102780

Change #1102781 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for pt from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102781

Change #1102782 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for pt-br from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102782

+2ed the second branch as Zuul was still pretty quiet this morning. Now it’s getting more busy so I think the rest will wait for tonight.

Change #1102798 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for se from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102798

Change #1102799 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for sl from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102799

Change #1102800 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for sq from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102800

Change #1102801 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for sw from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102801

Change #1102802 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for tg-cyrl from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102802

Change #1102803 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for tk from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102803

Change #1102804 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for tr from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102804

Change #1102805 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for uk from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102805

Change #1102806 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for vi from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102806

Change #1102807 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for zh-hans from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102807

Change #1102808 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for zh-hant from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102808

Change #1102816 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for acm from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102816

Change #1102817 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for crh-latn from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102817

Change #1102818 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for dag from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102818

Change #1102819 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for diq from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102819

Change #1102820 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for guw from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102820

Change #1102821 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for hno from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102821

Change #1102822 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for hyw from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102822

Change #1102823 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for ks-arab from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102823

Change #1102824 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for mrh from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102824

Change #1102825 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for mui from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102825

Change #1102826 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for nan-hant from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102826

Change #1102827 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for nb from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102827

Change #1102828 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for ojb from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102828

Change #1102829 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for pnb from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102829

Change #1102830 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for roa-tara from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102830

Change #1102831 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for skr-arab from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102831

Change #1102832 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for sr-ec from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102832

Change #1102833 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for tly from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102833

Change #1102834 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for ug-arab from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102834

Change #1102835 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for wuu-hans from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102835

Change #1102295 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for bn from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102295

Change #1102296 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for de from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102296

Change #1102297 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for en from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102297

Change #1102298 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for fa from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102298

Change #1102299 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for fr from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102299

Change #1102300 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for he from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102300

Change #1102301 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for id from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102301

Change #1102302 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for ja from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102302

Change #1102303 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for ms-arab from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102303

Change #1102304 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for nl from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102304

Change #1102305 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for pa from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102305

Change #1102306 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for ru from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102306

Change #1102735 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for sv from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102735

Change #1102736 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for tok from WikibaseLexeme

https://gerrit.wikimedia.org/r/1102736

Change #1105699 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/cldr@master] Add two LocalNames for rah from WikibaseLexeme

https://gerrit.wikimedia.org/r/1105699

I hacked together a shell script to confirm which language-names messages can be removed from WikibaseLexeme:

T352922.sh
#!/usr/bin/env bash

shopt -s lastpipe

declare -A mayremove dontremove

for wbl_file in extensions/WikibaseLexeme/i18n/language-names/*.json; do
    lang1=${wbl_file#extensions/WikibaseLexeme/i18n/language-names/}
    lang1=${lang1%.json}
    if [[ $lang1 = qqq ]]; then
        continue
    fi
    cldr_file_stem=${lang1//-/_}
    cldr_file=extensions/cldr/LocalNames/LocalNames${cldr_file_stem^}.php
    cldr_file2=extensions/cldr/CldrMain/CldrMain${cldr_file_stem^}.php
    jq -r 'keys | .[] | select(startswith("@") | not) | .[30:]' -- "$wbl_file" | while IFS= read -r lang2; do
        mayremove[$lang2]=1
        if ! grep -q -F "'$lang2' =>" -- "$cldr_file" "$cldr_file2" 2>/dev/null; then
            printf >&2 'not removing %s because not found in %s\n' "$lang2" "$cldr_file"
            dontremove[$lang2]=1
        fi
    done
done

for lang in "${!mayremove[@]}"; do
    if ! [[ -n "${dontremove[$lang]}" ]]; then
        printf 'Okay to remove: %s\n' "$lang"
    fi
done

With the above change, it appears that all of them can actually be removed.

Change #1105700 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/WikibaseLexeme@master] Remove $additionalLocalizedLanguages / wikibase-lexeme-language-name-*

https://gerrit.wikimedia.org/r/1105700

Change #1105914 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/cldr@master] Import language names for prg from WikibaseLexeme

https://gerrit.wikimedia.org/r/1105914

Change #1105699 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Add two LocalNames for rah from WikibaseLexeme

https://gerrit.wikimedia.org/r/1105699

Change #1105914 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Import language names for prg from WikibaseLexeme

https://gerrit.wikimedia.org/r/1105914

Change #1105927 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/WikibaseLexeme@master] Remove LexemeLanguageNameLookup

https://gerrit.wikimedia.org/r/1105927

Change #1105700 merged by jenkins-bot:

[mediawiki/extensions/WikibaseLexeme@master] Remove $additionalLocalizedLanguages / wikibase-lexeme-language-name-*

https://gerrit.wikimedia.org/r/1105700

Change #1105929 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[translatewiki@master] [WikibaseLexeme] Remove language names

https://gerrit.wikimedia.org/r/1105929

Change #1105929 merged by jenkins-bot:

[translatewiki@master] [WikibaseLexeme] Remove language names

https://gerrit.wikimedia.org/r/1105929

Change #1105927 merged by jenkins-bot:

[mediawiki/extensions/WikibaseLexeme@master] Remove LexemeLanguageNameLookup

https://gerrit.wikimedia.org/r/1105927

Change #1108397 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/WikibaseLexeme@master] Remove rest of language-names/

https://gerrit.wikimedia.org/r/1108397

Change #1108397 merged by jenkins-bot:

[mediawiki/extensions/WikibaseLexeme@master] Remove rest of language-names/

https://gerrit.wikimedia.org/r/1108397

Lucas_Werkmeister_WMDE assigned this task to ArthurTaylor.

I think we can close this. I can’t think of anything to verify that we didn’t already check during code review, and it seems to work fine in production AFAIK.