Similarly, when I go to https://en.wikipedia.org/wiki/Main_Page and click on the gear icon next to "Languages", I get the "Language settings" window; now when I click on "Input" and "Enable input tools", the list of languages I get under "Language used for writing" is "English, shqip, magyar". This is probably the same underlying issue.

http://www.unicode.org/cldr/charts/latest/supplemental/territory_language_information.html#RS has sr_Latn and sr_Cyrl as first language codes.

Wikidata just takes the top 3 suggestions from ULS. Fixes need to be made there.

So perhaps the bug is that sr_Latn and sr_Cyrl codes are not reduced to just sr?

This could be a case where Wikidata might need to have different behavior from ULS. ULS should offer both Cyrillic and Latin input tools, but Wikidata might not want that.

@Lydia_Pintscher Are you referring to the ULS getFrequentLanguages or getPreviousLanguages method?

We are using getFrequentLanguageList.

OK, now Serbian appears, but with Serbian Cyrillic and Serbian Latin as separate languages. This means that all the Serbian content entered so far, which was entered under the code 'sr', will not be available. Has there ever been a discussion to use two separate codes for Serbian language? Was there a final decision? If yes, what could be done about old content?

It seems like Wikidata doesn't fully support wikis with LanguageConverter enabled? Does this same issue occur for zh-hans and zh-hant (and zh-hk, etc)?

I think that this is fixed now, possibly thanks to https://github.com/wikimedia/jquery.uls/pull/275 .

I tested by setting the countryCode to RS in the debugger.

@Nikola_Smolenski, @Petar.petkovic, can you please test?

Thanks!

In T121747#4006309, @Amire80 wrote:

@Nikola_Smolenski, @Petar.petkovic, can you please test?

I was following this discussion for some time now. I regularly use Wikidata on Chrome and always got following languages (for at least last year):

• English
• Serbian
• Croatian
• Bosnian

Here is how it looks:

All this time I wasn't aware there is actually big issue with Serbian. Only realized now while trying on FF (which I use regularly, but not for wiki editing). When some Wikidata item is opened, I get:

• English
• српски (autonym name for Serbian in Cyrillic script)
• srpski (autonym name for Serbian in Latin script)
• Albanian

Expanding list of languages and searching for "Serbian" shows following items as well:

• Serbian
• Serbian (Cyrillic script)
• Serbian (Latin script)

Short list of languages, initially shown	Expanded list of language

That is actually five entries for Serbian in total, one general and two for Cyrillic and Latin each. I haven't even tried editing those entries with autonym names. Autonym name entries seem to be coming from some external (from Wikidata POV) service and opening list of all languages shows entries from Wikidata, which are populated and make sense.

Would be good to know what language codes are those five things treated internally.

In T121747#4006309, @Amire80 wrote:

I think that this is fixed now, possibly thanks to https://github.com/wikimedia/jquery.uls/pull/275 .

I tested by setting the countryCode to RS in the debugger.

@Nikola_Smolenski, @Petar.petkovic, can you please test?

Thanks!

Mega-facepalm: I tested this without reading the original task description. I assumed that this is about ULS, but it's about Wikidata item pages. Apologies! 🤦

So I suspect that the issue is with Wikidata and not with ULS. But generally, language codes are messy and perhaps using a more common repo of languages and their codes would be good.

There was a workaround in the past to be able to edit in Serbian. If you had sr in the list of your previous ULS languages, you would see it without variants, and be able to edit.
After 76551ed4a7fccbaf87cd850674406ae316f4f956, the workaround is no longer possible. It might be the case that no new additions are now possible in Serbian.

Every Wikidata item looks like this for me, both as anon and as logged user.

Change 532276 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/UniversalLanguageSelector@master] Revert "Return target of redirect languages in mw.uls.getFrequentLanguageList"

https://gerrit.wikimedia.org/r/532276

Change 532276 merged by jenkins-bot:
[mediawiki/extensions/UniversalLanguageSelector@master] Revert "Return target of redirect languages in mw.uls.getFrequentLanguageList"

https://gerrit.wikimedia.org/r/532276

Change 532341 had a related patch set uploaded (by Alaa Sarhan; owner: Ladsgroup):
[mediawiki/extensions/UniversalLanguageSelector@wmf/1.34.0-wmf.19] Revert "Return target of redirect languages in mw.uls.getFrequentLanguageList"

https://gerrit.wikimedia.org/r/532341

In T121747#4009298, @Nikerabbit wrote:

Would be good to know what language codes are those five things treated internally.

To reiterate: does Wikibase document somewhere what kind of language codes it expects to receive and use for languages and language variants? This bug is a failure of normalisation somewhere.

Change 532341 merged by jenkins-bot:
[mediawiki/extensions/UniversalLanguageSelector@wmf/1.34.0-wmf.19] Revert "Return target of redirect languages in mw.uls.getFrequentLanguageList"

https://gerrit.wikimedia.org/r/532341

Mentioned in SAL (#wikimedia-operations) [2019-08-26T11:34:25Z] <ladsgroup@deploy1001> Synchronized php-1.34.0-wmf.19/extensions/UniversalLanguageSelector: SWAT: [[gerrit:532341|Revert "Return target of redirect languages in mw.uls.getFrequentLanguageList" (T217770 T121747)]] (duration: 00m 46s)