Page MenuHomePhabricator

Support for language variants and multiple scripts
Open, Needs TriagePublic

Description

Description

Currently some Wikimedia projects support language variants or multiple scripts. My understanding is that this support is not natively part of MediaWiki/core, but rather has been added in a non-standardized way by the projects themselves(?) Based on this list (link) it seems like currently variants and multiple scripts are only used on a small percentage of Wikimedia projects (at least 9, maybe as many as 20 — link to spreadsheet), however with the upcoming focus on new communities perhaps that percentage would grow(?)

Examples
VariantsMultiple scripts
Serbian (example page)Konkani (example page)
Kurdish (example page)Kashmiri (example page)
Open questions
  1. The updated ULS (link to task) will support language variants. Would the same solution be appropriate for multiple scripts?
  2. Do people switch between variants and/or scripts more frequently than they switch between languages? If so, should we investigate adding an additional button/menu/toggle to allow for quicker switching? E.g.
    image.png (456×1 px, 216 KB)

Related Objects

StatusSubtypeAssignedTask
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
StalledNone
StalledNone
InvalidNone
StalledNone
StalledNone
StalledNone
StalledNone
StalledFeatureNone
StalledNone
StalledFeatureNone
StalledFeatureNone
StalledFeatureNone
StalledNone
StalledNone
OpenNone
OpenNone
OpenNone
StalledNone
OpenNone
OpenNone
OpenNone
OpenNone
DeclinedNone
OpenNone
OpenNone

Event Timeline

Generally, it makes a lot of sense to me to integrate language variant selection with other language settings, and especially interlanguage links. I've already mentioned it as an idea back when I made a presentation about Compact Languages Links back in 2018; check especially the parts about "ideas for the future" at the 15:55 mark.

However, and this is important, you really have to consult about this with the people who read and write in these languages. I can read a tiny bit of Serbian because I know Russian, but I'm really not an authority on it, and I don't write in the Serbian Wikipedia, and I don't know any of the other languages that have variants. So please, involve readers and editors in Serbian, Chinese, Uzbek, and other languages with variants in your design research. Some ideas of things to ask them when doing this research:

  • Editors:
    • How do you decide whether to write something in one script or the other?
    • Which reasons do you have for choosing another script?
  • Readers (and editors):
    • Which search engine do you usually use (Google, Bing, Yandex, Baidu, etc.)? How well can it find articles in the script that is more comfortable for you to read?
    • Does your (external) search engine usually find articles in the script you want automatically, without forcing you to choose manually?
    • Do you often have to switch the script manually? For what purpose?

Language variants with a converter, as it is with Serbian, Kurdish, and Chinese, are part of MediaWiki core, and it's supposed to be relatively easy to update the design of their selection in a uniform way. The iOS Wikipedia app designers worked on it recently, and perhaps they have ideas for you.

Languages with "multiple scripts" like Konkani and Kashmiri are harder. This is not part of core. In Kashmiri, Konkani, Fula, and some other languages, it's currently implemented in a totally manual way with templates and unsynchronized alternate pages. In the Portuguese Wikipedia there is something functionally similar to the core language converter to show articles in different orthographic variants (Brazil, Portugal, Angola, etc.), but it's implemented as a local gadget, so it's also difficult to implement a uniform design for it. Ideally, all these languages should be switched to the language converter, or even split to two wikis, as it has always been with Punjabi (pa/pnb), and as it was done with Azerbaijani, which used to host articles in Latin script and in Arabic script, and was later split to two wikis (az/azb). So unfortunately I'm not sure how can it be addressed properly at the moment through organized WMF MediaWiki design and development activities. Perhaps I can try to improve this as part of my volunteering with the Language committee, but I really cannot promise anything.

  1. The updated ULS (link to task) will support language variants. Would the same solution be appropriate for multiple scripts?

The design for variants on the new language selector was created as a way to support grouping some of the languages. It leaves to the specific application the decision on which options to group and what to do when the user selects them (the group item or its sub-items). The reason for this is that different types of variants may be supported in different contexts and in different ways. For example, British and US English are only distinguished on UI language selection but not for Wikipedia content.

So, I think that the design can accommodate groups where the options are the same language in different scripts.

language-groups.png (350×846 px, 39 KB)

  1. Do people switch between variants and/or scripts more frequently than they switch between languages? If so, should we investigate adding an additional button/menu/toggle to allow for quicker switching?

I shared the assumption that there may be a need to quickly switch between variants (which is something to check through research). In addition to being related, I expected that selecting the grouping element (e.g., English instead of the sub-items "British English" or "US English") implies selecting the default variant which may not be always the intended one, so a way to correct the decision quickly may be convenient.

The design proposal was to surface variants at the top of the language selector after users selected a language with variants. In this way, the user only has to open the language selector and the alternative variants will be right there:

top-section-variants.png (558×421 px, 36 KB)

Surfacing them directly to the top level of the UI next to the language selector may be another option. However, my expectation is not for a behaviour of continuous switching between them. So that may be too prominent.

VulpesVulpes825 subscribed.

My understanding is that this support is not natively part of MediaWiki/core, but rather has been added in a non-standardized way by the projects themselves(?)

No, Language Converter is part of the MediaWiki/core (under includes/language/LanguageConverter.php), thus adding MediaWiki-Language-converter and Chinese-Sites for tracking.