this is already done for the content language (only at the moment) if it is a variant language, with mw.wikibase.getEntity.

https://gerrit.wikimedia.org/r/#/c/178422/ fixes this for mw.wikibase.label convenience method for variant languages (this worked before).

as with the parser function, i don't know what kind of fallback we could further apply (user specific stuff is out of the question, due to caching). For mw.wikibase.getEntityObject (or whatever else provides full access to the entire entity data), all the language code => label (or description) could have fallback applied by default for the value.we might / probably want to provide additional information in the serialization structure that says the source language is and for a given target language (e.g. zh-tw) it might have multiple possible fallbacks (one being preferred, but others also valid part of the chain).

For mw.wikibase.getEntityObject (or whatever else provides full access to the entire entity data), maybe we also want to provide an option or method where fallback is not applied and the user gets the explicit labels / descriptions that are actually set for the languages.

I looked further into how fallback is working in lua and find that content language is always preferred (if label present) over any variants regardless if variants also have labels and variants are requested.

e.g. when requesting client page with variant=ku-arab (ku-arab + ku labels on the connected item), I get the ku label that has been transliterated when using mw.wikibase.label.

If I use variant=ku, have no ku label but have ku-arab, then I get ku-arab label + transliteration when using mw.wikibase.label.

with the way parser cache works and given the comments in Scribunto_LuaWikibaseLibrary, the way this all works might be a necessity and what we want, unless we want to further split parser cache by variant language.

// For the language we need $wgContLang, not parser target language or anything else.          
// See Scribunto_LuaLanguageLibrary::getContLangCode().

to clarify, I have to request variant=ku-latn to get the arabic -> latin transliteration. ku shows mixed scripts. (no conversion)

looks like lua and parser function handle preferred fallback / variant language differently when there are labels in all or multiple variants.

With the following, if the referenced item has labels in both ku and ku-arab and I use variants=ku-arab:

{{#property:P2}}

{{#invoke:Wikidata|formatStatements|property=P2}}

I get:

• ku-arab label for parser function

• transliterated ku label for lua.

https://gist.github.com/filbertkm/896ebff99c03f1744375 (my Wikidata module, sourced from ruwiki lua module)

also looks like the parser cache is maintaining the variant label differences in the client page for ku-arab vs. ku-latn vs. ku, as there is some split.

Confirmed that this is still an issue.

This is apparently still an issue

I just investigated this a little:

We only use LanguageFallbackChainFactory::FALLBACK_SELF | LanguageFallbackChainFactory::FALLBACK_VARIANTS in all places, except for mw.wikibase.getEntity():formatPropertyValues( … ) where we use the default (the default is fallback to all fallback languages) (by omitting the field in the FormatterOptions passed to it in Scribunto_LuaWikibaseEntityLibrary::newImplementation). I presume this happened by accident.

Next steps:

• Use LanguageFallbackChainFactory::FALLBACK_ALL for everything (in Lua/ the parser function), but create a setting to disable that.
• Switch Scribunto_LuaWikibaseEntityLibrary::newImplementation to not use the default, but explicitly supply the fallback chain to use.

Change 291502 had a related patch set uploaded (by Hoo man):
Use FALLBACK_ALL for all data access functionality

https://gerrit.wikimedia.org/r/291502

Change 291502 merged by jenkins-bot:
Use FALLBACK_ALL for all data access functionality

https://gerrit.wikimedia.org/r/291502