Change 78946 had a related patch set uploaded by Legoktm:
Allow the user to provide a translatable page as the message text

https://gerrit.wikimedia.org/r/78946

Thanks for filing the bug, I had been working on this for a while.

The basic idea I had was you would create "Tech news/subject line" and make it translatable with the text of what the subject line is. You would do the same for "Tech news/body". When you get to the form, you would type the page names in the field instead of text, and it would automagically realize those are translatable pages and pull up the text.

How would that sound?

That sounds great. I'd love to test it. ;)

Anyway, does this handle fallback languages?

(In reply to comment #3)

Anyway, does this handle fallback languages?

At the moment it doesn't. After speaking with Nikerabbit on IRC, it seems we'll have to loop through all the various fallbacks, which I'll work on.

Progress?

(In reply to comment #2)

Thanks for filing the bug, I had been working on this for a while.

The basic idea I had was you would create "Tech news/subject line" and make
it
translatable with the text of what the subject line is. You would do the same
for "Tech news/body". When you get to the form, you would type the page names
in the field instead of text, and it would automagically realize those are
translatable pages and pull up the text.

How would that sound?

How will this handle incomplete translations? E.g. it could post

• all existing translation pages including partial ones (maybe not such a good idea)

*or only those at 100% (there may be problems with bug 47864 though)

*or only those whose status has been set to "published" (analogous to what CentralNotice does)

BTW, regarding "automagically": Notice that the existing solution which relies on {{CONTENTLANG}} doesn't quite deal with the language village pumps on the multilingual projects on Commons and Wikidata (where {{CONTENTLANG}} = en).
Examples are [[commons:Project:Café]] (in [[m:Distribution_list/Global_message_delivery/es]])
or [[d:Project:Warung Kopi]] (in [[m:Global_message_delivery/Targets/Tech_ambassadors]]). It may be too much to demand from this bug to fix the way Commons and Wikidata handle their multilinguality, but one should be aware of this limitation.

(In reply to comment #5)

Progress?

Not much, I had been waiting on most the internal refactoring to get merged before picking this up. That's been done now so I'll take a stab at this again.

(In reply to comment #6)

How will this handle incomplete translations? E.g. it could post

• all existing translation pages including partial ones (maybe not such a

good
idea)

*or only those at 100% (there may be problems with bug 47864 though)

*or only those whose status has been set to "published" (analogous to what
CentralNotice does)

Right now the code just looks for a /langcode subpage and will use it if it exists, regardless of the translation status. I think this can be disadvantageous if there's a situation where there is an empty translation for the specific language, but the next fallback has a complete translation.

I'm not very familiar with how the "published" status works. Do translation admins set that? Is it commonly used?

We could do something like a percentage threshold, so if the status is above 80% (or some other arbitrary number) to use it.

BTW, regarding "automagically": Notice that the existing solution which
relies
on {{CONTENTLANG}} doesn't quite deal with the language village pumps on the
multilingual projects on Commons and Wikidata (where {{CONTENTLANG}} = en).
Examples are [[commons:Project:Café]] (in
[[m:Distribution_list/Global_message_delivery/es]])
or [[d:Project:Warung Kopi]] (in
[[m:Global_message_delivery/Targets/Tech_ambassadors]]). It may be too much
to
demand from this bug to fix the way Commons and Wikidata handle their
multilinguality, but one should be aware of this limitation.

Yeah, I don't think those cases can be easily handled until bug 9360 is solved.

CONTENTLANG has limitations, notably in Commons, Wikidata, and Meta, where additional language codes are used and needed for handling translations in languages (or variants) still not officially supported by MediaWiki, or being discussed before deployment.

Also CONTENTLANG does not work in many namespaces which are already trnslatable (in the Translate extension), because these namespaces theoretically do not have subpages enabled (but subpages are effectively used on translatable pages, even if this does not aplpy to all pages in the same namespace).

On Meta, we use the {{pagelang}} template to avoid these limitations.

But basically, if a page exists in the Translate extension, its /language-code extension should be used as the value returned by {{CONTENTLANGUAGE}}, even in namespaces that don't officially have subpages enabled.

Some poblems will still exist with language codes for which there is still no full support in MediaWiki (such as "sou" used by Southern Thai, which is a valid ISO 639-3 language code, already used in Meta, and which may also be wanted in Commons to store image descriptions, even if there's no full support of Southern Thai in MetaWiki).

Note that adding support for a language in MetaWiki is a necessary step before opening a new lingusitic edition of a project. But it does not mean that there will be a project for it, or that there even exists a Beta project in Incubator for that language. MediaWiki for example can add support for languages supported by sites outside Wikimedia projects (e.g. for Klingon, or English Boontling).

We have policies on Wikiemdia sites for opening or not a project for some language or variant, but translations is another affair and should be more liberal (as long as they use conforming BCP47 codes, including for variants). But these policies do not have to apply to MediaWiki itself, when it is used on other sites.

Another missing thing for MassMessage:

it should expand the wiki code (with all its templates and parser functions) on the source wiki where the message is composed.

Then this fully expanded wiki code could be safely posted to other wikis.

However expansion means that special handling must be done after full expansion to translate the wikilinks, by adding the necessary interwiki prefixes (because we cannot safely post the HTML conversion of the fully expanded wikicode, its URLs would break on every HTML link using the "a" element with href attributes.

Another thing to consider: CSS code that works on the source wiki will not work on the target one. As MessMessage cannot modify the target wiki stylesheets, it would then need to resolve the CSS classes and CSS settings on the source wiki, n order to set the appropriate style="" attributes in the posted wikicode.

The same consideration about non-portable stylesheets will apply to the conversion of that expanded wiki code (augmented with resolved style attributes on wiki elements) will apply before converting it to HTML when posting to emails (however in HTML emails we can insert embedded stylesheets instead of modifying the wikicode prior to the conversion, it will save data volume in the HTML generated for emails sent to lots of users)

Ideally also, MediaWiki should support natural expansion of transcluded pages that are alrady resolved by "Special:MyLanguage/", which should already use the user's language.

So instead of using

{{TNT|Template:template name|uselang={{uselang}}| ... extra params}}

which is asumed to transclude and expand the content of the template in the specified language, we could directly use:

{{Special/MyLanguage/Template:template name| ... extra params}}

without needing any Lua module for doing that. It would transclude and expand exactly the same page as the one linked to by:

[[Special/MyLanguage/Template:template name]]

However for now, MediaWiki cannot transclude any page that is not in a content or talk namespace of the local wiki. It rejects the transclusion of all Special pages. This limitation could be lifted by implementing a white list on allowed special pages, notably for:

Special:MyLanguage/*

but not for:

Special:Preference

{{uselang}} used above in the call to the TNT utility template is; on Meta; a template which is, more or less, what you have from {{int:lang}} except that this {{int:lang}} is not working everywhere and requires importing resources in the target wiki to match ''some'' user language, but not necesarily the effective user language because "Mediawiki:lang/code" may not always return the specified "code" but the code set in another "Mediawiki:lang/*" resource using fallbacks.

The intent of {{uselang}} is to be able to return exactly the prefered user language (set in his user preferences, if that language is supported in MediaWiki, or the preferred used language defined in his browser, xhich could be "de_LI", but that MEdiaWiki could normalize to BCP47, after resolcinv replaced codes from the IANA registry, and forcing it to lowercase using only hyphens and no underscores as "de-li"). Iteally we should not need any template to do that, and Meta wiki should expose {{USELANG}} drectly (it will match the user language, or the language requested in the "?uselang=*" URL query string to view a page.

Mediawiki should accept "?setlang=*" as well by normalizing the specified code to lowercase BCP47, and store it in user's preference, even if that language is not supported in the UI and not proposed in the list of supported languages: MediaWiki will use the normal fallback mechanisms where appropriate to find a supported language for its interface.

But more generally for translations (which hould be more liberal on multilingual sites like Commons, Meta or Incubator), every valid BCP47 code should be accepted, even if MEdiaWiki is not supporting it for now in its interface, or if there's no dedicated linguistic edition of Wikipedia or Wiktionary (or other project localized with specific editions, separate subdomains, and separate databases).

(In reply to comment #9)

Another missing thing for MassMessage:

Could you file any ideas you have as separate bugs to avoid them getting lost and this bug being cluttered?

(In reply to comment #10)

None of that functionality is provided by MassMessage and isn't really on-topic for this bug report. File a bug against Translate instead?

All this functionality is related to MassMessage, because it assumes that everything it posts in other wikis will work in the target one.

The important part is the fact that it does not expand these translations on the source wiki, and it is difficult to ensure that what seems correct on one wiki will be correct once posted to another wiki, or to an email.

The translation issues are inter-related (and this also includes the support of language codes which is effectively different across installed wikis, e.g. the legacy "nrm" code supported in Wikipedia, and is used in fact as a subdomain name, but as an invalid language code, but not supported on Meta as it severaly violates the BCP47 standard where it should be something conforming like "fr-x-norman").

(In reply to comment #12)

All this functionality is related to MassMessage, because it assumes that
everything it posts in other wikis will work in the target one.

Sorry I hadn't read the translators-l post before commenting, thanks for clarifying.

Note that when Massmessage posts to a wiki, it does not need to post the full message to every page of that wiki.

Instead it can create a subpage to the user page used by the bot on the target wiki, and then transclude that subpage to all registered places of that wiki (user pages, community pages...).

This would minimize the size of contents posted to all pages of that wiki.

@Legoktm We added support for delivering translatable pages in T165128: Allow delivery of multilingual messages using translatable pages in fallback languages. Can this task be closed? There is also an associated patch in https://gerrit.wikimedia.org/r/c/mediawiki/extensions/MassMessage/+/78946