In T223888#5361311, @Jpita wrote:

@santhosh anything I can check here?

Screenshot after the above patch - from en:Grüneisen_parameter

Here is an example where the MT calcualtion does not happen. From es.wiki

Screenshot based on the above patch

In T227571#5397881, @Jpita wrote:

@santhosh where can I find this api playground/documentation page?
Can you provide some info on how I can test this ?

Thanks. That works

Revert patch for T220495: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ContentTranslation/+/528406

I had to revert this fix because of T229906: Sentence pair highlighting broken
Revert patch: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ContentTranslation/+/528406

Assuming Apertium was the MT engine used.

We cannot remove the wrapping tags for math since that would cause unexpected rendering and editing behavior. Better keep the DOM as such. I propose excluding <dl> tags from validation. Math is under dl ->dd -> p tag.

The feature does not differentiate between wikis. I suspect caching related inconsistencies.

It seems there are two issues here. A gray reference link is unadapted reference. This ticket made the reference code show only the edit button for such references.

In T214256#5376632, @Jpita wrote:

This one is definitely a bug. Happening with Apertium. Checking.

@santhosh are you going to be fixing this issue on this task or a different one?

In T214256#5374430, @Jpita wrote:

@Pginer-WMF is this a different issue?
Is there a ticket already for this?

In T214256#5373818, @Jpita wrote:

update: this might not be related since the example on my last screenshot is a template and not a math formula

In T214256#5373782, @Jpita wrote:

not sure this is related to this ticket, but I never saw a math formula being translated like this.
it is like this in production as well.
waiting for @santhosh feedback to see if it is related or not.

Thanks @Krinkle. The usecase is not a widely used one to justify an effort to implement this feature. At least for now. For now, I used a workaround as https://gerrit.wikimedia.org/r/525521

yes, I think the math support need to improve and that can fix both. First thing to do is avoid sending math sections to MT engines so that they don't mangle it.

Found that if I used Yandex instead of Google, the above section translated correctly

I was not able to reproduce the case of https://en.wikipedia.org/wiki/Chronology_of_Gabon from en to es. But I was able to reproduce with https://en.wikipedia.org/wiki/Proof_that_%CF%80_is_irrational en to pt.

So the correct thing to do here is to implement this special mapping in LanguageTr::ucfirst().

When we say language in the context of wiki, there is a"interface language" - the language of user interface and then content language- that is the content language of wiki. So if you are in germal wikipedia, the content language is german. But the interface language can be anything(for example English). If interface language is English, the items in sidebar, top menu etc will be in English.

@Pginer-WMF, to clarify, I was writing down, the root cause, not the expected behavior. I have fixed the issue in the patch linked.

Analysis:
The reference content is a template:

This one uses localstorage.

@Krinkle Need an advice from you on this.
We have a requirement to show some UI messages in a language that is not wgUserLanguage. That may be surprising, but the context is Google translated page as shown in above screenshots. Eventhough the wgUserLanguage is translation source language, the real user language here is translation target language. So if user is reading an English wikipedia page, translated to Hindi, the messages we show should be in Hindi, even if wgUserLanguage=='en'.

ContentTranslation already uses every information found in templatedata for source and target language pairs. The parameters are matched in after a normalization(removing punctuations, spaces, making it case insetive), using the aliases defined etc. But none of these can substitute a "canonical parameter name or id" that will connect between parameters in language with another language. Wikidata Id could be an additional item in templatedata that help us to do this connections. It won't be difficult to add that to TemplateData schema and its editing UI.

Resource loader was going through refactoring and this change https://gerrit.wikimedia.org/r/c/mediawiki/core/+/518466 broke the hack we did in T214010: Show the messages in target language

jquery.i18n has grammar support. It will work if the corresponding language has the grammar rules defined. Currently there is a gap in those definitions. The rules defined in LanguageXX.php is not present in the language classes in Javascript

Thanks for writing this down. I agree with all of these points. While developing https://www.mediawiki.org/wiki/Extension:ExternalGuidance, I had the opportunity to extend and use it. The code was not that difficult to understand and extend to me, but I faced issues when APIs were changing. I got confused with the trade offs when there are more than one way to do the same.

About the missing title attribute for certain links, a debug log was added and monitored the production log.

In T206309#5308673, @Esanders wrote:

Replied on gerrit. I note that in the patch you are setting attributes.cx directly. If you are doing that anywhere else you should replace it with a transaction as the linear data should never been edited directly.

In T226611#5302850, @santhosh wrote:

In T226611#5285948, @Jpita wrote:

also happens with https://simple.wikipedia.org/wiki/Special:ContentTranslation?page=Chernobyl+Nuclear+Power+Plant&from=en&to=simple&targettitle=Chernobyl+Nuclear+Power+Plant when trying to translate the coordinates

It is a different issue: en->en translation with Google

{"status":500,"type":"internal_error","title":"Error","detail":"Translation with Google failed for en > en. Error: 400 : Bad language pair: en|en","method":"POST","uri":"/v2/translate/en/simple/Google"}

In T226611#5285948, @Jpita wrote:

also happens with https://simple.wikipedia.org/wiki/Special:ContentTranslation?page=Chernobyl+Nuclear+Power+Plant&from=en&to=simple&targettitle=Chernobyl+Nuclear+Power+Plant when trying to translate the coordinates

Hi @Esanders, we have a question in https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ContentTranslation/+/518922 about capturing the save event from reference editor.

In T144167#5299608, @Dvorapa wrote:

40 other catches: https://cs.wikipedia.org/w/index.php?search=insource%3A%2Fcx-segment%2Fi&title=Speci%C3%A1ln%C3%AD%3AHled%C3%A1n%C3%AD&go=J%C3%ADt+na (will remove them tomorrow)

The case of https://fr.wikipedia.org/w/index.php?title=Mattia_Runggaldier&diff=156608004&oldid=156490979 is investigated and found that the cx-segment was inside the reference content. Addressed that in https://gerrit.wikimedia.org/r/#/c/mediawiki/services/cxserver/+/520173

In https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ContentTranslation/+/511865 , we introduced mw.cx.ui.FeatureDiscoveryWidget.js which tries to achieve a use case of T216498: Invite to access Contributions for users that reached Content translation through other invites
We tried to make it somewhat generic.

As per T206310: CX2: Highlight references with a template that is missing mandatory parameters after being adapted now we don't show reference content if it is not adapted.

With the latest code, it is not possible to have different content for references with same name:

@Pginer-WMF Is there any other example with this issue? This particualr example is functioning correctly now.

This seems to be wrong - According to Wikidata ( https://www.wikidata.org/wiki/Q6242419#sitelinks-wikipedia ) the equivalent of sk:Harvbz is cs:Harvnb. Source language is Slovak (sk) and target language is Czech (cs). Will check

In T220752#5266162, @Vladis13 wrote:

How large performance issues can will? Will it a noticeable? From that link it not clear. AFAIU, the webfonts downloads once then caching for alltime.

As you can see non-English source languages also get ExternalGuidance features now:

Go to the first paragraph (after templates) that starts with "Józef Klemens Piłsudski..." When translated, the template will look like text without any formatting.

This plain text reference links are fixed in T216809: CX2: Wrong template is added despite being connected in Wikidata using https://gerrit.wikimedia.org/r/517796

In the master version of cxserver, the issue was not about mapping Harvnp to Sfn, but the translated content did not had any template. The reference link was coming as plain text

Enabling this in Wikisource seems ok, The webfonts feature when enabled by default in wikipedia, will cause large performance issues and this is the reason it is not enabled and left as a preference for user(See https://www.mediawiki.org/wiki/Universal_Language_Selector/WebFonts).

For the purpose of review/test, I am going to explain a template and its mapping using the databse here. This is the same template used for the unit test of above patch

I will clarify my question. We wanted to use a threshold score. Any mapping below that score wont be used. When referimento and referencia has a d value 0.40, and nom, nombre has 0.29, What does these numbers mean? Can they be interpreted as quality of match between 0 and 1? Which is best matching, which is bad matching based on these values? Are they really an indicator of good or bad matching or just an internal system value that is not useful?

After a brief analysis of the generated JSON mapping, this is what I am planning to do:

@diego I was processing the JSON files and trying to understand the values.