I tried to reproduce this, Yandex is working fine. But Apertium was throwing error:

{"status":500,"type":"internal_error","title":"Error","detail":"Translation with Apertium en>es failed: HTTPError: 504: internal_http_error","method":"POST","uri":"/v2/translate/en/es/Apertium"}

@Pginer-WMF I linked the "Why?" text to https://www.mediawiki.org/wiki/Content_translation/Machine_Translation. Is that ok?

Screenshot:

In T144467#4802407, @sbassett wrote:

1. I applied your patches from https://gerrit.wikimedia.org/r/477732 and https://gerrit.wikimedia.org/r/477757, which seem to work well in alleviating the concerns around <style> and <svg> content. For now, within the process flow of 1) reducing html via lineardoc 2) sending said html off to the Google or Yandex MT services 3) expanding the received translated html 4) sending it through DOMPurify.sanitize, the only potential for http leaks I'm seeing are if they already existed within the source html.

In T144467#4799236, @sbassett wrote:

Just to confirm - lineardoc is something developed in-house using JavaScript's sax module, correct?

In T144467#4795794, @sbassett wrote:

This is increasingly becoming a problem as npm audit is reporting more issues from the dependencies.

In order to support this what's proposed is for the plain text generated to be highlighted as long as the yellow marker appears next to the paragraph. That is, this "persistent warning highlight" will only go away when the user marks the warning as resolved. Visually this warning looks like the sentence highlighting but it should be possible to differentiate it internally in case that we decide to change the way we signal warnings.

Notes for front end changes:

In this table, 'disable-mt' , 'source-mt' , 'TestClient', 'no-mt', 'scratch' are ignorables. They are there because of development iterations. Obvious from the numbers too.

I am looking for a real example where cxserver could not map mandatory params.

This require additional adaptation information passed from cxserver. I will work on that part.

In T203160#4759136, @Esanders wrote:

I'm not hugely familiar with your subclasses/mixins and how they relate to un-adapted references. Could be a bit more specific about what the problem is?

I am closing this since we addressed this in https://gerrit.wikimedia.org/r/c/mediawiki/services/cxserver/+/461386

Need help from Cite extenion team to resolve this.

Let us start the VE behavior for the source artilce:

Found this error while checking T208386. I used the leading paragraph of Hugo Kołłątaj from English to Japanese.

However, plaintext which replaced the inline template is not highlighted with sentence highlighting.

Fixed that.

As per the master version of cxserver and MW CX,
This is the Yandex translation and adaptation result for the first para of en:Statue of Unity revision=865364435

// For cases where templateData is missing, Content Translation can consider matching the existing parameters by name as a fallback approach.

The en->ja pair now has template data in both languages. From history, I see User:Doc James added that to ja.wikipedia.org

In P7783#45771, @Nikerabbit wrote:

Only route that explicitly returns 404 is fetchPage.

I believe TestClient needs to be added to the config.yaml or config.prod.yaml. The tests when run from command line automatically load config.dev.yaml, but that does not happen in production.

The subclassing of ve.dm.MWReferenceNode to achieve this feature does not look working. The code changes in https://gerrit.wikimedia.org/r/472925 is giving this error:

Let us address this in iterations. In https://gerrit.wikimedia.org/r/472100 I added a feature to add edit tags based on campaign param in URLs. These tags will be validated against the configuration in the target wiki. This is for cx2.

It seems there is a problem with the registration of ve.dm.CXTransclusionNode. It inherits ve.dm.MWTransclusionNode and prevents ve.dm.MWReferencesListNode being used for the reflist.

<div about="#mwt221" data-mw={} typeof="mw:Transclusion"> 
<ol about="#mwt326 class="mw-references references" data-mw="{}" typeof="mw:Extension/references">

As illustrated in above patch, cxserver has metric reporting capacity already. We just need to emit appropriate counters to track errors or success. In production if cxserver is configured with statsd, we can monitor the services(actually anything in cxserver) using grafana.wikimedia.org or any graphite dashboard.

Since CX follows translations by adding a section at a time, at any order, there is a challenge to support named referneces. The content can be in

There are multiple issues here. Let us anlyse each one.

Found T208386: MT error while translating the big reflist section while debugging this issue

The root cause was the reflist contains template style as well. It is really unnecessary to machine translate the section if it is only a block level template.

I can confirm this is still happening.

In T144467#4692976, @cscott wrote:

I think you still need to sanitize -- for example, the <script> tag has its payload delivered by the node children, not the attributes. Potentially <style> is dangerous in the same way, if there is a CSS vulnerability. But again, a tag whitelist should be sufficient for this.

@Bawolff, Some updates on this:

It is a regression from https://gerrit.wikimedia.org/r/c/mediawiki/services/cxserver/+/461386. The attributes are restored on wrong elements in the reduce-expand procedure. You can see the section tag getting the attributes of a reference. In lineardoc model, the references usually goes to a sub document. the counter we used as id for the attribute dump is reset to 0 in that case. Fixed now.

After the above patch:

It need to be better rewritten using OOUI now. This was written couple of years back

There may be something we can do to avoid highlighting sentence in non-connected sections(as shown in image above). But if we do, it need to be a new ticket.

@Petar.petkovic That is a general problem with any restore. We have not worked on such a sentence matching algorithm. Infact, we did not promise accuracy for the sentence highlight than 'as best as we can do'.
+cc @Pginer-WMF please confirm.

I think we will need a new event there to indicate alignement and not a section change for save.

In cxserver master, Yandex also able to translate this. It seems cxserver for production did not catch up the fixes. @KartikMistry please note,

Timing analysis for translate api for infobox of en:Vantaa