Change title slighlty since CTX or CTX2 are not acronyms used for ContentTranslation

@Doc_James: For future reference, please follow the scheme on https://www.mediawiki.org/wiki/How_to_report_a_bug - thanks!

Here we have another article and language in which the translation tool struggles

https://or.wikipedia.org/w/index.php?title=%E0%AC%AC%E0%AD%8D%E0%AD%9F%E0%AC%AC%E0%AC%B9%E0%AC%BE%E0%AC%B0%E0%AC%95%E0%AC%BE%E0%AC%B0%E0%AD%80:Doc_James/Tricuspid_valve_stenosis&oldid=428053

A bunch of instances of the reference "<ref name=Stat2020/> have gone missing and some of them have been misformated into "<ref name="Stat2020">{{Cite journal|last=Golamari|first=R|last2=Bhattacharya|first2=PT|date=January 2020|title=Tricuspid Stenosis|pmid=29763166}}<cite class="citation journal cs1" data-ve-ignore="true" id="CITEREFGolamariBhattacharya2020">Golamari, R; Bhattacharya, PT (January 2020). "Tricuspid Stenosis". [[PMID (ପରିଚାୟକ)|PMID]]&nbsp;[//pubmed.ncbi.nlm.nih.gov/29763166 29763166].</cite><span data-ve-ignore="true"> </span><span class="cs1-hidden-error error citation-comment" data-ve-ignore="true">Cite journal requires <code class="cs1-code">&#x7C;journal=</code> ([[ସହଯୋଗ:CS1 errors|help]])</span></ref>"

These are the manual changes I made to fix the issue https://or.wikipedia.org/w/index.php?title=%E0%AC%9F%E0%AD%8D%E0%AC%B0%E0%AC%BE%E0%AC%87%E0%AC%95%E0%AC%B8%E0%AC%AA%E0%AC%BF%E0%AC%A1_%E0%AC%95%E0%AC%AA%E0%AC%BE%E0%AC%9F%E0%AC%BF%E0%AC%95%E0%AC%BE_%E0%AC%B8%E0%AD%8D%E0%AC%9F%E0%AD%87%E0%AC%A8%E0%AD%8B%E0%AC%B8%E0%AC%BF%E0%AC%B8&type=revision&diff=428054&oldid=428020

We, at Wiki Project Med, will try to build a work around. Basically what we will do is expand all the references in a userspace for the content to be translated. https://en.wikipedia.org/w/index.php?title=User:Mr._Ibrahem/Tropical_sprue2&diff=1023306255&oldid=1023305378&diffmode=source

People will translate using content translations.

And than we will have a bot shrink all the references once the translation is live. https://or.wikipedia.org/w/index.php?title=%E0%AC%AC%E0%AD%8D%E0%AD%9F%E0%AC%AC%E0%AC%B9%E0%AC%BE%E0%AC%B0%E0%AC%95%E0%AC%BE%E0%AC%B0%E0%AD%80:Doc_James/Tropical_sprue3&diff=428092&oldid=428088

This appears to solve the missing references issues.

I tried to capture the problematic content in this test page. However, the published result includes all the references from the test page.

Does anyone have any suggestion about which content to include in the test page so that we can have a minimal reproducible test case?

Although I could not reproduce the main issue I noticed that:

• The the published for the example has references with red errors, because references by name get duplicated. This is a known issue capture in T203772. We got details from the Editing team on how this can be solved, but @santhosh can confirm if we have everything that is needed, or further details may be needed about this.
• One reference (Stat2020) in the original article had some mandatory parameters missing (the journal title). Content Translation shows a warning about it. This was motivated by community requests complaining of malformed templates when parameters could not be adapted (although in this case, the parameter is missing from the beginning, not lost in translation). I wonder if the issues persist when such parameter is filled in the translation tool (or in the source article). See images below:

Original reference	Warning in Content Translation

Here is an example

Source: https://en.wikipedia.org/wiki/User:Doc_James/CTX2

Results: https://ja.wikipedia.org/w/index.php?title=%E5%88%A9%E7%94%A8%E8%80%85:Doc_James/CTX2&action=edit

Not a single reference of the format "<ref name=ABC/>" comes through as such.

The full references come through. And the "<ref name=ABC/>" references occasionally come through after having a bunch of error text added to them (such as your example).

Here is a smaller amount of content were the ref do not come through

4 refs in the starting text https://en.wikipedia.org/wiki/User:Doc_James/CTX3

2 refs in the end text https://ja.wikipedia.org/wiki/%E5%88%A9%E7%94%A8%E8%80%85:Doc_James/CTX3

I found a reproducible test case. This example page contains 3 instances of the same reference. When it is translated from English to Japanese (using Google translate for each paragraph) the published result only contains 2 instances of the reference (i.e., one got lost).

The expected result would be for the three instances to be present in the published article.

Below you can see the source, translation and target screenshots. Notice that the reference next to the "kidney failure" link is only missing in the published content (see the red zone highlighted):

Source example content	Source and target in Content Translation	Published content

(The published page also shows some reference errors because of the known issue of duplication of references by name: T203772: CX2: Improve support for references that are reused in multiple places)

In T282369#7114724, @Doc_James wrote:

Here is a smaller amount of content were the ref do not come through

4 refs in the starting text https://en.wikipedia.org/wiki/User:Doc_James/CTX3

2 refs in the end text https://ja.wikipedia.org/wiki/%E5%88%A9%E7%94%A8%E8%80%85:Doc_James/CTX3

Thanks! This is super useful. I'll add it in the description in addition to the one I reproduced, since sometimes apparently similar issues with references are caused by different underlying issues.

Change 701349 had a related patch set uploaded (by Santhosh; author: Santhosh):

[mediawiki/services/cxserver@master] Reference adaptaton: Support named references

https://gerrit.wikimedia.org/r/701349

Change 701349 merged by jenkins-bot:

[mediawiki/services/cxserver@master] Reference adaptaton: Support named references

https://gerrit.wikimedia.org/r/701349

Change 704659 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2021-07-14-124232-production

https://gerrit.wikimedia.org/r/704659

Change 704659 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2021-07-14-124232-production

https://gerrit.wikimedia.org/r/704659

Mentioned in SAL (#wikimedia-operations) [2021-07-15T05:50:09Z] <kart_> Updated cxserver to 2021-07-14-124232-production (T282369, T284450)

I now see all the references from @Doc_James example.

Can you confirm?

Yah so it is no longer dropping the references. But is now duplicating the reference metadata with some duplications differing from the original :-(

You can see the same issue here today https://ja.wikipedia.org/wiki/%E5%88%A9%E7%94%A8%E8%80%85:Doc_James/Short_bowel_syndrome

What we at Wiki Project Med did was duplicate all the references before the mediawiki markup was put into content translation and than we have a bot that gets ride of the duplicate after. Not sure if we need a new ticket for this.

In T282369#7485037, @Doc_James wrote:

Yah so it is no longer dropping the references. But is now duplicating the reference metadata with some duplications differing from the original :-(

I think this issue is the one captured in a different ticket: T203772: CX2: Improve support for references that are reused in multiple places

Please let us know if you think it is not the same or some key detail may be missing. The process of de-duplicating the references is not trivial, but we plan to work in this area as part of the Content Translation maintenance work.

Yah that looks like the issue. Thanks