Page MenuHomePhabricator

Ongoing translations refuse to load in the Content translation tool
Closed, ResolvedPublicBUG REPORT

Description

List of steps to reproduce:

  • Some users login, click on translations.
  • Selects a translation that is in progress, eg: for example: es:Mariana BO to continue the translation from Spanish to Italian.
  • The article or Content translation loads forever.

What happens:
Some users complained that they can't resume their translations in Content translation environment, it just loads forever in different language pairs. This is not restricted to one browser (chrome), but other browsers.
Also, one person got this error: 4 Javascript errors in terminal: Uncaught TypeError: this.targetSurface is null / Special:ContentTranslation:225:477.

What should have happened instead:
The Translations are supposed to load without the constant loading "animation" for user to continue pending translations.


Based on the original reports several users were experiencing the issue.


There is another report that is similar to this issue, This time, it is only one article out of many the user has in progress; translating from English to Portuguese using the Firefox browser. Maybe it is the same bug?

Event Timeline

Pginer-WMF triaged this task as Medium priority.May 10 2022, 8:10 AM
Pginer-WMF updated the task description. (Show Details)

I am able to reproduce this issue locally. Steps I followed:
Translate Mariano BO from es to en. Translate a few sections including the infobox.
Save, reload the translation again. The source article is not loading.

In debug mode, the following error is noted:

jQuery.Deferred exception: Cannot read properties of undefined (reading 'length') TypeError: Cannot read properties of undefined (reading 'length')
    at Object.ve.dm.MWTransclusionNode.static.getWikitext (http://en.mediawiki.mwdd.localhost:8080/w/extensions/VisualEditor/modules/ve-mw/dm/nodes/ve.dm.MWTransclusionNode.js?7c6cb:388:39)
    at VeDmCXTransclusionBlockNode.ve.dm.MWTransclusionNode.getWikitext (http://en.mediawiki.mwdd.localhost:8080/w/extensions/VisualEditor/modules/ve-mw/dm/nodes/ve.dm.MWTransclusionNode.js?7c6cb:492:33)
    at VeCeCXTransclusionBlockNode.ve.ce.MWTransclusionNode.generateContents (http://en.mediawiki.mwdd.localhost:8080/w/extensions/VisualEditor/modules/ve-mw/ce/nodes/ve.ce.MWTransclusionNode.js?bbaef:158:47)
    at VeCeCXTransclusionBlockNode.ve.ce.GeneratedContentNode.forceUpdate (http://en.mediawiki.mwdd.localhost:8080/w/extensions/VisualEditor/lib/ve/src/ce/nodes/ve.ce.GeneratedContentNode.js?85514:318:46)
    at VeCeCXTransclusionBlockNode.ve.ce.GeneratedContentNode.update (http://en.mediawiki.mwdd.localhost:8080/w/extensions/VisualEditor/lib/ve/src/ce/nodes/ve.ce.GeneratedContentNode.js?85514:297:8)
    at VeCeCXTransclusionBlockNode.ve.ce.CXTransclusionBlockNode.update (http://en.mediawiki.mwdd.localhost:8080/w/extensions/ContentTranslation/modules/ve-cx/ce/ve.ce.CXTransclusionNode.js?d2939:83:62)
    at VeCeCXTransclusionBlockNode.VeCeGeneratedContentNode (http://en.mediawiki.mwdd.localhost:8080/w/extensions/VisualEditor/lib/ve/src/ce/nodes/ve.ce.GeneratedContentNode.js?85514:25:7)
    at VeCeCXTransclusionBlockNode.VeCeMWTransclusionNode (http://en.mediawiki.mwdd.localhost:8080/w/extensions/VisualEditor/modules/ve-mw/ce/nodes/ve.ce.MWTransclusionNode.js?bbaef:26:29)
    at VeCeCXTransclusionBlockNode.VeCeCXTransclusionNode (http://en.mediawiki.mwdd.localhost:8080/w/extensions/ContentTranslation/modules/ve-cx/ce/ve.ce.CXTransclusionNode.js?d2939:18:33)
    at VeCeCXTransclusionBlockNode (http://en.mediawiki.mwdd.localhost:8080/w/extensions/ContentTranslation/modules/ve-cx/ce/ve.ce.CXTransclusionNode.js?d2939:40:38) undefined

Change 790680 had a related patch set uploaded (by Santhosh; author: Santhosh):

[mediawiki/services/cxserver@master] MT Sanitize: Avoid removing data-mw because of '/>' content

https://gerrit.wikimedia.org/r/790680

santhosh raised the priority of this task from Medium to High.

I started translating Mariano BO from es to it. I noticed the js error posted above in the console when I try to translate the infobox. The infox is https://es.wikipedia.org/wiki/Plantilla:Ficha_de_persona and there is no corresponding template in Italian. Missing template is not an issue here. The output we get from cxserver for translating the infobox template was missing data-mw attribute and that was the root cause of the js error from VE.

So I tried translating the template using the template debugger. Found that it is able to translate the template. The difference between how the template debugger does the translation and CX is, debugger does not pass an MT engine. So, I tried using Google, Yandex and other options in CX. All worked, except Apertium. Only when Apertium is used the js error is happening. When the js error is happening while reloading the translation, the page is stuck in 'loading source article' mode.

To check what is Apertium doing, I save the HTML of infobox in an html file and ran with debugger scripts in cxserver:

cat /tmp/MarianaBO.html|  ./bin/translate.js Apertium es it

The data-mw attribute was missing from output. I suspected the sanitizer we use on top of MT result. For other MT engines, we use a reduce-expand-html algorith and they don't use dom sanitizer. I found that translation has data-mw before sanitizing and missing after sanitizing. We had upgraded DOMSanitizer many times in the past. To figure out what content in the template caused the sanitizer to remove data-mw. After removing fragments of data-mw one after another, I found the issue is in a text with br />. If I remove that text, the sanitizer is not removing data-mw. After going through DOMSanitizer documentation and issues for some time, I found this ticket https://github.com/cure53/DOMPurify/issues/657 which talks about same issue. DOMSanitizer considers br /> problematic because it can potentially cause issues with jquery 3.0. So I added a sanitizer hook to replace br /> with br> so that effectively we will get <br> instead of <br/>

This issue must be happening for all templates with data-mw having br /> and when used with Apertium. I don't know for how long. The English version of this template - https://en.wikipedia.org/wiki/Template:Infobox_person - does not have any br /> - That makes this issue less frequent.

For record, here is the template content from es:Mariana BO article

<table class="infobox biography vcard" style="width:22.7em; line-height: 1.4em; text-align:left; padding:.23em;" about="#mwt1" typeof="mw:Transclusion" data-mw='{"parts":[{"template":{"target":{"wt":"Ficha de persona\n","href":"./Plantilla:Ficha_de_persona"},"params":{"nombre":{"wt":"Mariana BO"},"nombre de nacimiento":{"wt":"Sandra Mariana Borrego Robles"},"fecha de nacimiento":{"wt":"{{fecha|27|10|1990|edad}} &lt;br /> {{bandera|México}} [[Culiacán]], [[Sinaloa]], [[México]]."},"ocupación":{"wt":"[[Disc jockey|DJ]], [[productor discográfico|productor]]"},"pareja":{"wt":"[[Haziel Ariel]]"},"años activo":{"wt":"[[2011]]–presente"},"género":{"wt":"[[Bass house]] ,[[Hardstyle]]  [[Psytrance]]"},"relacionados":{"wt":"[[22Bullets]], [[Angemi]], [[Blasterjaxx]], [[Daddy&apos;s Groove]], [[Dubdogz]], [[Futuristic Polar Bears]], [[KSHMR]], [[MR. BLACK]], [[Seven Lions]], [[Yves V]], [[ZAA]]."},"4":{"wt":"[[Hardstyle]]"},"instrumento":{"wt":"[[Programación]], [[Violín]], [[Percusiones]]."},"3":{"wt":"[[dubstep]]"},"2":{"wt":"[[Trap music (EDM)|trap]]"},"1":{"wt":"\n"},"alias":{"wt":"Mariana BO"},"URL":{"wt":"{{URL|https://majomontemayor.com/}}"}},"i":0}}]}' id="mwAQ">

Change 790964 had a related patch set uploaded (by Santhosh; author: Santhosh):

[mediawiki/extensions/ContentTranslation@master] Allow restoring translations with invalid templates

https://gerrit.wikimedia.org/r/790964

Change 790680 merged by jenkins-bot:

[mediawiki/services/cxserver@master] MT Sanitize: Avoid removing data-mw because of '/>' content

https://gerrit.wikimedia.org/r/790680

Change 791052 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2022-05-11-135122-production

https://gerrit.wikimedia.org/r/791052

Change 791052 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2022-05-11-135122-production

https://gerrit.wikimedia.org/r/791052

Change 790964 merged by jenkins-bot:

[mediawiki/extensions/ContentTranslation@master] Allow restoring translations with invalid templates

https://gerrit.wikimedia.org/r/790964

Testing status: QA PASS

  • Able continue in-progress translations from one language to the other
  • english to igbo
  • spanish to italian
  • english to hebrew
AlicjaCz subscribed.

Hi, the problem still persists while translating article from French to Polish (article:Adelaide Valentin). I checked in multiple browsers and computers and the translation loads forever. I think the issue is the same as in this ticket: T224715. I was adding external references which seems to be the problem. Could you please check? Thank you!

@AlicjaCz I will check once again in some combinations and get back to you. Thanks for the heads up

@santhosh When I tested this it did get fixed now it seems the problem is back. Thanks @AlicjaCz

@santhosh See attached video

@AlicjaCz Apologies for my initial comment. I was looking at the wrong link. I did try a first CX translations and I was able to open ongoing translations or create new ones

A related issue was reported: T309487: Translation not loading

We may want to investigate a bit more if there are still issues with loading.

@Pginer-WMF agreed. Will try different scenarios to try to reproduce this. Maybe try other browsers as well to expand testing

@Pginer-WMF I did have 11 ongoing translations and where able to open all of them in the following browsers:

  • Chrome
  • Safari
  • Firefox

This issue does seem to be fixed as I am unable to reproduce it. Unless it can be attributed to certain articles to a certain language1 --> language2 combination. Pls let me know what you think

Hi,

I'm not sure if the e-mails are addressed to me but it's still not working
so the issue hasn't been fixed. I have an open ticket here: T309487.

Thank you
Ala

Le lun. 13 juin 2022 à 03:22, EChukwukere-WMF <
no-reply@phabricator.wikimedia.org> a écrit :

EChukwukere-WMF added a comment. View Task
https://phabricator.wikimedia.org/T307967

@Pginer-WMF https://phabricator.wikimedia.org/p/Pginer-WMF/ I did have
11 ongoing translations and where able to open all of them in the following
browsers:

  • Chrome
  • Safari
  • Firefox

This issue does seem to be fixed as I am unable to reproduce it. Unless it
can be attributed to certain articles to a certain language1 --> language2
combination. Pls let me know what you think

*TASK DETAIL*
https://phabricator.wikimedia.org/T307967

*EMAIL PREFERENCES*
https://phabricator.wikimedia.org/settings/panel/emailpreferences/

*To: *santhosh, EChukwukere-WMF
*Cc: *Pginer-WMF, AlicjaCz, EChukwukere-WMF, Louperivois, santhosh,
UOzurumba, Aklapper, Wangombe, PallaviPatke, Rileych, Nintendofan885,
50019062, chapulina, Alfa80, Soum213, Taiwania_Justo, Nizil, Ixocactus,
Amire80, Jsahleen, Nikerabbit, Arrbee, KartikMistry

@AlicjaCz can you pls try opening those ongoing translations in an incognito browser for any of chrome, Firefox or Safari ? I'm interested to see if its working on or not in that scenario

It's not working in incognito, regardless of the browser

Le lun. 13 juin 2022 à 16:59, EChukwukere-WMF <
no-reply@phabricator.wikimedia.org> a écrit :

EChukwukere-WMF added a comment. View Task
https://phabricator.wikimedia.org/T307967

@AlicjaCz https://phabricator.wikimedia.org/p/AlicjaCz/ can you pls try
opening those ongoing translations in an *incognito* browser for any of
chrome, Firefox or Safari ? I'm interested to see if its working on or in
that scenario

*TASK DETAIL*
https://phabricator.wikimedia.org/T307967

*EMAIL PREFERENCES*
https://phabricator.wikimedia.org/settings/panel/emailpreferences/

*To: *santhosh, EChukwukere-WMF
*Cc: *Pginer-WMF, AlicjaCz, EChukwukere-WMF, Louperivois, santhosh,
UOzurumba, Aklapper, Wangombe, PallaviPatke, Rileych, Nintendofan885,
50019062, chapulina, Alfa80, Soum213, Taiwania_Justo, Nizil, Ixocactus,
Amire80, Jsahleen, Nikerabbit, Arrbee, KartikMistry

@AlicjaCz thank you. I will try a few more things to try to reproduce this

@AlicjaCz I did further investigation combinaions on chrome, safari and firefox ( latest versions of the browers used) in incognito and also regular mode I am unable to reproduce this issue as it seems to be working properly now.

I will be closing this task.

On your part I will suggest if you can create another with a video of what you observe on your end or you can email the video to me (echukwukere@wikimedia.org, since phabricator might have some video size limits lower than gmails) so I can poke to see if there is something I am missing before we open up another task for it in phabricator.

Thanks for your patience on this