Page MenuHomePhabricator

Machine Translation is broken when content has a link
Closed, ResolvedPublicBUG REPORT

Description

Steps to replicate the issue:

  • Start translation, select en -> es for example.
  • Try translating a paragraph with a link.

What happens?:

  • Error of 'Automatic translation failed' is displayed.

What should have happened instead?:

  • Machine translation should work correctly.
Error in console
jQuery.Deferred exception: Cannot read properties of null (reading 'attributes') TypeError: Cannot read properties of null (reading 'attributes')
    at ve.dm.CXLinkAnnotation.static.toDataElement (https://es.wikipedia.org/w/load.php?lang=gu&modules=ext.CodeMirror.visualEditor%7Cext.CodeMirror.visualEditor.init%7Cext.TemplateSandbox.visualeditor%7Cext.abuseFilter.visualEditor%7Cext.cite.styles%2CvisualEditor%7Cext.cite.visualEditor.core%7Cext.citoid.visualEditor%7Cext.citoid.visualEditor.data%7Cext.confirmEdit.CaptchaInputWidget%2CvisualEditor%7Cext.disambiguator.visualEditor%7Cext.geshi.visualEditor%7Cext.graph.data%2CvisualEditor%7Cext.kartographer%7Cext.kartographer.editing%2Cutil%2CvisualEditor%7Cext.math.styles%2CvisualEditor%7Cext.score.visualEditor%7Cext.score.visualEditor.icons%7Cext.spamBlacklist.visualEditor%7Cext.templateDataGenerator.editPage%7Cext.titleblacklist.visualEditor%7Cext.visualEditor.data%2CmoduleIcons%2CmoduleIndicators%7Cext.wikihiero.visualEditor%7Cext.wikimediaEvents.visualEditor%7Cmediawiki.ForeignStructuredUpload.BookletLayout%7Cmediawiki.ForeignUpload%2CUpload%7Cmediawiki.Upload.BookletLayout%7Cmediawiki.language.names%7Cmediawiki.libs.jpegmeta%7Cmediawiki.page.gallery.styles%7Cmediawiki.widgets.CategoryMultiselectWidget%2CDateInputWidget%2CMediaSearch%2CTable%2CUserInputWidget%7Cmediawiki.widgets.DateInputWidget.styles%7Cmoment%7Cmw.cx.visualEditor%7Coojs-ui.styles.icons-editing-citation%2Cicons-editing-list%2Cicons-editing-styling%2Cicons-layout%2Cicons-location%2Cicons-media%2Cicons-moderation%2Cicons-movement%2Cicons-user%2Cicons-wikimedia&skin=contenttranslation&version=qon0n:542:438)
    at ve.dm.Converter.createDataElements (https://es.wikipedia.org/w/load.php?lang=gu&modules=ext.visualEditor.base%2Ccore%2Cdiffing%2Cmediawiki%2Cmwcore%2Cmwlink%2Cmwmeta%2CprogressBarWidget%2Csanitize%2CsupportCheck%2Cswitching%2CtargetLoader%2CtempWikitextEditorWidget%2Ctrack%2Cve%2Cwelcome%7Cext.visualEditor.core.desktop%2Cutils%7Cext.visualEditor.core.utils.parsing%7Cext.visualEditor.desktopArticleTarget.init&skin=contenttranslation&version=cmwmm:275:644)
    at ve.dm.Converter.getDataFromDomSubtree (https://es.wikipedia.org/w/load.php?lang=gu&modules=ext.visualEditor.base%2Ccore%2Cdiffing%2Cmediawiki%2Cmwcore%2Cmwlink%2Cmwmeta%2CprogressBarWidget%2Csanitize%2CsupportCheck%2Cswitching%2CtargetLoader%2CtempWikitextEditorWidget%2Ctrack%2Cve%2Cwelcome%7Cext.visualEditor.core.desktop%2Cutils%7Cext.visualEditor.core.utils.parsing%7Cext.visualEditor.desktopArticleTarget.init&skin=contenttranslation&version=cmwmm:282:90)
    at ve.dm.Converter.getDataFromDomSubtree (https://es.wikipedia.org/w/load.php?lang=gu&modules=ext.visualEditor.base%2Ccore%2Cdiffing%2Cmediawiki%2Cmwcore%2Cmwlink%2Cmwmeta%2CprogressBarWidget%2Csanitize%2CsupportCheck%2Cswitching%2CtargetLoader%2CtempWikitextEditorWidget%2Ctrack%2Cve%2Cwelcome%7Cext.visualEditor.core.desktop%2Cutils%7Cext.visualEditor.core.utils.parsing%7Cext.visualEditor.desktopArticleTarget.init&skin=contenttranslation&version=cmwmm:283:103)
    at ve.dm.Converter.getDataFromDomSubtree (https://es.wikipedia.org/w/load.php?lang=gu&modules=ext.visualEditor.base%2Ccore%2Cdiffing%2Cmediawiki%2Cmwcore%2Cmwlink%2Cmwmeta%2CprogressBarWidget%2Csanitize%2CsupportCheck%2Cswitching%2CtargetLoader%2CtempWikitextEditorWidget%2Ctrack%2Cve%2Cwelcome%7Cext.visualEditor.core.desktop%2Cutils%7Cext.visualEditor.core.utils.parsing%7Cext.visualEditor.desktopArticleTarget.init&skin=contenttranslation&version=cmwmm:285:663)
    at ve.dm.Converter.getDataFromDomSubtree (https://es.wikipedia.org/w/load.php?lang=gu&modules=ext.visualEditor.base%2Ccore%2Cdiffing%2Cmediawiki%2Cmwcore%2Cmwlink%2Cmwmeta%2CprogressBarWidget%2Csanitize%2CsupportCheck%2Cswitching%2CtargetLoader%2CtempWikitextEditorWidget%2Ctrack%2Cve%2Cwelcome%7Cext.visualEditor.core.desktop%2Cutils%7Cext.visualEditor.core.utils.parsing%7Cext.visualEditor.desktopArticleTarget.init&skin=contenttranslation&version=cmwmm:285:663)
    at ve.dm.Converter.getModelFromDom (https://es.wikipedia.org/w/load.php?lang=gu&modules=ext.visualEditor.base%2Ccore%2Cdiffing%2Cmediawiki%2Cmwcore%2Cmwlink%2Cmwmeta%2CprogressBarWidget%2Csanitize%2CsupportCheck%2Cswitching%2CtargetLoader%2CtempWikitextEditorWidget%2Ctrack%2Cve%2Cwelcome%7Cext.visualEditor.core.desktop%2Cutils%7Cext.visualEditor.core.utils.parsing%7Cext.visualEditor.desktopArticleTarget.init&skin=contenttranslation&version=cmwmm:276:923)
    at ve.init.mw.CXTarget.setSectionContent (https://es.wikipedia.org/w/load.php?lang=gu&modules=diffMatchPatch%2Cdompurify%2Cjquery%2Coojs%2Coojs-ui%2Coojs-ui-core%2Coojs-ui-toolbars%2Coojs-ui-widgets%2Coojs-ui-windows%2Cpapaparse%2Crangefix%2Csite%2Cspark-md5%2CtreeDiffer%2Cunicodejs%7Cext.centralNotice.geoIP%7Cext.centralauth.ForeignApi%7Cext.centralauth.centralautologin.clearcookie%7Cext.cx.eventlogging.campaigns%7Cext.cx.model%7Cext.cx.widgets.spinner%7Cext.cx.wikibase.link%7Cext.echo.api%2Cinit%7Cext.eventLogging%2CnavigationTiming%2Cpopups%2CwikimediaEvents%7Cext.uls.common%2Ccompactlinks%2Cinterface%2Cpreferences%2Cwebfonts%7Cjquery.client%2Ccookie%2Ctablesorter%2CtextSelection%7Cjquery.tablesorter.styles%7Cjquery.uls.data%7Cmediawiki.ForeignApi%2CString%2CTitle%2CUri%2Capi%2Cbase%2Ccldr%2Ccookie%2Cexperiments%2CjqueryMsg%2Clanguage%2Cpulsatingdot%2Cstorage%2Cuser%2Cutil%2CvisibleTimeout%2Cwidgets%7Cmediawiki.ForeignApi.core%7Cmediawiki.action.view.redirectPage%7Cmediawiki.editfont.styles%7Cmediawiki.interface.helpers.styles%7Cmediawiki.language.months%7Cmediawiki.libs.pluralruleparser%7Cmediawiki.page.ready%7Cmediawiki.skinning.content.parsoid%7Cmediawiki.ui.button%7Cmediawiki.widgets.styles%7Cmw.cx.SiteMapper%2Cdm%2Cinit%2Cui%2Cutil%7Cmw.cx.dm.Translation%7Cmw.cx.tools.TranslationTool%7Cmw.cx.ui.Infobar%7Coojs-ui-core.icons%2Cstyles%7Coojs-ui-toolbars.icons%7Coojs-ui-widgets.icons%7Coojs-ui-windows.icons%7Coojs-ui.styles.icons-accessibility%2Cicons-content%2Cicons-editing-advanced%2Cicons-editing-core%2Cicons-interactions%2Cindicators%7Cve.ce.CXLintableNode%7Cve.dm.CXLintableNode%7Cve.init.mw.CXTarget&skin=contenttranslation&version=1c5oy:1264:216)
    at VeInitMwCXTarget.<anonymous> (https://es.wikipedia.org/w/load.php?lang=gu&modules=diffMatchPatch%2Cdompurify%2Cjquery%2Coojs%2Coojs-ui%2Coojs-ui-core%2Coojs-ui-toolbars%2Coojs-ui-widgets%2Coojs-ui-windows%2Cpapaparse%2Crangefix%2Csite%2Cspark-md5%2CtreeDiffer%2Cunicodejs%7Cext.centralNotice.geoIP%7Cext.centralauth.ForeignApi%7Cext.centralauth.centralautologin.clearcookie%7Cext.cx.eventlogging.campaigns%7Cext.cx.model%7Cext.cx.widgets.spinner%7Cext.cx.wikibase.link%7Cext.echo.api%2Cinit%7Cext.eventLogging%2CnavigationTiming%2Cpopups%2CwikimediaEvents%7Cext.uls.common%2Ccompactlinks%2Cinterface%2Cpreferences%2Cwebfonts%7Cjquery.client%2Ccookie%2Ctablesorter%2CtextSelection%7Cjquery.tablesorter.styles%7Cjquery.uls.data%7Cmediawiki.ForeignApi%2CString%2CTitle%2CUri%2Capi%2Cbase%2Ccldr%2Ccookie%2Cexperiments%2CjqueryMsg%2Clanguage%2Cpulsatingdot%2Cstorage%2Cuser%2Cutil%2CvisibleTimeout%2Cwidgets%7Cmediawiki.ForeignApi.core%7Cmediawiki.action.view.redirectPage%7Cmediawiki.editfont.styles%7Cmediawiki.interface.helpers.styles%7Cmediawiki.language.months%7Cmediawiki.libs.pluralruleparser%7Cmediawiki.page.ready%7Cmediawiki.skinning.content.parsoid%7Cmediawiki.ui.button%7Cmediawiki.widgets.styles%7Cmw.cx.SiteMapper%2Cdm%2Cinit%2Cui%2Cutil%7Cmw.cx.dm.Translation%7Cmw.cx.tools.TranslationTool%7Cmw.cx.ui.Infobar%7Coojs-ui-core.icons%2Cstyles%7Coojs-ui-toolbars.icons%7Coojs-ui-widgets.icons%7Coojs-ui-windows.icons%7Coojs-ui.styles.icons-accessibility%2Cicons-content%2Cicons-editing-advanced%2Cicons-editing-core%2Cicons-interactions%2Cindicators%7Cve.ce.CXLintableNode%7Cve.dm.CXLintableNode%7Cve.init.mw.CXTarget&skin=contenttranslation&version=1c5oy:1267:359)
    at mightThrow (https://es.wikipedia.org/w/load.php?lang=gu&modules=diffMatchPatch%2Cdompurify%2Cjquery%2Coojs%2Coojs-ui%2Coojs-ui-core%2Coojs-ui-toolbars%2Coojs-ui-widgets%2Coojs-ui-windows%2Cpapaparse%2Crangefix%2Csite%2Cspark-md5%2CtreeDiffer%2Cunicodejs%7Cext.centralNotice.geoIP%7Cext.centralauth.ForeignApi%7Cext.centralauth.centralautologin.clearcookie%7Cext.cx.eventlogging.campaigns%7Cext.cx.model%7Cext.cx.widgets.spinner%7Cext.cx.wikibase.link%7Cext.echo.api%2Cinit%7Cext.eventLogging%2CnavigationTiming%2Cpopups%2CwikimediaEvents%7Cext.uls.common%2Ccompactlinks%2Cinterface%2Cpreferences%2Cwebfonts%7Cjquery.client%2Ccookie%2Ctablesorter%2CtextSelection%7Cjquery.tablesorter.styles%7Cjquery.uls.data%7Cmediawiki.ForeignApi%2CString%2CTitle%2CUri%2Capi%2Cbase%2Ccldr%2Ccookie%2Cexperiments%2CjqueryMsg%2Clanguage%2Cpulsatingdot%2Cstorage%2Cuser%2Cutil%2CvisibleTimeout%2Cwidgets%7Cmediawiki.ForeignApi.core%7Cmediawiki.action.view.redirectPage%7Cmediawiki.editfont.styles%7Cmediawiki.interface.helpers.styles%7Cmediawiki.language.months%7Cmediawiki.libs.pluralruleparser%7Cmediawiki.page.ready%7Cmediawiki.skinning.content.parsoid%7Cmediawiki.ui.button%7Cmediawiki.widgets.styles%7Cmw.cx.SiteMapper%2Cdm%2Cinit%2Cui%2Cutil%7Cmw.cx.dm.Translation%7Cmw.cx.tools.TranslationTool%7Cmw.cx.ui.Infobar%7Coojs-ui-core.icons%2Cstyles%7Coojs-ui-toolbars.icons%7Coojs-ui-widgets.icons%7Coojs-ui-windows.icons%7Coojs-ui.styles.icons-accessibility%2Cicons-content%2Cicons-editing-advanced%2Cicons-editing-core%2Cicons-interactions%2Cindicators%7Cve.ce.CXLintableNode%7Cve.dm.CXLintableNode%7Cve.init.mw.CXTarget&skin=contenttranslation&version=1c5oy:122:881) undefined

Event Timeline

Just to add: it is literally ruining the ongoing Ukrainian Wikipedia / Wikimedia Ukraine editing campaign.

We are organising Wikimarathon, the biggest annual edit-a-thon campaign in Ukrainian Wikipedia (20+ events over 7 days, 100+ newbies creating their first articles), and we have A LOT of newbies willing to use ContentTranslate - and failing. There are workarounds (e.g. copy English Wikipedia markup to Ukrainian Wikipedia and translate it manually) but they are very ugly and not newbie-friendly at all. At a workshop I organised today half of newbies wanted to use it but failed.

It looks like this is a consequence of this week's release, can perhaps relevant changes be reverted outside the cycle? If it will not be fixed until next Wednesday, this will ruin our entire campaign as this seriously damages the newbie experience and creates confusion.

Thanks

This comment was removed by Aholtman.

This is a regression from recent Visual Editor update: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/VisualEditor/+/868191. Everything works till its previous revision.
@matmarex Could you please help? Since this makes CX unusable, may be revert this patch first?

I will investigate and/or revert within the next couple of hours.

Change 884842 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/extensions/ContentTranslation@master] Fix issues with parsing sections with links

https://gerrit.wikimedia.org/r/884842

Based on discussion with @santhosh, the patch above doesn't fix the issue in every case, although it's not clear to me how to reproduce remaining issues.

Let's revert for now. Note that this will regress some relatively minor issues in VE (T325566, T327779), so we should come up with a better fix soon.


I think the issue is that something, somewhere (in CX or in VE) is either parsing a HTML document that doesn't have a <base> node to define the base URI, or parsing a HTML fragment without specifying the relevant HTML document.

Previously VE code ignored the base URI when parsing links, so this seemed to work, but now we take it into account, so it needs to be set correctly.

Change 884498 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/extensions/VisualEditor@master] Revert "Remove references to mediawiki.Uri"

https://gerrit.wikimedia.org/r/884498

Change 884499 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/extensions/VisualEditor@master] Revert "Rewrite mw.libs.ve.getTargetDataFromHref with URL API"

https://gerrit.wikimedia.org/r/884499

Change 884500 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/extensions/VisualEditor@wmf/1.40.0-wmf.20] Revert "Remove references to mediawiki.Uri"

https://gerrit.wikimedia.org/r/884500

Change 884501 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/extensions/VisualEditor@wmf/1.40.0-wmf.20] Revert "Rewrite mw.libs.ve.getTargetDataFromHref with URL API"

https://gerrit.wikimedia.org/r/884501

Change 884498 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Revert "Remove references to mediawiki.Uri"

https://gerrit.wikimedia.org/r/884498

Change 884499 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Revert "Rewrite mw.libs.ve.getTargetDataFromHref with URL API"

https://gerrit.wikimedia.org/r/884499

Change 884500 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@wmf/1.40.0-wmf.20] Revert "Remove references to mediawiki.Uri"

https://gerrit.wikimedia.org/r/884500

Change 884501 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@wmf/1.40.0-wmf.20] Revert "Rewrite mw.libs.ve.getTargetDataFromHref with URL API"

https://gerrit.wikimedia.org/r/884501

Mentioned in SAL (#wikimedia-operations) [2023-01-30T14:21:19Z] <lucaswerkmeister-wmde@deploy1002> Started scap: Backport for [[gerrit:884500|Revert "Remove references to mediawiki.Uri" (T328143)]], [[gerrit:884501|Revert "Rewrite mw.libs.ve.getTargetDataFromHref with URL API" (T328143)]]

Mentioned in SAL (#wikimedia-operations) [2023-01-30T14:22:57Z] <lucaswerkmeister-wmde@deploy1002> matmarex and lucaswerkmeister-wmde: Backport for [[gerrit:884500|Revert "Remove references to mediawiki.Uri" (T328143)]], [[gerrit:884501|Revert "Rewrite mw.libs.ve.getTargetDataFromHref with URL API" (T328143)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-01-30T14:33:27Z] <lucaswerkmeister-wmde@deploy1002> Finished scap: Backport for [[gerrit:884500|Revert "Remove references to mediawiki.Uri" (T328143)]], [[gerrit:884501|Revert "Rewrite mw.libs.ve.getTargetDataFromHref with URL API" (T328143)]] (duration: 12m 07s)

ContentTranslation should be working again now.

Next steps: Language team, please review https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ContentTranslation/+/884842 and test it together with https://gerrit.wikimedia.org/r/c/mediawiki/extensions/VisualEditor/+/884502. If it works correctly, then hopefully we can merge the patches to reinstate the fixes for VE bugs I mentioned in T328143#8569625. If issues still occur, then I'd be happy to review patches if you have ideas on how to fix them, or we could do some debugging together.

It'd be helpful to have a public test wiki with ContentTranslation and those two patches deployed for testing.

I also ran into an unrelated issue while doing the backports: T328310: CX doesn't load when testing backports on mwdebug servers

Change 885459 had a related patch set uploaded (by Santhosh; author: Santhosh):

[mediawiki/extensions/ContentTranslation@master] Make link parsing agnostic to baseURI, wgScript, wgArticlePath

https://gerrit.wikimedia.org/r/885459

Change 885459 merged by jenkins-bot:

[mediawiki/extensions/ContentTranslation@master] Make link parsing agnostic to baseURI, wgScript, wgArticlePath

https://gerrit.wikimedia.org/r/885459

Thanks for the patch @santhosh! And sorry once again that we broke this.

Change 884842 abandoned by Bartosz Dziewoński:

[mediawiki/extensions/ContentTranslation@master] Fix issues with parsing sections with links

Reason:

No longer needed

https://gerrit.wikimedia.org/r/884842

matmarex reassigned this task from matmarex to santhosh.