CX2: Content Translation creates articles that have tags with cx-segment
Open, Needs TriagePublic
Actions

Assigned To

None

Authored By

	Amire80
	Aug 29 2016, 11:06 AM

Description

In some cases, published content with Content Translation leaks cx-segment metadata elements into the published content. For example, in this article published with version 2 of Content Translation (on Feb 5 2019):

<ref> <span data-segmentid="129" class="cx-segment"><span title="A volte può capitare che un link presente su Wikipedia non sia più raggiungibile...

This abuse filter can be useful to get example and check if the issue persists over time.

Examples:

These were created long after the similar issue T113137 was resolved, so it's not a caching issue. I am marking this as Regression, even though the thing that caused it in this article might be different from what was addressed in T113137.

Details

	Subject	Repo	Branch	Lines +/-
	Do not segment content inside block templates	mediawiki/services/cxserver	master	+52 -1

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Open	None	T111155 ContentTranslation inserts unnecessary tags to translated pages (tracking)
Duplicate	None	T90202 Fix erroneous appearance of span tags in translation (tracking)
Open	None	T144167 CX2: Content Translation creates articles that have tags with cx-segment

Event Timeline

Amire80 created this task.Aug 29 2016, 11:06 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 29 2016, 11:06 AM

Amire80 added a parent task: T90202: Fix erroneous appearance of span tags in translation (tracking).Aug 29 2016, 11:07 AM

Amire80 moved this task from Needs Triage to Bugs on the ContentTranslation board.Sep 7 2016, 8:24 AM

Amire80 updated the task description. (Show Details)Sep 9 2016, 8:10 AM

Amire80 updated the task description. (Show Details)Sep 10 2016, 9:09 AM

Amire80 added a subscriber: Elinruby.

Amire80 updated the task description. (Show Details)Oct 8 2016, 3:39 PM

Amire80 added a project: ContentTranslation-out-of-beta.

Amire80 updated the task description. (Show Details)Oct 14 2016, 11:54 AM

Amire80 moved this task from Backlog to Mark-up and post-edit overhead on the ContentTranslation-out-of-beta board.Feb 7 2017, 12:24 PM

Amire80 triaged this task as High priority.May 9 2017, 5:44 PM

Amire80 moved this task from Bugs to Out of Beta on the ContentTranslation board.

Amire80 reopened this task as Open.Jun 28 2017, 9:04 AM

Amire80 closed this task as a duplicate of T154875: cx-segment injected to code by content translation.

Amire80 merged a task: T154875: cx-segment injected to code by content translation.

Amire80 added a subscriber: eranroz.

Arrbee edited projects, added CX-out-of-beta; removed ContentTranslation-out-of-beta, ContentTranslation.Jun 13 2018, 6:23 PM

Arrbee moved this task from Backlog to Mark-up and post-edit overhead on the CX-out-of-beta board.

Base subscribed.Dec 15 2018, 10:43 PM

I've created https://meta.wikimedia.org/wiki/Special:AbuseFilter/192 to mark new pages adding "cx-segment".

It is normal to have the phab ID in the diffs for edits made with that tag? https://fr.wikipedia.org/w/index.php?title=Ex-Mattatoio_de_Rome&oldid=156081838

In T144167#4901728, @Trizek wrote:

It is normal to have the phab ID in the diffs for edits made with that tag? https://fr.wikipedia.org/w/index.php?title=Ex-Mattatoio_de_Rome&oldid=156081838

That is not added by us. See Base's Abuse Filter above. It's a global filter so it is in use in all projects.

Thanks.

@Base, can you change the wording please? Have that Phab task is a bit disturbing. :)

Pginer-WMF renamed this task from Content Translation creates articles that have tags with cx-segment to CX2: Content Translation creates articles that have tags with cx-segment.Feb 13 2019, 10:47 AM

Pginer-WMF added a project: Language-Team (Language-2019-January-March).

Pginer-WMF updated the task description. (Show Details)

Based on the examples from the abuse filter it seems that these cases are less frequent with the new version (CX2), but they are still happening. I added one of such examples to the description to help investigate further.

Pginer-WMF mentioned this in T214799: NWE adds some weird "segment" span elements to the whole page.Feb 13 2019, 10:52 AM

Pginer-WMF merged a task: T214799: NWE adds some weird "segment" span elements to the whole page.

Pginer-WMF added subscribers: Dvorapa, JTannerWMF, matmarex, Urbanecm.

Pginer-WMF edited projects, added Language-Team (Language-2019-April-June); removed Language-Team (Language-2019-January-March).Mar 29 2019, 11:39 PM

New examples spotted: https://cs.wikipedia.org/wiki/Matchmaking_(video_hry), https://cs.wikipedia.org/wiki/Metroidvania

In T144167#5073649, @Dvorapa wrote:

New examples spotted: https://cs.wikipedia.org/wiki/Matchmaking_(video_hry), https://cs.wikipedia.org/wiki/Metroidvania

These two articles does not look like created using CX. They are created using VisualEditor:

https://cs.wikipedia.org/w/index.php?title=Matchmaking_(video_hry)&action=history
https://cs.wikipedia.org/w/index.php?title=Metroidvania&action=history

But it is strange to see the content has cx attributes. Wondering whether the user copied html from CX and pasted in VE in another browser tab?

Yeah, this possibility was discussed in the merged duplicate task. Perhaps CX should remove these tags when copied? Or produce some no-tag result? Or VE should perhaps remove these tags on paste?

Restricted Application added a project: VisualEditor. · View Herald TranscriptApr 1 2019, 8:24 PM

Dvorapa added a project: VisualEditor-MediaWiki-2017WikitextEditor.Apr 1 2019, 8:26 PM

In T144167#5075816, @Dvorapa wrote:

Yeah, this possibility was discussed in the merged duplicate task. Perhaps CX should remove these tags when copied? Or produce some no-tag result? Or VE should perhaps remove these tags on paste?

We should explore whether it is possible to clean the metadata when the user copies content from Content translation. We need to make sure that removing the metadata does not break pasting the contents in Content translation itself. Otherwise it would be problematic when moving contents around in the tool. Based on that we can propose for Visual Editor to support cleaning up the pasted contents.
I captured this case in a separate ticket: T220495: Content copied from Content Translation into Visual Editor exposes internal attributes

Hey @Pginer-WMF @Arrbee , will the language team take this task on?

In T144167#5097623, @JTannerWMF wrote:

Hey @Pginer-WMF @Arrbee , will the language team take this task on?

Yes. The scope for this one is for content created with Content translation. So maybe the VisualEditor-related tags can be removed.

For the particular case of content copied from Content translation and pasted into Visual Editor, I created a separate ticket (T220495). Even for that one, the plan is for the Language team to evaluate first if this can be solved from Content translation first.

Pginer-WMF moved this task from Backlog to Priority backlog on the Language-Team (Language-2019-April-June) board.Jun 3 2019, 4:02 PM

JTannerWMF removed projects: VisualEditor-MediaWiki-2017WikitextEditor, VisualEditor, VisualEditor-CopyPaste.Jun 13 2019, 1:38 AM

• santhosh claimed this task.Jul 1 2019, 11:53 AM

• santhosh moved this task from Priority backlog to In Progress on the Language-Team (Language-2019-April-June) board.

Pginer-WMF edited projects, added Language-Team (Language-2019-July-September); removed Language-Team (Language-2019-April-June).Jul 1 2019, 12:32 PM

Pginer-WMF moved this task from Backlog to In Progress on the Language-Team (Language-2019-July-September) board.

Today catch: https://cs.wikipedia.org/wiki/ITVBe

https://fr.wikipedia.org/w/index.php?title=Mattia_Runggaldier&diff=156608004&oldid=156490979	Checking...
https://cs.wikipedia.org/w/index.php?title=Lionsgate_Films&diff=next&oldid=16883945&diffmode=source	A case of content pasted from CX to VE
https://en.wikipedia.org/w/index.php?title=Chronology_of_Gabon&action=edit&oldid=736711208	Edit dated 29 August 2016
https://en.wikipedia.org/w/index.php?title=French_judiciary_courts&action=edit&oldid=738439575	Edit dated 9 Sept 2016
https://en.wikipedia.org/w/index.php?title=Code_p%C3%A9nal_(France)&action=edit&oldid=738621470	Edit dated 10 Sept 2016
https://en.wikipedia.org/w/index.php?title=Ignacio_Solano&action=edit&oldid=742570098	Edit dated 4 October 2016
https://he.wikipedia.org/w/index.php?title=%D7%92%D7%9C%D7%99%D7%95%D7%9E%D7%94&action=edit&oldid=19412830	Edit dated 10 October 2016

Change 520173 had a related patch set uploaded (by Santhosh; owner: Santhosh):
[mediawiki/services/cxserver@master] Do not segment content inside block templates

https://gerrit.wikimedia.org/r/520173

gerritbot added a project: Patch-For-Review.Jul 2 2019, 6:45 AM

The case of https://fr.wikipedia.org/w/index.php?title=Mattia_Runggaldier&diff=156608004&oldid=156490979 is investigated and found that the cx-segment was inside the reference content. Addressed that in https://gerrit.wikimedia.org/r/#/c/mediawiki/services/cxserver/+/520173

• santhosh moved this task from In Progress to In Review on the Language-Team (Language-2019-July-September) board.Jul 2 2019, 6:47 AM

40 other catches: https://cs.wikipedia.org/w/index.php?search=insource%3A%2Fcx-segment%2Fi&title=Speci%C3%A1ln%C3%AD%3AHled%C3%A1n%C3%AD&go=J%C3%ADt+na (will remove them tomorrow)

In T144167#5299608, @Dvorapa wrote:

40 other catches: https://cs.wikipedia.org/w/index.php?search=insource%3A%2Fcx-segment%2Fi&title=Speci%C3%A1ln%C3%AD%3AHled%C3%A1n%C3%AD&go=J%C3%ADt+na (will remove them tomorrow)

Thanks. I went through the list. Articles created using earlier version of CX(CX1) in 2015-2018 are there in the list - Though they are valid issues, CX1 code base is now not used for starting new articles. I am looking for these kind of issues if any created using CX2.

Change 520173 merged by jenkins-bot:
[mediawiki/services/cxserver@master] Do not segment content inside block templates

https://gerrit.wikimedia.org/r/520173

Maintenance_bot removed a project: Patch-For-Review.Jul 4 2019, 7:10 AM

• Petar.petkovic moved this task from In Review to Needs QA on the Language-Team (Language-2019-July-September) board.Jul 4 2019, 11:29 PM

Arrbee moved this task from Needs QA to Recheck after deployment on the Language-Team (Language-2019-July-September) board.Jul 9 2019, 8:10 AM

Arrbee moved this task from Recheck after deployment to Needs QA on the Language-Team (Language-2019-July-September) board.

• Jpita moved this task from Needs QA to Recheck after deployment on the Language-Team (Language-2019-July-September) board.Jul 9 2019, 12:49 PM

• Jpita closed this task as Resolved.Jul 16 2019, 12:05 PM

• Jpita moved this task from Recheck after deployment to Done on the Language-Team (Language-2019-July-September) board.

I'm here following the tag T144167. Please see- https://bn.wikipedia.org/w/index.php?title=পাপুয়া_বিশ্ববিদ্যালয়&diff=5933027&oldid=5933012&diffmode=source

Trizek-WMF removed a subscriber: Trizek.Jun 21 2022, 1:22 PM

Still loads of this crap being added, not sure why the task was closed: https://meta.wikimedia.org/w/index.php?title=Special:AbuseLog&wpSearchFilter=192