Page MenuHomePhabricator
Create Task
Maniphest T411238

Unexpected wikitext changes & whitespace removals by VisualEditor edits
Open, HighPublicBUG REPORT
Actions

Description

see https://de.wikipedia.org/w/index.php?title=Salpingoeca&diff=prev&oldid=261885884

The only Visual Editor change we have done here was a typo correction: kkennt → kennt (deleted one letter only)

After saving the change we got a diff, that deleted many other characters in the wikitext, and this must not have happened.

This bug is reproduceable by different users with same result.

Steps for the wikilink underscore example to reproduce:

  • Open a page for visual editing
  • Insert a Cite ref
  • Give the ref the body [[a b]].
  • Review changes as wikitext

The link will show as [[a_b]], it should have remained [[a b]].

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Bump parsoid to v0.23.0-a7.1 on wmf.4mediawiki/corewmf/1.46.0-wmf.4+1 -1
Bump parsoid to v0.23.0-a7.1 on wmf.4mediawiki/vendorwmf/1.46.0-wmf.4+89 -22
Bump wikimedia/parsoid to 0.23.0-a7.1mediawiki/vendormaster+341 -440
Bump wikimedia/parsoid to 0.23.0-a7.1mediawiki/coremaster+1 -1
Fix dirty-diffs in ref contentmediawiki/services/parsoidmaster+69 -2
Bump wikimedia/parsoid to 0.23.0-a8mediawiki/vendormaster+296 -166
Customize query in gerrit

Related Objects

Event Timeline

doctaxon created this task.Nov 28 2025, 7:47 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 28 2025, 7:47 AM
Johannnes89 subscribed.Nov 28 2025, 7:57 AM
doctaxon added a comment.EditedNov 28 2025, 11:00 AM

https://de.wikipedia.org/w/index.php?diff=261940052

a further unexpected automatic change has happened here: after saving there have been underscores now in one of the wikilinks in the wikitext: [[Bundeszentrale für Politische Bildung]] → [[Bundeszentrale_für_Politische_Bildung]]

Raymond subscribed.Nov 28 2025, 11:21 AM
doctaxon added a comment.EditedNov 28 2025, 11:52 AM

one further wikilink underscore example: https://de.wikipedia.org/w/index.php?diff=261941024

awight updated the task description. (Show Details)Nov 28 2025, 12:08 PM
MartinRulsch subscribed.Nov 28 2025, 12:39 PM
doctaxon updated the task description. (Show Details)Nov 28 2025, 12:50 PM
thiemowmde triaged this task as Unbreak Now! priority.Nov 28 2025, 12:51 PM
thiemowmde edited projects, added Regression, Parsoid, Content-Transform-Team, WMDE-TechWish-Sprint-2025-11-25-Spekulatius, Cite; removed VisualEditor.
thiemowmde added subscribers: thiemowmde, Lina_Farid_WMDE, cscott, Arlolra.

I was able to reproduce the underscore bug locally. When I do any edit the content of my example reference gets re-serialized as <ref>[[a_b]]</ref> in wikitext, with underscores instead of spaces. I can confirm it is caused by https://gerrit.wikimedia.org/r/1208418 and the linked Parsoid patch https://gerrit.wikimedia.org/r/1208139. Temporarily marking as "unbreak now" for the Content-Transform-Team to have a look.

thiemowmde moved this task from Sprint Backlog to Epics / Watching / Stalled on the WMDE-TechWish-Sprint-2025-11-25-Spekulatius board.Nov 28 2025, 12:51 PM
thiemowmde renamed this task from unexpected wikitext deletions by Visual Editor edits to Unexpected wikitext changes & whitespace removals by VisualEditor edits.Nov 28 2025, 12:54 PM
thiemowmde added subscribers: awight, WMDE-Fisch.
doctaxon added a project: User-notice.Nov 28 2025, 1:03 PM
cscott added a comment.Nov 28 2025, 3:42 PM

The original change in this set was DataMwAttrib: Use live DocumentFragment for html values, which uncovered various issues with serialization of attribute-embedded data-parsoid requiring follow ups. Mentioning this in case it is relevant to the investigation.

gerritbot added a comment.Nov 29 2025, 1:48 AM

Change #1212798 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/services/parsoid@master] [WIP] Fix regression

https://gerrit.wikimedia.org/r/1212798

gerritbot added a project: Patch-For-Review.Nov 29 2025, 1:48 AM
ihurbain mentioned this in T411227: CTT tasks week of 2025-11-28.Nov 29 2025, 10:07 AM
Johannnes89 merged a task: T411302: Visual editor randomly adding underscores in internal links.Nov 29 2025, 2:40 PM
Johannnes89 added a subscriber: strakhov.
Johannnes89 merged a task: T411323: Add Link is adding underscores to existing links.Nov 30 2025, 1:01 PM
Johannnes89 added a subscriber: Titore.
Valepert subscribed.Nov 30 2025, 1:30 PM
matmarex subscribed.Dec 1 2025, 11:29 AM
Boshomi_Phabricator subscribed.Dec 1 2025, 2:12 PM
ihurbain edited projects, added Content-Transform-Team (Work In Progress); removed Content-Transform-Team.Dec 1 2025, 4:30 PM
ihurbain added a project: Essential-Work.
MSantos moved this task from Backlog to Code Review on the Content-Transform-Team (Work In Progress) board.Dec 1 2025, 4:31 PM
gerritbot added a comment.Dec 1 2025, 5:20 PM

Change #1212798 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Fix dirty-diffs in ref content

https://gerrit.wikimedia.org/r/1212798

gerritbot added a comment.Dec 1 2025, 5:29 PM

Change #1213536 had a related patch set uploaded (by Isabelle Hurbain-Palatin; author: Isabelle Hurbain-Palatin):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.23.0-a8

https://gerrit.wikimedia.org/r/1213536

gerritbot added a comment.Dec 1 2025, 6:06 PM

Change #1213536 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.23.0-a8

https://gerrit.wikimedia.org/r/1213536

Maintenance_bot removed a project: Patch-For-Review.Dec 1 2025, 6:31 PM
Michael subscribed.Dec 1 2025, 7:20 PM
Aschmidt subscribed.Dec 1 2025, 10:28 PM
cscott assigned this task to Arlolra.Dec 2 2025, 3:52 AM
Sjoerddebruin merged a task: T411496: VisualEditor appears to be replacing spaces with underscores in wikilinks.Dec 2 2025, 1:34 PM
Sjoerddebruin added a subscriber: Nirmos.
Nirmos added a subscriber: ssastry.Dec 2 2025, 1:46 PM

@cscott , @ssastry , @Arlolra Thanks for whipping up code to fix this! Now, if I remember correctly, for Parsoid changes to take effect on Wikimedia wikis, Stashbot needs to be run? What I mean to say is: I don't think this can wait until Thursday evening.

gerritbot added a comment.Dec 2 2025, 2:02 PM

Change #1214045 had a related patch set uploaded (by Isabelle Hurbain-Palatin; author: Isabelle Hurbain-Palatin):

[mediawiki/core@master] Bump wikimedia/parsoid to 0.23.0-a7.1

https://gerrit.wikimedia.org/r/1214045

gerritbot added a project: Patch-For-Review.Dec 2 2025, 2:02 PM

Change #1214047 had a related patch set uploaded (by Isabelle Hurbain-Palatin; author: Isabelle Hurbain-Palatin):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.23.0-a7.1

https://gerrit.wikimedia.org/r/1214047

gerritbot added a comment.Dec 2 2025, 2:11 PM

Change #1214045 abandoned by Isabelle Hurbain-Palatin:

[mediawiki/core@master] Bump wikimedia/parsoid to 0.23.0-a7.1

Reason:

i don't want to do that, actually

https://gerrit.wikimedia.org/r/1214045

gerritbot added a comment.Dec 2 2025, 2:12 PM

Change #1214047 abandoned by Isabelle Hurbain-Palatin:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.23.0-a7.1

Reason:

i don't want to do that, actually

https://gerrit.wikimedia.org/r/1214047

Maintenance_bot removed a project: Patch-For-Review.Dec 2 2025, 2:31 PM
gerritbot added a comment.Dec 2 2025, 2:47 PM

Change #1214069 had a related patch set uploaded (by Isabelle Hurbain-Palatin; author: Isabelle Hurbain-Palatin):

[mediawiki/vendor@wmf/1.46.0-wmf.4] Bump parsoid to v0.23.0-a7.1 on wmf.4

https://gerrit.wikimedia.org/r/1214069

gerritbot added a project: Patch-For-Review.Dec 2 2025, 2:47 PM
gerritbot added a comment.Dec 2 2025, 2:52 PM

Change #1214070 had a related patch set uploaded (by Isabelle Hurbain-Palatin; author: Isabelle Hurbain-Palatin):

[mediawiki/core@wmf/1.46.0-wmf.4] Bump parsoid to v0.23.0-a7.1 on wmf.4

https://gerrit.wikimedia.org/r/1214070

gerritbot added a comment.Dec 2 2025, 3:10 PM

Change #1214069 merged by jenkins-bot:

[mediawiki/vendor@wmf/1.46.0-wmf.4] Bump parsoid to v0.23.0-a7.1 on wmf.4

https://gerrit.wikimedia.org/r/1214069

gerritbot added a comment.Dec 2 2025, 3:11 PM

Change #1214070 merged by jenkins-bot:

[mediawiki/core@wmf/1.46.0-wmf.4] Bump parsoid to v0.23.0-a7.1 on wmf.4

https://gerrit.wikimedia.org/r/1214070

Maintenance_bot removed a project: Patch-For-Review.Dec 2 2025, 3:30 PM
Stashbot mentioned this in T410960: CTT tasks week of 2025-11-21.Dec 2 2025, 4:35 PM

Mentioned in SAL (#wikimedia-operations) [2025-12-02T16:35:25Z] <ihurbain@deploy2002> Started scap sync-world: Backport for [[gerrit:1214069|Bump parsoid to v0.23.0-a7.1 on wmf.4 (T411238 T410960)]], [[gerrit:1214070|Bump parsoid to v0.23.0-a7.1 on wmf.4 (T411238 T410960)]]

Stashbot added a comment.Dec 2 2025, 4:37 PM

Mentioned in SAL (#wikimedia-operations) [2025-12-02T16:37:32Z] <ihurbain@deploy2002> ihurbain: Backport for [[gerrit:1214069|Bump parsoid to v0.23.0-a7.1 on wmf.4 (T411238 T410960)]], [[gerrit:1214070|Bump parsoid to v0.23.0-a7.1 on wmf.4 (T411238 T410960)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Stashbot added a comment.Dec 2 2025, 4:44 PM

Mentioned in SAL (#wikimedia-operations) [2025-12-02T16:44:46Z] <ihurbain@deploy2002> Finished scap sync-world: Backport for [[gerrit:1214069|Bump parsoid to v0.23.0-a7.1 on wmf.4 (T411238 T410960)]], [[gerrit:1214070|Bump parsoid to v0.23.0-a7.1 on wmf.4 (T411238 T410960)]] (duration: 09m 21s)

ihurbain subscribed.Dec 2 2025, 4:50 PM

@Nirmos we backported that patch to wmf.4, so the fix should be available on all wikis as of a few minutes ago.

Nirmos added a comment.Dec 2 2025, 5:33 PM

Thank you so much for your work on this, @ihurbain ! I do believe backporting this was the right call.

STei-WMF subscribed.Dec 3 2025, 3:46 PM

Re: Tech News - What wording would you suggest as the content, and When should it be included? Thanks!

Johannnes89 added a comment.Dec 3 2025, 5:31 PM

I don't think this is worth a Tech News entry. Others might disagree (perhaps @doctaxon who added the user-notice tag?) but I would still consider this a "limited-scale incident" -> https://meta.wikimedia.org/wiki/Tech/News/For_contributors#What_is_typically_not_included

doctaxon added a comment.Dec 3 2025, 7:19 PM

@Johannnes89 : I think it's worth a Tech News because many contributors using Visual Editor have run into this error, and it was not only dewiki. But I am not that professional in technical things to create a proper wording.

doctaxon added a comment.Dec 3 2025, 7:33 PM

@STei-WMF : Tech News wording suggested: "A regression bug in Parsoid caused VisualEditor to make unintended changes to wikitext, including removing whitespace and replacing spaces with underscores in wikilinks inside citations. The bug was introduced on November 28 and affected multiple wikis. A fix was deployed to all Wikimedia wikis on December 2, 2025. Editors who used VisualEditor between these dates should review their recent edits for unexpected modifications."

MSantos lowered the priority of this task from Unbreak Now! to High.Dec 4 2025, 3:25 PM
MSantos moved this task from Code Review to To Verify on the Content-Transform-Team (Work In Progress) board.
MSantos subscribed.

Changing the priority to reflect reality even though we are still verifying it.

MSantos moved this task from Needs Triage to Bugs & Crashers on the Parsoid board.Dec 4 2025, 3:28 PM
STei-WMF moved this task from To Triage to Announce in next Tech/News on the User-notice board.Dec 4 2025, 4:41 PM
STei-WMF moved this task from Announce in next Tech/News to To Triage on the User-notice board.
Aschmidt unsubscribed.Dec 4 2025, 5:21 PM
Nirmos added a comment.Dec 5 2025, 8:32 AM

I wonder if this is truly fixed. VE is till making changes like this. Is there perhaps a different task for that?

Quiddity moved this task from To Triage to In current Tech/News draft on the User-notice board.Dec 5 2025, 10:18 PM
Nirmos added a comment.Dec 8 2025, 2:02 PM

VE is also seemingly replacing Image with File. Should I open a new task for these issues?

STei-WMF moved this task from In current Tech/News draft to Already announced/Archive on the User-notice board.Dec 8 2025, 8:17 PM
MonX94 mentioned this in T412343: CX 2 replicate the same reference for other passage with different ref in original text.Dec 12 2025, 10:07 AM
Wellverywell subscribed.Dec 12 2025, 11:07 AM
ABreault-WMF claimed this task.Mon, Jan 12, 9:49 PM
ABreault-WMF removed a subscriber: Arlolra.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits