Page MenuHomePhabricator

VisualEditor automatically converts backslashes to forward slashes in wikilinks
Open, MediumPublicBUG REPORT

Description

A minimal example for the problem is the wikitext

[[o\i]]

Steps to replicate the issue:

What happens?:

  • VisualEditor converts the test case to
[[o/i|o\i]]

What should have happened instead?:

  • No change for "[[o\i]]"

Impact:

  • Links are silently corrupted without user action

Other information :

Event Timeline

zoe subscribed.

I think this is Parsoid. I've clipped out the HTML that ve is building its model from:

<a rel="mw:WikiLink" href="./O/i" title="O/i" class="mw-redirect" id="mwAw">o\\i</a>

For a fuller reference, here's what VE is reading on my machine:

<!DOCTYPE html>\n<html prefix="dc: http://purl.org/dc/terms/ mw: http://mediawiki.org/rdf/"><head prefix="mwr: https://de.wikipedia.org/wiki/Special:Redirect/"><meta charset="utf-8"/><meta property="mw:pageId" content="13752163"/><meta property="mw:pageNamespace" content="2"/><meta property="mw:revisionSHA1" content="d13caf4bd8cd5e774f11e09490f120829cad3cf7"/><meta property="mw:htmlVersion" content="2.8.0"/><meta property="mw:html:version" content="2.8.0"/><link rel="dc:isVersionOf" href="//de.wikipedia.org/wiki/Benutzer%3AKallichore/VE"/><base href="//de.wikipedia.org/wiki/"/><title>Benutzer:Kallichore/VE</title><link rel="stylesheet" href="/w/load.php?lang=de&amp;modules=mediawiki.skinning.content.parsoid%7Cmediawiki.skinning.interface%7Csite.styles&amp;only=styles&amp;skin=vector"/><meta http-equiv="content-language" content="de"/><meta http-equiv="vary" content="Accept"/></head><body lang="de" class="mw-content-ltr sitedir-ltr ltr mw-body-content parsoid-body mediawiki mw-parser-output" dir="ltr" data-mw-parsoid-version="0.23.0.0-alpha12" data-mw-html-version="2.8.0" id="mwAA"><section data-mw-section-id="0" id="mwAQ"><p id="mwAg"><a rel="mw:WikiLink" href="./O/i" title="O/i" class="mw-redirect" id="mwAw">o\\i</a></p>\n</section></body></html>

I've tagged in the team for Parsoid, who hopefully can help more.

The answer might be browsers.

Try http:\\www.google.com\search?q=example in your browser, for example.

  • Going to a /w/index.php?Title=O\I behaves
  • Going to /wiki/O\I translates to /

Also investigating from the VE side:

  • Raw fetch() behaves
  • $.ajax() behaves

BROWSERS

a = document.createElement('a');
a.setAttribute('href', './O\\i');
[ a.getAttribute('href'), a.href ];
// (2) ['./O\\i', 'https://de.wikipedia.org/w/O/i']

I think this might be best handled by having Parsoid escape \ to %5C. If you visit the read page with parsoid turned on then [[o\i]] is also rendered incorrectly.

The fact that it changes the behavior between Parsoid and legacy (and breaks Parsoid read views) is indeed a sign that we CTT should do something there.

It's not entirely clear to me if it should be a full escaping of \ to %5C on all hrefs (internal *and* external), and what to do with query strings (and what legacy does with these, as well). It's also not entirely clear to me if there's any risk associated with fully escaping the href if it's not necessary (I'd expect not, but I'm not confident there's not a surprise in there), or if there's a place where we'd rely on transforming \ to / in VE, or or or.

(It feels like it'd be fine, but I'm always a bit nervous about URL transformations). Any thoughts?

Change #1235157 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/services/parsoid@master] URL-encode backslashes in title hrefs

https://gerrit.wikimedia.org/r/1235157

This should be safe, patch above: we already have a "sanitize link" method, this just adds one character to it. We don't use the href for round tripping, so adding a bit of URL encoding is relatively harmless.

Change #1235157 merged by jenkins-bot:

[mediawiki/services/parsoid@master] URL-encode backslashes in title hrefs

https://gerrit.wikimedia.org/r/1235157

Change #1235865 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.23.0-a14

https://gerrit.wikimedia.org/r/1235865

Change #1235865 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.23.0-a14

https://gerrit.wikimedia.org/r/1235865

I verified on test.wikipedia.org with 1.46.0-wmf.14 that the problem with the minimal example has been fixed.

However, I also found that [[o%5Ci]] becomes [[o\i]] after "round tripping", while [[Hello%20World]] remains unchanged. Is this intentional?

Looks like some edge case in selective serialization:

$ echo "Testing: [[o%5Ci]] and [[o\i]] and [[Hello%20World]]." | php bin/parse.php --wt2wt
Testing: [[o%5Ci]] and [[o\i]] and [[Hello%20World]].

But, on mediawiki.org, the diff is dirty.

Local selser testing results:

$  ~/bin/selser.sh 
1c1
< Testing: [[o%5Ci]] and [[o\i]] and [[Hello%20World]].
---
> Testing: [[o%5Ci]], [[o\i]], [[Hello%20World]].

So, not selser inherently, something in the interaction between Parsoid & VE in that case.

(EDIT: bin/selser.sh is a local binary to simplify testing and is effectively this script:

$ php bin/parse.php $* --selser --html2wt --revtextfile /tmp/wt --revhtmlfile /tmp/old.html < /tmp/new.html > /tmp/wt.new
$ diff /tmp/wt /tmp/wt.new

)

So, the first HTML snippet is the original HTML from Parsoid's wt2html on [[o%5Ci]]. The second snippet removes data-parsoid from it. The third snippet retains the data-parsoid attribute, but changes the href by decoding the entity.

$ cat /tmp/x.html 
<p><a rel="mw:WikiLink" href="./O%5Ci?action=edit&amp;redlink=1" title="O\i" class="new" typeof="mw:LocalizedAttrs" data-parsoid='{"stx":"simple","a":{"href":"./O%5Ci"},"sa":{"href":"O%5Ci"}}' data-mw-i18n='{"ti
tle":{"lang":"x-page","key":"red-link-title","params":["O\\i"]}}'>O\i</a></p>

<p><a rel="mw:WikiLink" href="./O\i" title="O\i" class="new" typeof="mw:LocalizedAttrs" data-parsoid='{"stx":"simple","a":{"href":"./O%5Ci"},"sa":{"href":"O%5Ci"}}' data-mw-i18n='{"title":{"lang":"x-page","key":
"red-link-title","params":["O\\i"]}}'>O\i</a></p>

<p><a rel="mw:WikiLink" href="./O%5Ci?action=edit&amp;redlink=1" title="O\i" class="new" typeof="mw:LocalizedAttrs" data-mw-i18n='{"title":{"lang":"x-page","key":"red-link-title","params":["O\\i"]}}'>O\i</a></p>

$  php bin/parse.php --html2wt < /tmp/x.html
[[O%5Ci]]

[[O\i]]

[[O\i]]

As it turns out, when you dump the edited HTML from VE, you get the HTML form from the 3rd snippet where the redlink part of the url is removed and the entity is decoded.

For the wikitext seen in this revid.

Here is Parsoid's HTML as seen by VE:

<body class="mw-content-ltr sitedir-ltr ltr mw-body-content mediawiki mw-parser-output parsoid-body" lang="en" dir="ltr" data-mw-parsoid-version="0.24.0.0-alpha14" data-mw-html-version="2.8.0"><p id="mwAg">Testing: <a rel="mw:WikiLink" href="./O%5Ci?action=edit&amp;redlink=1" title="O\i" class="new" typeof="mw:LocalizedAttrs" data-mw-i18n="{&quot;title&quot;:{&quot;lang&quot;:&quot;x-page&quot;,&quot;key&quot;:&quot;red-link-title&quot;,&quot;params&quot;:[&quot;O\\i&quot;]}}" id="mwAw">o\i</a> and <a rel="mw:WikiLink" href="./O%5Ci?action=edit&amp;redlink=1" title="O\i" class="new" typeof="mw:LocalizedAttrs" data-mw-i18n="{&quot;title&quot;:{&quot;lang&quot;:&quot;x-page&quot;,&quot;key&quot;:&quot;red-link-title&quot;,&quot;params&quot;:[&quot;O\\i&quot;]}}" id="mwBA">o\i</a> and <a rel="mw:WikiLink" href="./Hello_World?action=edit&amp;redlink=1" title="Hello World" class="new" typeof="mw:LocalizedAttrs" data-mw-i18n="{&quot;title&quot;:{&quot;lang&quot;:&quot;x-page&quot;,&quot;key&quot;:&quot;red-link-title&quot;,&quot;params&quot;:[&quot;Hello World&quot;]}}" id="mwBQ">Hello World</a>.</p></body>

Here is the edited URL as sent by VE (after a trivial edit):

<body class="mw-content-ltr sitedir-ltr ltr mw-body-content mediawiki mw-parser-output parsoid-body" lang="en" dir="ltr" data-mw-parsoid-version="0.24.0.0-alpha14" data-mw-html-version="2.8.0"><p id="mwAg">Testing: <a href="./O\i" rel="mw:WikiLink" title="O\i" class="new" typeof="mw:LocalizedAttrs" data-mw-i18n="{&quot;title&quot;:{&quot;lang&quot;:&quot;x-page&quot;,&quot;key&quot;:&quot;red-link-title&quot;,&quot;params&quot;:[&quot;O\\i&quot;]}}" id="mwAw">o\i</a> and <a href="./O\i" rel="mw:WikiLink" title="O\i" class="new" typeof="mw:LocalizedAttrs" data-mw-i18n="{&quot;title&quot;:{&quot;lang&quot;:&quot;x-page&quot;,&quot;key&quot;:&quot;red-link-title&quot;,&quot;params&quot;:[&quot;O\\i&quot;]}}" id="mwBA">o\i</a> and <a href="./Hello_World" rel="mw:WikiLink" title="Hello World" class="new" typeof="mw:LocalizedAttrs" data-mw-i18n="{&quot;title&quot;:{&quot;lang&quot;:&quot;x-page&quot;,&quot;key&quot;:&quot;red-link-title&quot;,&quot;params&quot;:[&quot;Hello World&quot;]}}" id="mwBQ">Hello World</a>.</p><p id="mwAg">testing</p></body>

As you see that and [[o%5Ci]] has a decoded entity, but for the space, Parsoid did space normalization and used "_" in the href which Parsoid knows how to handle in the html2wt direction.

So, (a) VE is decoding entities in urls (b) Parsoid has special handling for space. Given (a) and (b), except for %20C, looks like all other entities will get normalized on save during VE edits, from what I can tell.

ssastry triaged this task as Medium priority.Jul 14 2026, 10:30 PM