Page MenuHomePhabricator
Create Task
Maniphest T178037

Parsoid uses non-canonical URL encoding in <link> in <head>
Open, MediumPublic
Actions

Description

Compare:

: and $ are reserved charcters so URLs which differ in how these characters are encoded are not considered equal. The web server will consider them equivalent so this is not a big deal in practice, but it will split Varnish and other caches, and maybe confuse semantic web applications.

Related Objects

Event Timeline

Tgr created this task.Oct 12 2017, 12:42 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 12 2017, 12:42 AM
• mobrovac triaged this task as Medium priority.Oct 12 2017, 7:48 AM
• mobrovac added a project: Services (later).
• Pchelolo added a project: Parsoid.Oct 17 2017, 10:00 AM
• Pchelolo added subscribers: BBlack, • Pchelolo.

: and $ are reserved charcters so URLs which differ in how these characters are encoded are not considered equal. The web server will consider them equivalent so this is not a big deal in practice, but it will split Varnish and other caches, and maybe confuse semantic web applications.

As far as I know Varnish treats URI-encoded and non-uri-encoded URIs equally for REST API after T127370 and T127387 however I think it's specific to REST API.

Here the values in the link tag is actually coming from Parsoid so I guess Parsoid could duplicate the MW-specific URI encoding logic and use that for /wiki requests. However, as I understand in Varnish we already have logic to normalize url-encoded and non-uri-encoded and MW-specific-uri-encoded variants, so we might consider applying that logic to /wiki requests as well.

@BBlack probably has way better understanding of this.

Tgr added a comment.Oct 17 2017, 8:03 PM

Ideally this should be fixed in Parsoid. There are minor side effects other than Varnish cache splitting, such as client-side cache, or visited URL coloring.

• ssastry renamed this task from RESTBase uses non-canonical URL encoding to Parsoid uses non-canonical URL encoding.Nov 6 2017, 5:00 PM
• ssastry raised the priority of this task from Medium to High.
• ssastry moved this task from Needs Triage to Next Up on the Parsoid board.
• ssastry moved this task from Next Up to Needs Triage on the Parsoid board.Dec 18 2017, 9:33 PM
• mobrovac added a project: Platform Team Legacy (Later).Dec 20 2018, 12:07 PM
LGoto moved this task from Needs Triage to Backlog on the Parsoid board.Feb 15 2020, 9:42 PM
LGoto lowered the priority of this task from High to Medium.Mar 20 2020, 4:26 PM
LGoto moved this task from Backlog to Needs Investigation on the Parsoid board.
cscott subscribed.Mar 20 2020, 4:26 PM

Needs re-investigation now that we've ported to PHP, who knows how we're encoding URLs now. Mostly likely we're using the same mechanism that core uses.

• ssastry assigned this task to cscott.Mar 23 2020, 3:38 PM
cscott added a comment.EditedApr 10 2020, 3:57 PM

Parsoid/PHP still generates <link rel="dc:isVersionOf" href="//www.mediawiki.org/wiki/Manual%3A%24wgEnableAPI"/>.
That appears to be standard PHP urlencode:

$ psysh
>>> urlencode('Foo:$Bar')
=> "Foo%3A%24Bar"

If we were to change urlencoding we'd have to be very careful, since there are certainly cases where we can get tripped up: eg, File:Foo.jpg doesn't parse as a relative link, while File%3AFoo.jpg does. I think we typically use ./Foo.jpg now, and:

$ echo '[[Ke$ha]]' | php bin/parse.php 
<a rel="mw:WikiLink" href="./Ke$ha" title="Ke$ha" data-parsoid='{"stx":"simple","a":{"href":"./Ke$ha"},"sa":{"href":"Ke$ha"},"dsr":[0,9,2,2]}' class="mw-redirect">Ke$ha</a>

So is the issue just the encoding of the canonical links in our <head>? We could probably use the same encoding routine we use for titles...

cscott renamed this task from Parsoid uses non-canonical URL encoding to Parsoid uses non-canonical URL encoding in <link> in <head>.Apr 10 2020, 3:57 PM
cscott moved this task from Needs Investigation to Missing Functionality on the Parsoid board.
• ssastry removed cscott as the assignee of this task.Apr 26 2020, 5:11 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits