Page MenuHomePhabricator
Create Task
Maniphest T150112

Internal links pointing to interwikis are not encoded at all
Open, MediumPublic
Actions

Description

An internal link inside flow, that points to an interwiki (that is, not on the wiki itself but another that can be referenced using internal link syntax with the corresponding prefix), it's not encoded at all.

Example:

[[w:en:Wikipedia:FAQ#How do I change my username/delete my account?]]

Points to:

https://en.wikipedia.org/wiki/Wikipedia:FAQ#How do I change my username/delete my account?

But it should point to:

https://en.wikipedia.org/wiki/Wikipedia:FAQ#How_do_I_change_my_username.2Fdelete_my_account.3F

If the link points to the local wiki, the anchor is encoded properly. See Topic:Tesyta6rspc1pysm for a test.

Note also that the link is rendered as class="external" instead of class="extiw" as it would otherwise on normal wiki pages. Normal wiki pages render this correctly.

Another interesting case:

[[w:Wikipedia:Sandbox?action=edit]]

Points to

https://en.wikipedia.org/wiki/Wikipedia:Sandbox?action=edit

But when editing the message, the edit text is now:

[w:Wikipedia:Sandbox?action=edit w:Wikipedia:Sandbox?action=edit]

Details

SubjectRepoBranchLines +/-
Munge link fragments and element ids as in the php parsermediawiki/services/parsoidmaster+65 -35
Customize query in gerrit

Related Objects

Event Timeline

Ciencia_Al_Poder created this task.Nov 6 2016, 1:57 AM
Restricted Application added a project: Collaboration-Team-Triage. · View Herald TranscriptNov 6 2016, 1:57 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Ciencia_Al_Poder updated the task description. (Show Details)Nov 6 2016, 2:00 AM
Ciencia_Al_Poder renamed this task from The anchor of internal links for interwikis are not encoded to Internal links pointing to interwikis are not encoded at all.Nov 8 2016, 1:41 AM
Ciencia_Al_Poder updated the task description. (Show Details)
• Mattflaschen-WMF added a project: Parsoid.Nov 15 2016, 6:45 PM
ssastry subscribed.Nov 17 2016, 10:15 PM

I am surprised how the "Internal: [[Manual:Errors and symptoms#Image Thumbnails not working and/or appearing]]" link got encoded properly. But, https://gerrit.wikimedia.org/r/#/c/320929/ starts generated section anchor ids identical to what core parser does with the right encoding. We now need to apply it to links that point to sections. I'll take a look once that other patch is reviewed.

ssastry claimed this task.Nov 17 2016, 10:21 PM
ssastry triaged this task as Medium priority.
• Mattflaschen-WMF removed projects: StructuredDiscussions, Collaboration-Team-Triage.Nov 17 2016, 10:30 PM
ssastry reassigned this task from ssastry to Arlolra.Nov 30 2016, 10:59 PM
gerritbot subscribed.Dec 2 2016, 10:38 PM

Change 324564 had a related patch set uploaded (by Arlolra):
Munge link fragments and element ids as in the php parser

https://gerrit.wikimedia.org/r/324564

gerritbot added a project: Patch-For-Review.Dec 2 2016, 10:38 PM
gerritbot added a comment.Dec 6 2016, 7:40 PM

Change 324564 merged by jenkins-bot:
Munge link fragments and element ids as in the php parser

https://gerrit.wikimedia.org/r/324564

Nemo_bis subscribed.Dec 7 2016, 2:21 PM

If/when all is done on the Parsoid's side, please edit or file a report against Flow to address the issue originally reported in T94949: Interwiki links to other MediaWiki wikis in the same cluster don't encode section fragment, so that the imported threads get fixed.

Stashbot mentioned this in T110910: Implement <gallery> extension natively inside Parsoid.Dec 7 2016, 9:36 PM
Stashbot mentioned this in T102209: Derive heading ids from heading name, the same way MW core does.
Stashbot mentioned this in T94949: Interwiki links to other MediaWiki wikis in the same cluster don't encode section fragment.
Stashbot subscribed.

Mentioned in SAL (#wikimedia-operations) [2016-12-07T21:36:38Z] <arlolra> updated Parsoid to version 3cf19c6b (T110910, T102209, T94949, T150112, T151570, T149209, T150213)

Stashbot mentioned this in T151570: Create Wikivoyage Finnish.Dec 7 2016, 9:36 PM
Stashbot mentioned this in T149209: Parsoid serialised an edit to a wikitext table adding a /n without stripping the double-pipes, breaking the table format (`\n|| align="right" | …`).
Stashbot mentioned this in T150213: Unknown contentmodels.
Arlolra merged a task: T94949: Interwiki links to other MediaWiki wikis in the same cluster don't encode section fragment.Dec 7 2016, 10:15 PM
Arlolra added subscribers: He7d3r, Arlolra, • Mattflaschen-WMF.
• GWicke subscribed.Dec 7 2016, 10:17 PM

See also: T152540: Migrate to HTML5 section ids.

Arlolra added a comment.Dec 7 2016, 10:26 PM

Current output for echo "[[w:en:Wikipedia:FAQ#How do I change my username/delete my account?]]" | node bin/parse --prefix frwiki is,

<a rel="mw:ExtLink" href="https://en.wikipedia.org/wiki/en:Wikipedia:FAQ#How_do_I_change_my_username.2Fdelete_my_account.3F" title="w:en:Wikipedia:FAQ" data-parsoid='{"stx":"simple","a":{"href":"https://en.wikipedia.org/wiki/en:Wikipedia:FAQ#How_do_I_change_my_username.2Fdelete_my_account.3F"},"sa":{"href":"w:en:Wikipedia:FAQ#How do I change my username/delete my account?"},"isIW":true}'>w:en:Wikipedia:FAQ#How do I change my username/delete my account?</a>

ie. https://en.wikipedia.org/wiki/en:Wikipedia:FAQ#How_do_I_change_my_username.2Fdelete_my_account.3F

Arlolra edited projects, added StructuredDiscussions; removed Patch-For-Review.Dec 7 2016, 10:33 PM

Note also that the link is rendered as class="external" instead of class="extiw" as it would otherwise on normal wiki pages. Normal wiki pages render this correctly.

Parsoid doesn't add these classes, so I'm guessing that's a Flow issue? It does have mw:ExtLink and "isIW":true though.

Restricted Application added a project: Collaboration-Team-Triage. · View Herald TranscriptDec 7 2016, 10:33 PM
Arlolra removed Arlolra as the assignee of this task.Dec 7 2016, 10:39 PM

Another interesting case

echo "[[w:Wikipedia:Sandbox?action=edit]]" | node bin/parse --prefix frwiki --wt2wt

[w:Wikipedia:Sandbox?action=edit w:Wikipedia:Sandbox?action=edit]

but

echo "[[w:Wikipedia:Sandbox?action=edit]]" | node bin/parse --wt2wt

[[w:Wikipedia:Sandbox?action=edit]]

Probably related to T102556 or https://gerrit.wikimedia.org/r/#/c/293799/

@ssastry wanna take a look?

Catrope subscribed.Dec 7 2016, 11:20 PM
In T150112#2855738, @Arlolra wrote:

Note also that the link is rendered as class="external" instead of class="extiw" as it would otherwise on normal wiki pages. Normal wiki pages render this correctly.

Parsoid doesn't add these classes, so I'm guessing that's a Flow issue? It does have mw:ExtLink and "isIW":true though.

Flow adds class="external" when it postprocesses the Parsoid output. See https://phabricator.wikimedia.org/diffusion/EFLW/browse/master/includes/Parsoid/Fixer/ExtLinkFixer.php

Arlolra added a comment.Dec 7 2016, 11:35 PM

@Catrope Thanks for the pointer. I guess this is a dupe of T97093 then.

Catrope added a comment.Dec 7 2016, 11:57 PM
In T150112#2855931, @Arlolra wrote:

@Catrope Thanks for the pointer. I guess this is a dupe of T97093 then.

Is it? I mean this task is also about serialization/round-trip issues.

Arlolra added a comment.Dec 8 2016, 12:00 AM

Is it? I mean this task is also about serialization/round-trip issues.

Sorry, I didn't mean the entire task, just the specific part we're discussing (ie. the link is rendered as class="external" instead of class="extiw").

Catrope added a comment.Dec 8 2016, 12:14 AM
In T150112#2855999, @Arlolra wrote:

Is it? I mean this task is also about serialization/round-trip issues.

Sorry, I didn't mean the entire task, just the specific part we're discussing (ie. the link is rendered as class="external" instead of class="extiw").

Ah, I see. Yes, partly, although the other part of it is that the specific class name is needed for MW-specific CSS to kick in.

ssastry moved this task from Needs Triage to Link syntax (links & media) on the Parsoid board.Sep 18 2017, 5:14 PM
Aklapper removed a project: Collaboration-Team-Triage.Oct 12 2019, 4:11 AM
Restricted Application added a project: Growth-Team. · View Herald TranscriptOct 12 2019, 4:11 AM
ssastry moved this task from Link syntax (links & media) to Needs Investigation on the Parsoid board.Feb 28 2020, 5:33 PM
ssastry assigned this task to cscott.Mar 23 2020, 3:42 PM
Arlolra removed cscott as the assignee of this task.Mar 24 2021, 10:47 PM
Arlolra added a subscriber: cscott.
Arlolra moved this task from Needs Investigation to Needs Triage on the Parsoid board.Mar 24 2021, 10:55 PM
Arlolra moved this task from Needs Triage to Bugs & Crashers on the Parsoid board.Apr 14 2021, 4:16 PM

Nowadays,

> echo "[[w:en:Wikipedia:FAQ#How do I change my username/delete my account?]]" | php bin/parse.php --domain fr.wikipedia.org
<p data-parsoid='{"dsr":[0,69,0,0]}'><a rel="mw:WikiLink/Interwiki" href="https://en.wikipedia.org/wiki/en:Wikipedia:FAQ#How_do_I_change_my_username/delete_my_account?" title="w:en:Wikipedia:FAQ" data-parsoid='{"stx":"simple","a":{"href":"https://en.wikipedia.org/wiki/en:Wikipedia:FAQ#How_do_I_change_my_username/delete_my_account?"},"sa":{"href":"w:en:Wikipedia:FAQ#How do I change my username/delete my account?"},"isIW":true,"dsr":[0,69,2,2]}'>w:en:Wikipedia:FAQ#How do I change my username/delete my account?</a></p>

> echo "[[w:en:Wikipedia:FAQ#How do I change my username/delete my account?]]" | php bin/parse.php --domain fr.wikipedia.org --wt2wt
[w:en:Wikipedia:FAQ#How do I change my username/delete my account? w:en:Wikipedia:FAQ#How do I change my username/delete my account?]

so roundtripping isn't doing so great.

The other case,

> echo "[[w:Wikipedia:Sandbox?action=edit]]" | php bin/parse.php --domain fr.wikipedia.org
<p data-parsoid='{"dsr":[0,35,0,0]}'><a rel="mw:WikiLink/Interwiki" href="https://en.wikipedia.org/wiki/Wikipedia:Sandbox%3Faction=edit" title="w:Wikipedia:Sandbox?action=edit" data-parsoid='{"stx":"simple","a":{"href":"https://en.wikipedia.org/wiki/Wikipedia:Sandbox%3Faction=edit"},"sa":{"href":"w:Wikipedia:Sandbox?action=edit"},"isIW":true,"dsr":[0,35,2,2]}'>w:Wikipedia:Sandbox?action=edit</a></p>

> echo "[[w:Wikipedia:Sandbox?action=edit]]" | php bin/parse.php --domain fr.wikipedia.org --wt2wt
[[w:Wikipedia:Sandbox?action=edit]]

seems ok.

MBinder_WMF added a project: Growth-Team-Filtering.Apr 15 2021, 7:00 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits