Page MenuHomePhabricator

After fixing subpage links in Parsoid read views, TOC links are broken
Closed, ResolvedPublic

Description

Parsoid usually sets a <base href> in the document, but when embedded page in an article this doesn't exist so our links are all broken.

See the [[Opportunities for the Content Transform Team]] link at https://en.wikipedia.org/wiki/User:Cscott/Ideas?useparsoid=1

EDIT: After adding the missing <base href>, all TOC links are broken; see comment below.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

I think we should fix this sooner than later .. I suspect officewiki has subpage links.

The solution to this might also fix T310520, if we're implementing a parsoid-read-views-specific href rewriting postprocessing pass.

Things to be careful of:

  • Don't break VE or other Parsoid clients who probably rely on href="./ in edit-mode HTML
  • Don't break links in the skin (ie, can we "just" change the <base href> on the page like parsoid does, or will that break other stuff surrounding the article?)

Change 973361 had a related patch set uploaded (by Isabelle Hurbain-Palatin; author: C. Scott Ananian):

[mediawiki/core@master] Fix Parsoid <base href>

https://gerrit.wikimedia.org/r/973361

Change 973361 merged by jenkins-bot:

[mediawiki/core@master] Fix Parsoid <base href>

https://gerrit.wikimedia.org/r/973361

cscott renamed this task from Parsoid read views don't handle subpage links to After fixing subpage links in Parsoid read views, TOC links are broken.Nov 15 2023, 3:54 PM
cscott updated the task description. (Show Details)
MSantos triaged this task as High priority.Nov 16 2023, 3:20 PM
MSantos edited projects, added: Content-Transform-Team-WIP; removed: Content-Platform-Team.
MSantos moved this task from Needs Triage to Bugs & Crashers on the Parsoid board.

Change 975051 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/core@master] [parsoid] Fix Parsoid relative links

https://gerrit.wikimedia.org/r/975051

Change 975081 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/core@master] [parsoid] Fix Parsoid relative links, take 2

https://gerrit.wikimedia.org/r/975081

Change 975051 merged by jenkins-bot:

[mediawiki/core@master] [parsoid] Fix Parsoid relative links

https://gerrit.wikimedia.org/r/975051

MSantos moved this task from Backlog to Code Review on the Content-Transform-Team-WIP board.
MSantos moved this task from Code Review to To Deploy on the Content-Transform-Team-WIP board.
MSantos moved this task from To Deploy to Code Review on the Content-Transform-Team-WIP board.

Change 975081 abandoned by C. Scott Ananian:

[mediawiki/core@master] [parsoid] Fix Parsoid relative links, take 2

Reason:

Abandoned in favor of I60165a9946a35cfb42a78ed2f833c34570fefffc

https://gerrit.wikimedia.org/r/975081

Change 975596 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/core@wmf/1.42.0-wmf.5] [parsoid] Fix Parsoid relative links

https://gerrit.wikimedia.org/r/975596

Change 975596 merged by jenkins-bot:

[mediawiki/core@wmf/1.42.0-wmf.5] [parsoid] Fix Parsoid relative links

https://gerrit.wikimedia.org/r/975596

Mentioned in SAL (#wikimedia-operations) [2023-11-20T22:27:22Z] <catrope@deploy2002> Started scap: Backport for [[gerrit:975596|[parsoid] Fix Parsoid relative links (T350952)]]

Mentioned in SAL (#wikimedia-operations) [2023-11-20T22:28:40Z] <catrope@deploy2002> catrope and cscott: Backport for [[gerrit:975596|[parsoid] Fix Parsoid relative links (T350952)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2023-11-20T22:46:54Z] <catrope@deploy2002> Finished scap: Backport for [[gerrit:975596|[parsoid] Fix Parsoid relative links (T350952)]] (duration: 19m 32s)

Merged and backported to production; you might need ?action=purge on Parsoid Read Views pages for a while to get the sidebar links correct, since we need to regenerate the page without the <base href> in the ParserOutput::addHeadItems.

So, as part of reviewing T425211: Link colours don't change to purple in Safari on Parsoid-served pageviews, we decided to look at why we decided to introduce protocol-relative links instead of either retaining Parsoid's (the library) "./" relative links OR using absolute links, and this task is the source of those changes. I'm going to summarize the info I gathered so far (since I didn't remember the details and we don't want to go digging again).

  • This phab task was originally created as "Parsoid read views don't handle subpage links"
  • Parsoid HTML from the Parsoid library code generates a full doc with <head> and a base tag with a base href that is used to resolve the relative urls.
  • But, the core readview transform code (OutputTransformPipeline) didn't include these hrefs. So, Scott's first attempt in https://gerrit.wikimedia.org/r/975051 was to copy over the base href from Parsoid library's DOM to the read view DOM
  • On deploying this patch to the beta cluster, we found that TOC links were broken.
  • We repurposed this phab task and renamed it to "After fixing subpage links in parsoid read views, TOC links are broken" (in retrospect, we should have created a different phab task).
  • So, here is where we seem to have decided to merge https://gerrit.wikimedia.org/r/c/mediawiki/core/+/975051 which converts the relative links to protocol-relative links. We unfortunately didn't document this decision (here or in the commit message). Why did we decide to fix the TOC breakage with this solution instead of handling it in the TOC code? And, why did we decide to use protocol-relative urls instead of absolute urls?
  • It might be worth clarifying answers to those last two questions to figure out how to handle T425211.

My recollection is that it wasn't just TOCs, a lot of skin code and user gadgets also assumed that the base href of the page was unmodified. Setting the base href to something "unexpected" broke a *lot* of things on subpages; the TOC links were just the thing which got written into the phab task.

We could have tweaked parsoid to use ../ relative prefixes instead of converting to absolute links. We'd done that a lot during the early days of VE and had some reasonable confidence that VE would just ignore any (..?/)* prefix. But this was considered a breaking DOM change, even though it probably would have been the wiser long-term approach. We went back and forth between ./ and ../ many times during early VE development, and I don't remember all the factors that led us to where we are now. This would have been a more disruptive change. Also, it would break 3rd-party wikis which use a ?title=%s URL pattern. T310520: Parsoid content not compatible with `index.php?title=` URLs/T270219: VisualEditor does not support wikis without short URLs

The "convert to absolute URLs" code was already in the postprocessing pipeline, and we were backporting fixes; in that environment "just" using the existing pass to convert all of the URLs seemed like the safer immediate fix.

wrt protocol-relative URLs: in this case Parsoid (and the postprocessing code) is just following the mediawiki default. There were good reasons to use protocol-relative URLs when not all WMF wikis were https-only. Those reasons are years in the past, and WMF could/should probably change its defaults to http (T54253: Protocol-relative URLs are poorly supported or unsupported by a number of HTTP clients). That's a mediawiki-config change, though, not a Parsoid change (T118413: Wikimedia wikis should use https:// in $wgServer) and there's a bunch of code that needs to be fixed before we can do this: T118413#10973765