Page MenuHomePhabricator

Make RelatedArticles extension compatible with Parsoid
Closed, ResolvedPublic

Description

Parsoid has its own extension API - see https://www.mediawiki.org/wiki/Parsoid/Extension_API.

The RelatedArticles extension needs an update to work directly with Parsoid so that we can switch Wikimedia wikis to use Parsoid instead of core parser in late 2021.

The Parsing Team will work with you as required.

Looking at the code, it appears that it uses the setFunctionHook API. ParsoidExtensionAPI is yet to add this support. But, looking at the extension code, once that is done, the actual retargeting work should be quite straightforward and simple.

Requirement

Update the RelatedArticles extension to be compatible with Parsoid, ensuring that it functions correctly on Wikimedia wikis as they transition from the core parser to Parsoid.

BDD

Feature: Make RelatedArticles Extension Compatible with Parsoid

  Scenario: Display related articles using Parsoid
    Given the user is viewing a page on a wiki using Parsoid
    When the user scrolls to the bottom of the page
    Then more than one related article should be rendered

Test Steps

(Production only)

Test Case 1: Verify Related Articles Rendering with Parsoid

  1. Visit India on Wikivoyage in production.
  2. Scroll to the bottom of the page.
  3. AC1: Confirm that more than one related article is rendered.

QA Results - Prod

ACStatusDetails
1T263772#10055747

Event Timeline

Jdlrobson added subscribers: JTannerWMF, ovasileva, Jdlrobson.

Please contact Jazmin or Olga to set expectations about timeline and ensure this work gets scheduled.

Yes web team maintain this extension. Please reach out to @ovasileva to make sure time is allocated to do this work.

Jdlrobson changed the task status from Open to Stalled.Nov 29 2023, 12:22 AM

Also currently it's not clear what exactly you need the web team to do - from the ticket my understanding is ParsoidExtensionAPI needs to add support for setFunctionHook API before we can go anything. Is there a ticket tracking that work?

The RelatedArticles extension registers a parser function for {{#related|page1|page2}}. That should be supported with Parsoid currently, and the plan (T268144#7950406) is to maintain legacy compatibility for these simple text-in/text-out parser functions (although the legacy API might shift to get rid of the ParserFIrstCallInit hook, but that's an orthogonal issue, T299528).

The main incompatibility I can see is in:

	public static function onFuncRelated( Parser $parser, ...$args ) {
		$parserOutput = $parser->getOutput();
		$relatedPages = $parserOutput->getExtensionData( 'RelatedArticles' );
		if ( !$relatedPages ) {
			$relatedPages = [];
		}

		// Add all the related pages passed by the parser function
		// {{#related:Test with read more|Foo|Bar}}
		foreach ( $args as $relatedPage ) {
			$relatedPages[] = $relatedPage;
		}
		$parserOutput->setExtensionData( 'RelatedArticles', $relatedPages );

		return '';
	}

The use of getExtensionData and setExtensionData in this read-modify-write sort of way is deprecated (T300981) and because Parsoid processes the parser functions invocations independently will currently result in throwing away the data from all but the last invocation of {{#related}}. If there's typically only one invocation per page that might not be immediately obvious.

In any case, the fix is simple: the new-ish ParserOutput::appendExtensionData was added specifically to allow composition of lists of this sort. Switching to use that function should eliminate the Parsoid incompatibility AFAICT.

Do you just need code review/testing for this? If the fix is simple it is something the content transform team plans to post patches for?

Do you just need code review/testing for this? If the fix is simple it is something the content transform team plans to post patches for?

That's a good question! Ideally we would work in a way that shares knowledges about the Parsoid APIs with everyone involved since Parsoid is expected to be the long-term MW parser.

My idea is to prioritise CTT patching for unmaintained Extensions and support owners to work on their extensions with our review/support. How that sounds to you?

Jdlrobson changed the task status from Stalled to Open.Dec 5 2023, 5:33 PM
Jdlrobson assigned this task to ovasileva.

Yep thanks! Passing to Olga to prioritize then.

ovasileva triaged this task as Medium priority.Dec 5 2023, 5:34 PM
Jdlrobson raised the priority of this task from Medium to High.Jul 30 2024, 6:02 PM

https://en.wikivoyage.org/wiki/India?useparsoid=1 is only showing the first related article, not 6 that are in the article (https://en.wikivoyage.org/wiki/India?useparsoid=0)

Change #1058236 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/RelatedArticles@master] Use `appendJsConfigVar` to track related page titles

https://gerrit.wikimedia.org/r/1058236

Change #1058236 merged by jenkins-bot:

[mediawiki/extensions/RelatedArticles@master] Use `appendJsConfigVar` to track related page titles

https://gerrit.wikimedia.org/r/1058236

@cscott do you think we should arrange a cache flush with traffic after next week's deployment? The project seems small enough that it would be acceptable and that way we don't have pages in a broken state for 2 weeks...

Edtadros subscribed.

Test Result - Prod

Status: ✅ PASS
Environment: enwikivoyage
OS: macOS Sonoma
Browser: Chrome
Device: MBA
Emulated Device: NA

Test Artifact(s):

Test Steps

Test Case 1: Verify Related Articles Rendering with Parsoid

  1. Visit India on Wikivoyage in production.
  2. Scroll to the bottom of the page.
  3. AC1: Confirm that more than one related article is rendered.

screenshot 1.png (932×1 px, 318 KB)