Prototype selective HTML updates in Parsoid
Open, HighPublic
Actions

Assigned To

Authored By

	ABreault-WMF
	Apr 24 2024, 10:48 PM

Description

From WE5.3 Draft,

Draft hypothesis idea: On template edits, if we can implement an algorithm in Parsoid to reuse HTML of a page that depends on the edited template without processing the page from scratch and demonstrate 1.5x or higher processing speedup, we will have a potential incremental parsing solution for efficient page updates on template edits.

NOTE: We are only planning to implement this in the Parsoid library and test it on the command line. The actual integration with the processing pipeline will be followup work and will be more involved. In this prototype, we will start with templates that produce well-balanced DOM fragments.

Details

	Subject	Repo	Branch	Lines +/-
	[WIP] Selective HTML Updates	mediawiki/services/parsoid	master	+290 -88

Customize query in gerrit

Related Objects

Mentioned In: T361975: [Session] 🥳 Wikimedia Hackathon 2024 Project Showcase & Closing

Event Timeline

ABreault-WMF created this task.Apr 24 2024, 10:48 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 24 2024, 10:48 PM

ABreault-WMF moved this task from Backlog to Hacking projects on the Wikimedia-Hackathon-2024 board.Apr 24 2024, 10:48 PM

Lofhi subscribed.Apr 25 2024, 12:52 AM

SDelbecque-WMF subscribed.May 2 2024, 6:40 PM

ABreault-WMF renamed this task from Prototype incremental parsing in Parsoid to Prototype selective HTML updates in Parsoid.May 3 2024, 2:59 PM

Change #1026985 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/services/parsoid@master] [WIP] Selective HTML Updates

https://gerrit.wikimedia.org/r/1026985

gerritbot added a project: Patch-For-Review.May 3 2024, 4:54 PM

@ssastry reports on Slack,

very early iniital results show 5x-10x speedup on reparsing of pages after template edits (as reproduced on our local wikis).
╰─➤  time MW_INSTALL_PATH=../core php bin/parse.php --integrated --pageName "Hampi"  < /dev/null > /tmp/hampi.html
 MW_INSTALL_PATH=../core php bin/parse.php --integrated --pageName "Hampi" <    3.07s user 0.24s system 71% cpu 4.618 total
╭─subbu@earth ~/work/wmf/parsoid  ‹T363421*› 
╰─➤  time MW_INSTALL_PATH=../core php bin/parse.php --integrated --pageName "Hampi" --selpar --revtextfile /tmp/hampi.wt --revhtmlfile /tmp/hampi.html --editedtemplatetitle "About" < /dev/null > /tmp/hampi.new.html
 MW_INSTALL_PATH=../core php bin/parse.php --integrated --pageName "Hampi"      0.36s user 0.05s system 89% cpu 0.459 total
Rest of this work will happen in Q1. Finishing up, addressing FIXMEs, polishing it up, adding tests will probably take at least be a week's worth full time work.

Don-vip awarded a token.May 5 2024, 6:16 PM

Lofhi awarded a token.May 5 2024, 6:17 PM

Quiddity mentioned this in T361975: [Session] 🥳 Wikimedia Hackathon 2024 Project Showcase & Closing.May 13 2024, 6:51 PM

• ssastry moved this task from Needs Triage to Feature requests on the Parsoid board.Thu, May 23, 3:51 PM

• ssastry added a project: Content-Transform-Team-WIP.

MSantos added a project: OKR-Work.Mon, Jun 3, 2:31 PM

ABreault-WMF triaged this task as High priority.Sat, Jun 15, 1:08 AM