Purge all old html version code from Parsoid repo once we switch VE from RESTBase to ParserCache
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	ssastry
	May 3 2023, 9:29 AM

Description

RESTBase is a store, not a cache. As such, there was on reliable way of knowing when old HTML versions would clear out from it and we also didn't have a mechanism to purge old versions. As such, Parsoid's codebase has a number of FIXMEs / TODOs to get rid of some code when the HTML for a version turns over in RESTBase.

But, we are now moving away from RESTBase to ParserCache where all entries turn over after X days (currently X = 21). So, once we fully switch over all wkies to ParserCache, after that X-day period, we should be able to get rid of all the b/c code in Parsoid that exists to deal with older HTML versions.

Details

Related Changes in Gerrit:

Subject	Repo	Branch	Lines +/-
Bump wikimedia/parsoid to 0.22.0-a2	mediawiki/vendor	master	+645 -662
Remove inHTMLPre	mediawiki/services/parsoid	master	+2 -22
Bump wikimedia/parsoid to 0.21.0-a19	mediawiki/vendor	master	+73 -77
Remove html pre b/c	mediawiki/services/parsoid	master	+0 -29
Bump wikimedia/parsoid to 0.21.0-a14	mediawiki/vendor	master	+352 -704
Remove a bunch of backwards compatibility	mediawiki/services/parsoid	master	+18 -446
WIP: Purge b/c compatibility code for old HTML versions	mediawiki/services/parsoid	master	+18 -190

Customize query in gerrit

Related Objects
Search...

Status	Subtype	Assigned	Task
Resolved		ABreault-WMF	T335843 Purge all old html version code from Parsoid repo once we switch VE from RESTBase to ParserCache
Open		None	T209120 Upgrade Parsoid HTML stored in the StructuredDiscussions tables
Open		None	T174372 Store content as wikitext, not HTML
Open		None	T174374 Use a mixed storage format, setting the system to slowly migrate over from HTML to wikitext storage
Declined		Trizek-WMF	T178390 Support Structured discussion storage format change
Declined	BUG REPORT	None	T93457 Babel templates do not render correctly in posts and boards provided by StructuredDiscussions
Open		None	T93838 Add CSS modules that Parsoid tells us are relevant

Event Timeline

ssastry created this task.May 3 2023, 9:29 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 3 2023, 9:29 AM

ssastry added projects: RESTBase Sunsetting, Content-Transform-Team-WIP.May 3 2023, 10:32 AM

ssastry updated the task description. (Show Details)

MSantos moved this task from Needs Triage to Tech Debt / Big changes on the Parsoid board.May 4 2023, 2:12 PM

daniel triaged this task as Low priority.Jun 5 2023, 6:17 PM

daniel moved this task from Unsorted to Parsoid pile on the RESTBase Sunsetting board.

Change 930649 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/services/parsoid@master] WIP: Purge b/c compatibility code for old HTML versions

https://gerrit.wikimedia.org/r/930649

gerritbot added a project: Patch-For-Review.Jun 15 2023, 3:01 PM

DAlangi_WMF awarded a token.Jun 15 2023, 3:01 PM

Change 930649 abandoned by Subramanya Sastry:

[mediawiki/services/parsoid@master] WIP: Purge b/c compatibility code for old HTML versions

Reason:

Arlo pointed out that this is also blocked on https://phabricator.wikimedia.org/T174372 .. our dear friend Flow!

https://gerrit.wikimedia.org/r/930649

Maintenance_bot removed a project: Patch-For-Review.Jun 15 2023, 3:11 PM

Arlolra merged a task: T203211: See what backwards compatibility code can be safely removed.Jun 15 2023, 3:17 PM

Arlolra added subtasks: T209120: Upgrade Parsoid HTML stored in the StructuredDiscussions tables, T174372: Store content as wikitext, not HTML, T174374: Use a mixed storage format, setting the system to slowly migrate over from HTML to wikitext storage.

Arlolra added subscribers: Arlolra, RhinosF1.

ssastry removed a subtask: T174374: Use a mixed storage format, setting the system to slowly migrate over from HTML to wikitext storage.Jun 15 2023, 5:01 PM

ihurbain added a project: Essential-Work.Apr 18 2024, 2:29 PM

MSantos removed a project: Content-Transform-Team-WIP.Aug 22 2024, 2:11 PM

Change #1102282 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/services/parsoid@master] [WIP] Remove a bunch of backwards compatibility

https://gerrit.wikimedia.org/r/1102282

gerritbot added a project: Patch-For-Review.Dec 11 2024, 12:12 PM

ABreault-WMF claimed this task.Dec 11 2024, 2:26 PM

In T209114, Flow started storing Parsoid's content version number, which was 2.0.0 at the time. The code that's gated on content versions before that seem like they would be no-ops now. That's mainly,

Whitespace heuristics, from before 1.7.0, which might mean superfluous whitespace when serializing wikitext characters

Otherwise, in T335843#10396771, we're dropping,

Old LST serialization, from before 1.3.0, which doesn't seem like it would be used on Flow boards
stx_v, from before 1.5.0, which at worst would mean serializing table rows to a newline
mw:DisplaySpace no longer sets mw:Placeholder as well. This was to avoid selser diffs during a two phase deployment. Selser shouldn't be relevant for Flow, regardless
A briefly deployed AddRedLinks bug that is unlikely have affected many posts
Support for the li hack, which at worst would drop the liHackSrc when serializing
Old rendering of mw:PageProp/defaultsort and mw:PageProp/displaytitle, which don't seem like they'd be used on Flow boards