Following up here on a thread I started at T431603#12101539.

@Ottomata wrote:

How much of 'VE data model' or 'edit check logic' can be extracted out of the browser and into callable JavaScript functions/libraries? Instead of duplicating code, could a common JS library be used both server side and in a browser?

@DLynch wrote:

as we're in a context where there's a DOM available, we can spin up the current VE data model from an HTML string, and just not instantiate the UI layers.

Like https://github.com/cheeriojs/cheerio ? I know nothing please ignore me. :D

a lot of "what should actually be loaded?" requires knowledge from mediawiki

I'm curious, can you give an example of what knowledge from MW is needed to generate edit suggestions? I understand complexities of TextMatch so let's exclude that one.

I'm asking because I sympathize with the duplicate code problem. If the edit check logic for simpler could be fully extracted into a function like getDoubleBoldSuggestions(revision_html) (or maybe getDoubleBoldSuggestions(dom_root), or whatever) then maybe the logic can be run in both VE as well as Node?

I'm curious, can you give an example of what knowledge from MW is needed to generate edit suggestions? I understand complexities of TextMatch so let's exclude that one.

I was mostly referring there to the actual literal bits of VisualEditor which drive certain features. E.g. references, as a thing in the data model, are defined in the Cite extension (see also: Math, Chart, Music, etc etc etc). So for a given wiki, "what extensions are enabled, and what resourceloader modules do they register as a VisualEditor PluginModules attribute in their extension.json is something we'd need to know to spin up a VE data model that correctly represents that wiki's HTML.

If the edit check logic for simpler could be fully extracted into a function like getDoubleBoldSuggestions(revision_html) (or maybe getDoubleBoldSuggestions(dom_root), or whatever) then maybe the logic can be run in both VE as well as Node?

That'd actually be a bit of a pain, because at the point where suggestions are being generated and shown inside VE, there's not something that's like a Parsoid DOM to be used to generate them. We deserialize from that when the document loads, and serialize back down when we're saving it. Refactoring everything so it worked off of that would make the in-VE experience a lot more expensive and slow.

getDoubleBoldSuggestions(revision_html) (or maybe getDoubleBoldSuggestions(dom_root)

That'd actually be a bit of a pain, [...]

Ah well, I suppose if

as we're in a context where there's a DOM available, we can spin up the current VE data model from an HTML string

works, then this could be getDoubleBoldSuggestions(ve_document_model) then?

E.g. references, as a thing in the data model, are defined in the Cite extension

Hm! "as a thing in the data model", you mean VE's data model? Cite tells VE what a reference is? I know of work outside of VE for finding references in HTML that works outside of MW.

But ya

Math, Chart, Music, etc etc etc

Understood. What would be duplicated is you wouldn't be able to ask these extensions for their definition of what a Chart looks like in Parsoid HTML, right?

FWIW, this is pretty common in data platform world where we don't have MW. Folks mostly just develop separate algorithms to scrape needed data out of HTML.

So for a given wiki, "what extensions are enabled, and what resourceloader modules do they register as a VisualEditor PluginModules attribute in their extension.json is something we'd need to know to spin up a VE data model that correctly represents that wiki's HTML.

Are these things needed to generate the simple(r) edit suggestions? Could a default pluginless VE still do a lot?

I understand VE does a ton of fancy stuff with all the per wiki (and user!?) context it
has. And apologies for all my ignorant questions...I've never looked under the hood of VE ;)

Hm! "as a thing in the data model", you mean VE's data model? Cite tells VE what a reference is? I know of work outside of VE for finding references in HTML that works outside of MW.

Yeah, here's the VE datamodel being told about <ref> in the Cite extension: https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/Cite/+/refs/heads/master/modules/ve-cite/ve.dm.MWReferenceNode.js

Are these things needed to generate the simple(r) edit suggestions? Could a default pluginless VE still do a lot?

VE with no other extensions could do most of the "simpler logic" suggestions from the description. About half if we don't also assume that a mw.Api is available.

(Unfortunately, based on my browsing experience with suggestion mode turned on, the ones that extensionless and API-less VE can get are also really uncommon.)

Yeah, here's the VE datamodel being told about <ref> in the Cite extension:

Specifically the VE converter, which converts back and forth between Parsoid HTML and VE data model - will look up every DOM node in the Parsoid HTML using the modelRegistry to determine what VE type a specific DOM Node is. Every node is registered this way (e.g. paragraphs, headings in core), and extensions can register new handlers, so Cite tells the model registry to convert anything with a typeof of mw:Extension/ref to an mwReference node.

Without the Cite-VE plugin loaded, it would get converted to a generic uneditable alienNode (or maybe handled as a generic span/link).