Hi, it looks like this is reaching the stage at which Data Engineering and Data Persistence SRE can help!

Specifically: we can help the Editing team surface requirements and co-write a technical design that addresses the deliverables for this ticket, and more: see Prep Pantry, the onboarding process we're soft-launching.

Naturally, many Suggestions will be ML-generated, and we'll need to team up here. Harmonizing our processes and responsibilities is something @Sucheta-Salgaonkar-WMF and I are actively trying to figure out.

It looks like you'll be scheduling this spike starting next week; if you're ready at that point, we can work together to document requirements starting then?

Current architecture

EditCheck is a client-side system for giving feedback about the current state of the document. It does this by examining the VisualEditor data model for a document, and outputting a series of data-ranges within that model to which checks/suggestions should apply. This is regularly recalculated as the model is changed during an edit session, such that the checks remain applicable to the current state of the document.

This is currently no storage of any data outside of the edit session.

There is currently no way to access suggestions without the VisualEditor data model being loaded for evaluation.

Suggestions exposed outside of VisualEditor

There are two plausible approaches for showing suggestions outside of VisualEditor:

1. Creating storage for suggestions
2. Loading VisualEditor in the background and generating the suggestions

Storage

We could create a storage system for check-data. Or use Multi-Content Revisions derived slots, perhaps? Once this existed, other tools would be able to query whether a given revision had any associated checks.

The difficulty of this approach depends on whether VisualEditor is the only input into this storage. If it is, it would presumably be essentially serializing unresolved checks/suggestions at the time of save so that other tools could later access them. If it's not, other tools would need to be able to produce data in a way that's legible to VisualEditor.

Pros:

1. Suggestions are widely available and we can funnel people towards articles that need work
2. Extremely expensive checks that're impractical to run client-side could be created

Cons:

1. Storage would need to be invalidated on every revision.
2. VisualEditor is not the only way to edit a page, so new revisions would not automatically have stored-from-VisualEditor suggestions.
3. Non-VisualEditor tools will have a hard time pointing to content in a way that VisualEditor can use
4. If suggestions are being generated in ways that cannot be replicated client-side, we can't show these as checks in response to the user's immediate edits
5. If suggestions are being generated in ways that cannot be replicated client-side, VisualEditor will have a hard time showing them to people once they've made any changes to the article within their session.

To illustrate that last point, consider a suggestion that has been generated server-side which has checked the validity of a reference. Once the user has edited anywhere near that reference, we'd no longer know whether it's still invalid -- maybe they fixed it by updating the article to match the reference. (We see the limitations on this from the Tone model, where even if we started with a list of tone-violating paragraphs we have to keep on checking back in with the model when the user changes things to see whether they've actually fixed the issue.)

Background loading

It is possible to load a headless VisualEditor data model and generate checks from it. This requires that there be Parsoid HTML available, so this would be significantly simpler once Parsoid read-views are available.

(Technically this could be done on the server via running VisualEditor inside node or similar, but we have not explored this at all so there could be any number of technical issues with that.)

Pros:

1. Always available on every article.
2. Always up-to-date with the state of the article/.
3. Easy to pass through into editing with VisualEditor, or saving directly without technically launching VisualEditor.

Cons:

1. We haven't run VisualEditor headless like this before, so there might be issues where our code expects the view / UI to exist.
2. We haven't attempted to bootstrap a VE data model from Parsoid read-mode HTML yet, so there might be missing essential data.
3. Repeats a lot of work for every client rather than centralizing it.
4. If Parsoid read-views aren't available would require double-loading each page, essentially.
5. This would only make checks available in a limited way; we wouldn't be able to have a dashboard showing check on many pages.

Open questions

Is the requirement that we expose suggestions from within VisualEditor elsewhere, or that we allow others to contribute suggestions for use within VisualEditor without having to write client-side code themselves?

The biggest constraint on what we might create is other tools having a way to reference specific ranges in the VisualEditor data model. We could investigate:

1. An agreed-upon spec for mapping between offsets in wikitext and the VisualEditor data model (note: this is not one-to-one)
2. An official tool for interacting with the Parsoid DOM that could output VisualEditor offsets

It might be worth looking into headless VisualEditor client-side to unlock passive read-mode suggestions, and server-side to unlock generating suggestions on articles which weren't last-edited with VisualEditor.

Effort

Complicated, since there's a wide range of possible approaches. In rough order of difficulty:

1. Serializing unaddressed checks/suggestions into MCR derived slots would be pretty trivial (based on my understanding of that system), and if we scoped it all down to ignoring any time that the last editor wasn't VE, we could probably have a basic data-source available quickly. If we wanted this to be independent of client-side VE, or interpretable by other users outside of a basic "there are a few suggestions here, follow this link to see what they are", the effort rapidly increases.
2. Most likely to be something we could hack-together quickly would be the "background VE" approach. Assuming we didn't discover any disqualifying technical issues, we could do this with our current technical stack as an expansion of suggestions into articles, and I suspect we could have a very unpolished version put together in a week or two.
3. Bespoke server-side client-side model suggestions that we build in parallel (e.g. the Add A Link suggestions) are a known path, but require a bunch of individual work to take a server-side data source and use our client-side tech to match it up with the article in VE in ways that're highly specific to the given check. Individual effort varies, but this doesn't really scale to being reusable.
4. A system where a server-side model can insert suggestions and client-side code that doesn't know about that specific suggestion can pick it up and present it to the user inside VE. (This is the one that requires us to solve all the "the server knows how to express VE ranges" / "decide what disqualifies a suggestion as the document is edited" questions that're fairly open-ended.)

Next steps

• @ppelberg to review and define what the next steps here. Will do after this week's meeting with @GGoncalves-WMF and DPE folks.