In T7984#1967520, @TheDJ wrote:

@Cenarium, I think such an approach would be a nice general solution, but I don't think page_props is particularly suited for this purpose. Perhaps a memcached backed cache + restbase api in front of it ?

Regarding page_props, it looks like the graph extension had the same size issue in T100942, and TemplateData in T53740. It was mostly fixed by compressing the json. Could this be done here ? I've looked at some of the most pathological enwiki articles, and there are at least a couple dozen articles whose gz-compressed json-encoded mRefs (member variable of the Cite class) are in excess of 64 KB. So indeed, it might not work here.
I'm not familiar with the restbase API. Memcache could be a good idea: each ref could be stored in its own cache identified by the article id and the ref key, or in a single cache of all refs for the article id.

And even then, in the context of section editing, how would you deal with the problem of synchronizing the pre and post edited version of it ?

Refs from the edited section as retrieved by the preview parser would be checked first, and only when one has missing text the backed-up ref would be fetched.
The only potential issue I can think of is if a ref that was defined in the section gets removed but is still called in the section: there would be no error in the preview (only after save). But it is anyway inadvisable to remove a named ref definition in section editing since it might be used elsewhere in the article.

@Cenarium great! Let me know when you need help or review.

Change 267515 had a related patch set uploaded (by Cenarium):
Show refs defined elsewhere in section preview

https://gerrit.wikimedia.org/r/267515

Could there be some indication in the rendering of the particular citation defined elsewhere that indeed the citation is defined elsewhere? Basically, the current message (or shortened?) plus the content of the citation. Alternatively, a span with class "ref_defined_elsewhere" or similar would allow me to manipulate CSS to display a notification.

Also, it seems like the functionality requested in this change would make previewing a section where <references/> lives possible. Is that the case?

I could add such a notice. Wouldn't it be better in parenthesis after the reference text ?

Placement-wise, I have no preference. Before makes it more "obvious" it's defined external to the section, after makes it less "disruptive", which is probably the other reason it would be good to have a span of some sort around the text. <span class="cite-defined-external">Defined in external section.</span> seems about right.

I've added specific classes for each cite warning. In this case, the class is "mw-ext-cite-cite_warning_sectionpreview_outside".

Why the references list is displayed as unordered list here: https://de.wikipedia.org/w/index.php?title=Nostradamus&oldid=151227799&action=submt&section=14

This will be fixed along T125981.

Mashing this into page_props strikes me as a bad idea, especially if it needs to be broken up into several parts.

How about this: store the required info in the ParserOutput (with setExtensionData - if this isn't the case already), and then, when previewing a section, fetch current ParserOutput from the ParserCache, grab whatever info is needed for rendering (and missing from the section), and use it to render the citation.

PageProps is *not* a good place for big JSON blobs.

In T124840#2653575, @daniel wrote:

Mashing this into page_props strikes me as a bad idea, especially if it needs to be broken up into several parts.

How about this: store the required info in the ParserOutput (with setExtensionData - if this isn't the case already), and then, when previewing a section, fetch current ParserOutput from the ParserCache, grab whatever info is needed for rendering (and missing from the section), and use it to render the citation.

PageProps is *not* a good place for big JSON blobs.

I agree, actually I made several commits that did that a while ago, including https://gerrit.wikimedia.org/r/#/c/277445.
I've found out though that the best way to go about it (have a fully and efficiently rendered html) is to create a custom parser cache for Cite, since this requires way too much customization (would need plenty of hooks otherwise). This is done in https://gerrit.wikimedia.org/r/#/c/278703 - I got this to work well. It also completely solves the problem of outdated parses.
I've just rebased those, I'd welcome any feedback :)

@Cenarium Looks like it's worth further investigation. The interaction with FlaggedRevisions is somewhat ugly, though. The relevant revisions is always the edit's base revision. That would typically the the current revision, not the last flagged revision, right?

In T124840#2656486, @daniel wrote:

@Cenarium Looks like it's worth further investigation. The interaction with FlaggedRevisions is somewhat ugly, though. The relevant revisions is always the edit's base revision. That would typically the the current revision, not the last flagged revision, right?

The interaction with FlaggedRevs is designed for MobileFrontend, specifically when an unregistered user views a page with flagging enabled, hence the 'stable' revision; it isn't related to section preview. Section preview doesn't even need the separate parser cache, it's only needed for MobileFrontend.

@Cenarium @TheDJ are you going to be at the Barcelona hackathon? If yes, we could try to get this task rolling again?

Just for reference, see also T146072: Parsoid DOM spec: Improve Cite representation to support section editing & simpler clients. It *may* be that we can collaborate on improving the infrastructure for citations to solve both of our problems at once? (/me dreams)

Change 267515 abandoned by Thiemo Kreuz (WMDE):
Show refs defined elsewhere in section preview

Reason:
This proposal relies heavily on a feature that was never enabled on any production wiki, and removed recently. If we are going to fix this, which would certainly be nice, we would need to re-do this patch anyway.

https://gerrit.wikimedia.org/r/267515