Page MenuHomePhabricator

VisualEditor doesn't show newly added references when using {{reflist}}
Closed, ResolvedPublicBUG REPORT

Description

Steps to replicate the issue
  • Create a page with the following minimal content on a wiki using the reflist-template (e.g. enwiki):
Test.<ref>Reference 1</ref>

== References ==
{{Reflist}}
  • Edit the page in VisualEditor and add another reference.
What happens?
  • The newly added reference doesn't appear in the reference list as long as you're editing (it shows in the reference pop-up).

Screenshot from 2026-05-01 20-50-38.png (553×222 px, 12 KB)

  • The reference is displayed after saving the page – and also when switching to wikitext (and back to VE) in the same edit session.
What should have happened instead?
  • Always show newly added references in the reference list while editing in VE.
Other information
Background

The Cite VisualEditor implementation tries to detect a references tag that's transcluded from a template. The code for that was mainly added and maintained for {{reflist}}[1]. The detection assumes a certain structure in the Parsoid output to work. It seems that two changes broke that detection recently and that's why the support for {{reflist}} on English Wikipedia degraded.

This is a minimal version of {{reflist}} to reproduce the issue:

<templatestyles src="Reflist/styles.css" /><div>{{#tag:references}}</div>

You also need to create the Reflist/styles.css file so that TemplateStyles does not throw errors.

You can also use this import:

Issue

The Parsoid output VE expects looks something like

<style data-mw-deduplicate="TemplateStyles:r2129" typeof="mw:Extension/templatestyles mw:Transclusion" about="#mwt2" 
       data-mw="{&quot;name&quot;:&quot;templatestyles&quot;,&quot;attrs&quot;:{&quot;src&quot;:&quot;Reflist/styles.css&quot;},&quot;parts&quot;:[{&quot;template&quot;:{&quot;target&quot;:{&quot;wt&quot;:&quot;reflist&quot;,&quot;href&quot;:&quot;./Template:Reflist&quot;},&quot;params&quot;:{},&quot;i&quot;:0}}]}"
       id="mwBw"></style>
<div about="#mwt2" id="mwCA">
	<div class="mw-references-wrap" typeof="mw:Extension/references" about="#mwt4"
	     data-mw="{&quot;name&quot;:&quot;references&quot;,&quot;attrs&quot;:{}}" id="mwCQ">

If the template maintainer adds a newline after the <templatestyles> tag this is already messing with how Parsoid parses that and VE's detection stops working. A change[2] on English Wikipedia already messed with that.

But furthermore a recent Parsoid change also "broke" the detection because it changed the output of the above template transclusion. So even with the "correct" template format it would not work anymore.

See the difference in the output with Version 0.23.0-a29

<div about="#mwt2" typeof="mw:Transclusion"
     data-mw="{&quot;parts&quot;:[{&quot;template&quot;:{&quot;target&quot;:{&quot;wt&quot;:&quot;reflist&quot;,&quot;href&quot;:&quot;./Template:Reflist&quot;},&quot;params&quot;:{},&quot;i&quot;:0}}]}"
     id="mwBw">
	<style data-mw-deduplicate="TemplateStyles:r2129" typeof="mw:Extension/templatestyles" about="#mwt3"
	       data-mw="{&quot;name&quot;:&quot;templatestyles&quot;,&quot;attrs&quot;:{&quot;src&quot;:&quot;Reflist/styles.css&quot;}}"></style>
	<div class="mw-references-wrap" typeof="mw:Extension/references" about="#mwt4"
	     data-mw="{&quot;name&quot;:&quot;references&quot;,&quot;attrs&quot;:{}}" id="mwCA">
Solution

TBD I guess we need to clearify what to expect from Parsoid and either improve the VisualEditor detection. Or both :-).

[1] https://gerrit.wikimedia.org/g/mediawiki/extensions/Cite/+/6388c77976b1c668838aed42468737733e1c03df/modules/ve-cite/ve.dm.MWReferencesListNode.js#63
[2] https://en.wikipedia.org/w/index.php?title=Template:Reflist&diff=prev&oldid=1327269895

Details

Related Changes in Gerrit:
Related Changes in GitLab:
TitleReferenceAuthorSource BranchDest Branch
Cite: Use a {{reflist}} template closer to the production versionrepos/wmde/docker-dev!6wmde-fischreflist-template-stylesmain
Customize query in GitLab

Event Timeline

Initial investigation by @WMDE-Fisch indicates that the issue might be related to recent changes to Parsoid.

Honestly I'm not so sure anymore. I'm currently having problems finding a version where it worked :-/

  • went back to REL1_43
  • used the Wikitext above
  • Template:Reflist
<templatestyles src="Reflist/styles.css" />
<div>{{#tag:references|{{{refs|}}}|group={{{group|}}}|responsive={{#if:{{{1|}}}{{{colwidth|}}}|0|1}}}}</div>
  • an empty Template:Reflist/styles.css

The issue remains.

Initial investigation by @WMDE-Fisch indicates that the issue might be related to recent changes to Parsoid.

Honestly I'm not so sure anymore. I'm currently having problems finding a version where it worked :-/

Based on T52769: Adding or updating a reference in VisualEditor does not update fake references blocks inside templates (resolved in 2018) and regression T310407: {{reflist}} no longer detected as a reference list and therefore updating as references are added (resolved in 2022) it used to work at some point.

This is a microcosm of a larger problem, which is that so much of how articles are formatted is cargo culting. What we're talking about here is essentially an API. VE exposes some functionality for managing references, so it needs a well-defined way for an editor to say "this is a reference", "this is where the list of references goes", etc. Then there's a fixed target that people can aim for. The folks who maintain VE. The folks who maintain the myriad of other reference-adjacent tools. The folks who write templates. And just plain editors who prefer to hand-edit wikitext. Then when something breaks, instead of finger-pointing about whether the problem is a VE update that was rolled a few days ago or a template change that was made back in December, everybody can point to the spec and say "This piece of the puzzle violates the spec: go fix that piece".

awight merged https://gitlab.wikimedia.org/repos/wmde/docker-dev/-/merge_requests/6

Cite: Use a {{reflist}} template closer to the production version

The spec for what Parsoid generates inside a transcluded block is https://www.mediawiki.org/wiki/Specs/HTML/2.8.0#Contents_of_transcluded_blocks so it should be possible to use that spec to reliably detect a <references/> node by the typeof="mw:Extension/references". We believe that WMDE is working on a path to make the detection of <references> inside VE more robust.

We're going to untag Content-Transform-Team from this issue as it is being fixed in Cite/VE. Feel free to retag us or find us on slack etc if you need more assistance from our team.

I did a bit of testing, and in an article with a single reference and a {{reflist}} I get the following (trimmed for legibility) HTML:

<span class="mw-empty-elt" about="#mwt7" typeof="mw:Transclusion" data-mw='{"parts":[{"template":{"target":{"wt":"Reflist","href":"./Template:Reflist"},"params":{},"i":0}}]}' id="mwDw">
    <style data-mw-deduplicate="TemplateStyles:r1327269900" typeof="mw:Extension/templatestyles" about="#mwt8" data-mw='{"name":"templatestyles","attrs":{"src":"Reflist/styles.css"}}'>...</style>
</span>
<div about="#mwt7" id="mwEA">
    <div class="mw-references-wrap" typeof="mw:Extension/references" about="#mwt9" data-mw='{"name":"references","attrs":{"group":"","responsive":"1"},"body":{"extsrc":""}}' id="mwEQ">
        <ol class="mw-references references" id="mwEg">
            <li about="#cite_note-1" id="cite_note-1" data-mw-footnote-number="1">
                <span class="mw-cite-backlink" id="mwEw">...</span>
                <span id="mw-reference-text-cite_note-1" class="mw-reference-text reference-text">
                    <style data-mw-deduplicate="TemplateStyles:r1333433106" typeof="mw:Extension/templatestyles mw:Transclusion" about="#mwt4" data-mw='...' id="mwFg">...</style>
                    <cite class="citation web cs1" about="#mwt4" id="mwFw">...</cite>
                    <span title="..." class="Z3988" about="#mwt4" id="mwHA"></span>
                </span>
            </li>
        </ol>
    </div>
</div>

The issue that the MWReferenceListNode detection is having with this is that it doesn't expect that span-wrap around the templatestyle. It assumes that when it's asked about the element, the mw:Transclusion element will also have typeof mw:Extension/templatestyles. Because the wrapper means this is not true, it doesn't think it's dealing with the templatestyles case, and it abandons as soon as it realizes the initial span isn't itself a reflist of a direct wrapper of a reflist.

The support for reflists with templatestyles was added back in T310407 (via patch) in 2022, and it was probably working somewhat-recently just because I'm fairly sure we tested it when we added the edit-refs-from-the-reference-list feature in 2023.

So, theory: the span wrapper around the templatestyles <style> is new, either from Parsoid or from TemplateStyles itself.

Change #1285397 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/Cite@master] MWReferencesListNode: fix templatestyles detection

https://gerrit.wikimedia.org/r/1285397

^ that makes the detection work once more. It might be being too flexible now, because I'm not sure whether the wrapping span is guaranteed to be present so I made it work either way.

From the spec @cscott referenced:

We don't guarantee that transcluded content will have exactly the same <span> wrappers as top-level content; for example as noted above internal template boundaries are not marked. Similarly, additional <span> wrappers may be added; for example to accommodate trailing categories (phab:T378906) and those wrappers could span content from multiple child templates since internal template boundaries are not tracked.

So I think that the previous and current markup are both within the behavior the spec allows, and we just need to make sure to cope with any of the top-level parts of the transclusion about-group potentially being wrappers.

Change #1285397 merged by jenkins-bot:

[mediawiki/extensions/Cite@master] MWReferencesListNode: fix templatestyles detection

https://gerrit.wikimedia.org/r/1285397

DLynch claimed this task.

Sorry, forgot to close this because of the hackathon.