Page MenuHomePhabricator
Create Task
Maniphest T397501

[Epic] Fix issues with {{reflist}} or missing references lists and main+details sub-references
Closed, ResolvedPublic
Actions

Assigned To
None
Authored By
WMDE-Fisch
Jun 20 2025, 8:38 AM
Project Tags
Referenced Files
None
Subscribers
Aklapper
awight
Hakimi97
Johannes_Richter_WMDE
Lina_Farid_WMDE
matmarex
RoySmith
View All 10 Subscribers

Description

Context

This is meant as an umbrella ticket to manage WMDE-TechWish 's refactoring work in Cite and VE to fix know issues with the {{reflist}} template or missing references lists and main+details[1] sub-references. The main reason for these issues is the current implementation of how a reference including main and details content is handled in VE. Currently this is done by adding a "synthetic list defined" MWReferenceNode for storing the main content to the references list. When the reference list is not present on fresh articles, when it gets deleted or due to the use of a {{reflist}} template this leads to some problems.

[1]

<ref name="book" details="page 1">Miller</ref>

Solution

We identified the need to store references more independently of a synthetic MWReferenceNode. The reference nodes are very close to <ref> tags in Wikitext and will only appear once in the document when a ref or a main+details occurs. InternalItemNodes are the objects that store the content of references. So to solve the described problems we're working on changes that allow us to access two different InternalItemNodes from sub-references without the need that the main content is also associated with a distinct MWReferenceNode. It still can be for cases where the main ref is ( manually ) list defined or elsewhere reused in the article, but it does not has to.

Goals

Known use cases with issues

When using main+details with the {{reflist}} template or with a missing reference list a VE user can...

Publishing step:

Optional

VisualDiff:

Related Bugs

{{Reflist}}

T395083: VE: Cannot render main content from inline main+details when {{reflist}} is used

Deleted/Missing References List

T402900: Avoid orphaning subrefs when a reference list is deleted
T404297: Adding main+details references to an empty article leads to content loss

Other

T403573: Synthetic list defined refs can be orphaned

WIP

https://docs.google.com/document/d/13ZScENROJkVj1J-PJjF2qgVLtTg5TgEO7meguT9v724/edit?tab=t.q8g9ceit3aom#heading=h.dmy7uhvasd4o

Details

Related Changes in Gerrit:
Show related patches Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 DuplicateNoneT411856 [Epic] Avoid unintended content loss due the usage of synthetic ldr's
 ResolvedNoneT397501 [Epic] Fix issues with {{reflist}} or missing references lists and main+details sub-references
 DuplicateNoneT397211 [draft] VE should create a list-defined main ref when a regular main ref is deleted but still needed by a subref
 DeclinedNoneT397950 Store ref metadata in InternalItem so that we can recreate main refs when needed
 ResolvedawightT401313 Investigate whether InternalItemNodes can be used to recreate content of deleted nodes
 InvalidNoneT401314 Investigate if the list of InternalItemNodes is reliable when the document changes
 InvalidNoneT401329 Discuss whats missing in InteralList/InternalItemNode to retrieve delete ReferenceNode's content
 ResolvedawightT401333 Investigate: How to create a list defined ref from an InternalItemNode
 DuplicateNoneT406861 [refactor] Reference group can look up internal item node by key
 DeclinedNoneT406862 [refactor] Reference group can look up internal item node by key
 ResolvedawightT406871 [refactor] Stop emitting synthetic list-defined refs from Parsoid
 InvalidNoneT406858 [refactor] Find a way to create a unique reference list item id in VE
 DeclinedSpikeNoneT414340 Spike: Look into reflist template compatibility for the Parsoid "round-trip"
 InvalidNoneT417612 [Refactor] VE: Load main body content for sub-refs from DOM
 InvalidNoneT417878 [Refactor] VE: Let VE generate id's to match content with nodes
 ResolvedNoneT421055 Disable synthetic list-defined ref logic on the beta cluster
 ResolvedBUG REPORTTobi_WMDE_SWT404297 Adding main+details references to an empty article leads to content loss
 DuplicateNoneT406869 [Epic] VisualEditor must stop relying on synthetic LDRs to access main content
 DeclinedNoneT406588 [refactor] Don't remove internal item nodes from InternalList's indexes
 InvalidNoneT406570 [refactor] Parsoid must always emit data-mw for reference list items
 ResolvedNoneT408535 Look into the failing tests when rebuilding keyIndexes
 ResolvedNoneT411396 Paste regressions when rebuilding keyIndexes
 ResolvedNoneT415521 Display InternalList keyIndex mapping and InternalItem indexes in VE debug mode
 ResolvedNoneT415526 [Refactoring] Create a dev only feature flag that disables synthetic refs in Visual Editor
 ResolvedBUG REPORTNoneT402903 Custom groups are broken VE when no <references> tag is present with main+details
 ResolvedBUG REPORTTobi_WMDE_SWT403573 Synthetic list defined refs can be orphaned
 ResolvedNoneT406022 [Refactor] Allow editing main content in the ReferenceDialog of a sub-ref node without relying on a main node
 ResolvedNoneT421992 [Refactor] Write a new container for the reference edit target
 ResolvedNoneT415884 [Refactor] Replace MWReferenceNode.mainRefKey with an index to the main ref InternalItemNode
 ResolvedMareikeHeuerWMDET415523 [Refactor] Add helper method to retrieve InternalItems for content and use it in the ContextItem
 ResolvedNoneT415525 [Refactor] Show main content in the ReferencesList of a sub-ref node without relying on a main node
 ResolvedNoneT419849 [Refactor] Reimplement group references structure without relying on synthetic ref nodes or list key
 ResolvedNoneT421987 [Refactor] Consolidate numbering logic so that footnote markers and reference list use the same algorithm
 ResolvedNoneT415524 [Refactor] Show main content in the ReferenceSearchWidget of a sub-ref node without relying on a main node
 ResolvedNoneT416577 [Refactor] Rebuild methods in ve.dm.MWGroupReferences to work with `listIndex` instead of `listKey`
 ResolvedNoneT420971 [Refactor] Allow editing main content in the CitationDialog of a sub-ref node without relying on a main node
 ResolvedNoneT421248 [Refactor] Implement a method to find Citation commands from InternalItems
 ResolvedTobi_WMDE_SWT422932 [Refactor] Changed main content should be saved back to Wikitext sub-ref node without relying on a main node
 ResolvedTobi_WMDE_SWT423643 [Refactor] Make sure main content of main+details can be saved back independent of synth refs
 ResolvedTobi_WMDE_SWT423646 [Refactor] Changed main content should be saved back when a {{reflist}} template is used
 InvalidNoneT423647 [Refactor] Figure out a way to bring main content back to Parsoid when there's no references list node
 ResolvedTobi_WMDE_SWT424319 [Refactor] Wire the main content back using data-mw
 ResolvedNoneT423276 [Refactor] Make sure there's no dangling main content when deleting a sub-ref
 ResolvedNoneT423277 [Refactor] Make sure there's no orphaned sub-ref when deleting a main

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
WMDE-Fisch added a subtask: T404297: Adding main+details references to an empty article leads to content loss.Mar 24 2026, 5:30 PM
WMDE-Fisch added a subtask: T402903: Custom groups are broken VE when no <references> tag is present with main+details.
Maintenance_bot removed a project: Patch-For-Review.Mar 24 2026, 5:30 PM
WMDE-Fisch merged a task: T411856: [Epic] Avoid unintended content loss due the usage of synthetic ldr's.Mar 24 2026, 5:31 PM
WMDE-Fisch added a subscriber: Lina_Farid_WMDE.
WMDE-Fisch updated the task description. (Show Details)Mar 24 2026, 5:55 PM
WMDE-Fisch updated the task description. (Show Details)Mar 24 2026, 6:11 PM
WMDE-Fisch removed a subtask: T415760: [Epic] Implement a new mechanism that connects MWReferenceNodes with InternalItemNodes.Mar 24 2026, 6:14 PM
WMDE-Fisch mentioned this in T406869: [Epic] VisualEditor must stop relying on synthetic LDRs to access main content.Mar 24 2026, 6:18 PM
WMDE-Fisch merged a task: T406869: [Epic] VisualEditor must stop relying on synthetic LDRs to access main content.
WMDE-Fisch added a subtask: T403573: Synthetic list defined refs can be orphaned.
WMDE-Fisch added subscribers: Hakimi97, matmarex.
WMDE-Fisch updated the task description. (Show Details)Mar 24 2026, 6:26 PM
WMDE-Fisch renamed this task from [Epic] Fix known issues with the {{reflist}} template or missing references lists and main+details sub-references to [Epic] Fix issues with {{reflist}} or missing references lists and main+details sub-references.Mar 25 2026, 9:03 AM
lilients_WMDE added a project: WMDE-TechWish-Sprint-2026-03-17-all-of-the-beans.Mar 25 2026, 9:39 AM
lilients_WMDE moved this task from Sprint Backlog to Epics / Watching / Stalled on the WMDE-TechWish-Sprint-2026-03-17-all-of-the-beans board.
WMDE-Fisch added a subtask: T406022: [Refactor] Allow editing main content in the ReferenceDialog of a sub-ref node without relying on a main node.Mar 25 2026, 3:45 PM
Tobi_WMDE_SW mentioned this in WMDE-TechWish-Sprint-2026-03-17-all-of-the-beans.Mar 26 2026, 9:15 AM
Zamkorus subscribed.Mar 29 2026, 6:30 PM
WMDE-Fisch updated the task description. (Show Details)Apr 1 2026, 8:34 AM
WMDE-Fisch updated the task description. (Show Details)
WMDE-Fisch added a subtask: T415884: [Refactor] Replace MWReferenceNode.mainRefKey with an index to the main ref InternalItemNode.Apr 10 2026, 10:56 AM
WMDE-Fisch updated the task description. (Show Details)Apr 10 2026, 11:44 AM
WMDE-Fisch created subtask T422932: [Refactor] Changed main content should be saved back to Wikitext sub-ref node without relying on a main node.Apr 10 2026, 11:56 AM
WMDE-Fisch updated the task description. (Show Details)
WMDE-Fisch updated the task description. (Show Details)Apr 13 2026, 8:25 AM
thiemowmde closed subtask T421055: Disable synthetic list-defined ref logic on the beta cluster as Resolved.Apr 14 2026, 8:27 AM
awight updated the task description. (Show Details)Apr 14 2026, 8:37 AM
WMDE-Fisch mentioned this in WMDE-TechWish-Sprint-2026-04-14-Greeny-Zucchini.Apr 14 2026, 9:13 AM
WMDE-Fisch added a project: WMDE-TechWish-Sprint-2026-04-14-Greeny-Zucchini.Apr 14 2026, 9:17 AM
WMDE-Fisch moved this task from Sprint Backlog to Epics / Watching / Stalled on the WMDE-TechWish-Sprint-2026-04-14-Greeny-Zucchini board.
WMDE-Fisch created subtask T423276: [Refactor] Make sure there's no dangling main content when deleting a sub-ref.Apr 14 2026, 12:42 PM
WMDE-Fisch created subtask T423277: [Refactor] Make sure there's no orphaned sub-ref when deleting a main.Apr 14 2026, 12:47 PM
WMDE-Fisch updated the task description. (Show Details)
WMDE-Fisch updated the task description. (Show Details)Apr 17 2026, 9:33 AM
WMDE-Fisch updated the task description. (Show Details)Apr 17 2026, 11:21 AM
WMDE-Fisch mentioned this in T423906: Test various scenarios on how VE can be used with {{reflist}} and details.Apr 20 2026, 4:17 PM
zdev updated the task description. (Show Details)Apr 24 2026, 3:47 PM
WMDE-Fisch updated the task description. (Show Details)Apr 27 2026, 8:56 AM
WMDE-Fisch closed subtask T406858: [refactor] Find a way to create a unique reference list item id in VE as Invalid.
Tobi_WMDE_SW added a project: WMDE-TechWish-Sprint-2026-04-27-Multicolor-Potatoes.Apr 27 2026, 8:59 AM
Tobi_WMDE_SW moved this task from Sprint Backlog to Epics / Watching / Stalled on the WMDE-TechWish-Sprint-2026-04-27-Multicolor-Potatoes board.
gerritbot added a comment.Apr 27 2026, 2:12 PM

Change #1277560 had a related patch set uploaded (by Thiemo Kreuz (WMDE); author: Thiemo Kreuz (WMDE)):

[mediawiki/extensions/Cite@master] Reduce usages of MWDocumentReferences.static.refsForDoc

https://gerrit.wikimedia.org/r/1277560

gerritbot added a project: Patch-For-Review.Apr 27 2026, 2:12 PM
Johannes_Richter_WMDE subscribed.Apr 27 2026, 3:47 PM
Tobi_WMDE_SW mentioned this in WMDE-TechWish-Sprint-2026-04-27-Multicolor-Potatoes.Apr 28 2026, 4:46 AM
WMDE-Fisch mentioned this in T424659: [Epic] Code cleanup for sub-referencing.Apr 28 2026, 1:13 PM
Tobi_WMDE_SW closed subtask T423277: [Refactor] Make sure there's no orphaned sub-ref when deleting a main as Resolved.Apr 29 2026, 11:30 AM
Tobi_WMDE_SW closed subtask T423276: [Refactor] Make sure there's no dangling main content when deleting a sub-ref as Resolved.
Tobi_WMDE_SW closed subtask T406022: [Refactor] Allow editing main content in the ReferenceDialog of a sub-ref node without relying on a main node as Resolved.
gerritbot added a comment.May 11 2026, 4:05 PM

Change #1277560 merged by jenkins-bot:

[mediawiki/extensions/Cite@master] Reduce usages of MWDocumentReferences.static.refsForDoc

https://gerrit.wikimedia.org/r/1277560

Maintenance_bot removed a project: Patch-For-Review.May 11 2026, 4:31 PM
WMDE-Fisch closed subtask T415884: [Refactor] Replace MWReferenceNode.mainRefKey with an index to the main ref InternalItemNode as Resolved.May 11 2026, 5:10 PM
WMDE-Fisch closed subtask T417878: [Refactor] VE: Let VE generate id's to match content with nodes as Invalid.
WMDE-Fisch closed subtask T417612: [Refactor] VE: Load main body content for sub-refs from DOM as Invalid.May 11 2026, 5:39 PM
WMDE-Fisch merged a task: T397813: [Epic] Incompatibilities of Cite's details syntax and {{Reflist}}.May 11 2026, 5:42 PM
Tobi_WMDE_SW closed subtask T404297: Adding main+details references to an empty article leads to content loss as Resolved.May 12 2026, 8:59 AM
Tobi_WMDE_SW closed subtask T403573: Synthetic list defined refs can be orphaned as Resolved.
Tobi_WMDE_SW closed subtask T422932: [Refactor] Changed main content should be saved back to Wikitext sub-ref node without relying on a main node as Resolved.
WMDE-Fisch updated the task description. (Show Details)May 13 2026, 8:36 AM
WMDE-Fisch closed subtask T402903: Custom groups are broken VE when no <references> tag is present with main+details as Resolved.May 13 2026, 8:46 AM
RoySmith subscribed.May 20 2026, 1:21 PM
gerritbot added a comment.Jun 1 2026, 11:59 AM

Change #1180837 merged by jenkins-bot:

[VisualEditor/VisualEditor@master] Optimize/document ve.dm.InternalListNodeGroup implementation

https://gerrit.wikimedia.org/r/1180837

gerritbot added a comment.Jun 2 2026, 10:40 AM

Change #1170169 merged by jenkins-bot:

[VisualEditor/VisualEditor@master] Remove InternalList's dependency on document.buildingNodeTree

https://gerrit.wikimedia.org/r/1170169

gerritbot added a comment.Jun 5 2026, 3:05 PM

Change #1298305 had a related patch set uploaded (by Esanders; author: Esanders):

[mediawiki/extensions/VisualEditor@master] Update VE core submodule to master (f49818bd3)

https://gerrit.wikimedia.org/r/1298305

gerritbot added a project: Patch-For-Review.Jun 5 2026, 3:05 PM
gerritbot added a comment.Jun 5 2026, 3:26 PM

Change #1298305 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Update VE core submodule to master (f49818bd3)

https://gerrit.wikimedia.org/r/1298305

WMDE-Fisch removed a subtask: T402900: Avoid orphaning subrefs when a reference list is deleted.Jun 16 2026, 6:46 AM
WMDE-Fisch removed a subtask: T406866: [Epic] Stop creating new synthetic refs in VisualEditor.Jun 16 2026, 6:52 AM
WMDE-Fisch closed this task as Resolved.Jun 16 2026, 8:46 AM
WMDE-Fisch claimed this task.

Issues regarding the {{reflist}} template are resolved. So the epic is done. We're ready for deploying the feature to Wikis with {{reflist}}

Leftovers are tracked separately

WMDE-Fisch removed WMDE-Fisch as the assignee of this task.Jun 16 2026, 8:46 AM
gerritbot added a comment.Jun 16 2026, 7:46 PM

Change #1302948 had a related patch set uploaded (by DLynch; author: DLynch):

[VisualEditor/VisualEditor@master] Restore InternalList's dependency on document.buildingNodeTree

https://gerrit.wikimedia.org/r/1302948

gerritbot added a comment.Jun 16 2026, 8:11 PM

Change #1302948 merged by jenkins-bot:

[VisualEditor/VisualEditor@master] Restore InternalList's dependency on document.buildingNodeTree

https://gerrit.wikimedia.org/r/1302948

gerritbot added a comment.Jun 16 2026, 8:12 PM

Change #1302951 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/VisualEditor@master] Update VE core submodule to master (0930c3a9e)

https://gerrit.wikimedia.org/r/1302951

gerritbot added a comment.Jun 16 2026, 8:13 PM

Change #1302952 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/VisualEditor@wmf/1.47.0-wmf.7] Update VE core submodule to master (0930c3a9e)

https://gerrit.wikimedia.org/r/1302952

gerritbot added a comment.Jun 16 2026, 8:15 PM

Change #1302953 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/VisualEditor@wmf/1.47.0-wmf.6] Update VE core submodule to master (0930c3a9e)

https://gerrit.wikimedia.org/r/1302953

gerritbot added a comment.Jun 16 2026, 9:45 PM

Change #1302951 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Update VE core submodule to master (0930c3a9e)

https://gerrit.wikimedia.org/r/1302951

gerritbot added a comment.Jun 16 2026, 9:53 PM

Change #1302953 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@wmf/1.47.0-wmf.6] Update VE core submodule to master (0930c3a9e)

https://gerrit.wikimedia.org/r/1302953

gerritbot added a comment.Jun 16 2026, 9:58 PM

Change #1302952 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@wmf/1.47.0-wmf.7] Update VE core submodule to master (0930c3a9e)

https://gerrit.wikimedia.org/r/1302952

Stashbot mentioned this in T429355: TypeError: can't access property "length", data is null when starting ve on specific enwiki page.Jun 16 2026, 9:59 PM

Mentioned in SAL (#wikimedia-operations) [2026-06-16T21:58:59Z] <kemayo@deploy1003> Started scap sync-world: Backport for [[gerrit:1302953|Update VE core submodule to master (0930c3a9e) (T406841 T429174 T397501 T424632 T429355)]], [[gerrit:1302952|Update VE core submodule to master (0930c3a9e) (T397501 T424632 T429355)]]

Stashbot mentioned this in T429174: Data model corruption when editing a specific page.Jun 16 2026, 9:59 PM
Stashbot mentioned this in T406841: Detect large IME insertions as pastes.
Stashbot mentioned this in T424632: Remove unused code from Cite and VE after refactoring.
Stashbot added a comment.Jun 16 2026, 10:01 PM

Mentioned in SAL (#wikimedia-operations) [2026-06-16T22:01:11Z] <kemayo@deploy1003> kemayo: Backport for [[gerrit:1302953|Update VE core submodule to master (0930c3a9e) (T406841 T429174 T397501 T424632 T429355)]], [[gerrit:1302952|Update VE core submodule to master (0930c3a9e) (T397501 T424632 T429355)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Stashbot added a comment.Jun 16 2026, 10:07 PM

Mentioned in SAL (#wikimedia-operations) [2026-06-16T22:07:10Z] <kemayo@deploy1003> Finished scap sync-world: Backport for [[gerrit:1302953|Update VE core submodule to master (0930c3a9e) (T406841 T429174 T397501 T424632 T429355)]], [[gerrit:1302952|Update VE core submodule to master (0930c3a9e) (T397501 T424632 T429355)]] (duration: 08m 11s)

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits